Package-level declarations
Types
Link copied to clipboard
class Lexer(ignorePattern: Regex = Regex("""\s+"""), singleLineComments: Regex? = null, multilineComments: Pair<Regex, Regex>? = null, identifiers: Regex = Regex("""[a-zA-Z_]\w*"""), hardKeywords: Set<String> = emptySet(), operators: Set<String> = emptySet(), separators: Set<String> = emptySet(), literals: Literals = Literals())
The lexer is responsible to convert the given string into a stream of Tokens. The lexer take in multiple settings that configure how it behaves. It will perform lexical analysis on a line-by-line basis and return the next unconsumed token. A newline character is always separates a token.
Link copied to clipboard
class Literals(integerLiteral: Regex? = Regex("""[-+]?[0-9_]+"""), floatingLiteral: Regex? = Regex("""[-+]?[0-9_]*\.[0-9_]+(?:[eE][-+]?[0-9_]+)?"""), singleLineString: Set<String> = setOf("\"", "\'"), escapeSequences: List<Pair<Regex, (String) -> Char>> = emptyList())
A configuration class to lex literals. There are only three types of literals the lexer manages.