new Lexer(input)
General lexer
Transforms an input string into a stream of tokens
Parameters:
| Name | Type | Description |
|---|---|---|
input |
string | The string to transform |
- Source:
Methods
next() → {Parser.Token}
Get the next token and advance the input position
- Source:
Returns:
the next token from the input stream,
or an end-of-string token if the end of the string has been reached
- Type
- Parser.Token