summaryrefslogtreecommitdiffhomepage
path: root/lexer.h
AgeCommit message (Collapse)Author
2021-07-09lexer: rename UT_ prefixed constants to UC_Jo-Philipp Wich
This is a cosmetic change to bring the code in line with the common prefix format of the other code in the tree. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2021-06-08lexer, vm: reorder token and instruction numbersJo-Philipp Wich
Ensure that relational operators in the lexer token and vm instruction lists are both ordered in the same way according to the general operator precedence of the grammar. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2021-05-25lexer, compiler: separate TK_BOOL token into TK_TRUE and TK_FALSE tokensJo-Philipp Wich
The token type split allows us to drop the token value union in the reserved word list with a subsequent commit. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2021-05-18syntax: introduce `const` supportJo-Philipp Wich
Introduce support for declaring constant variables through the `const` keyword. Variables declared with `const` follow the same scoping rules as `let` declared ones. In contrast to normal variables, `const` ones may not be assigned to after their declaration. Any attempt to do so will result in a syntax error during compilation. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2021-05-18syntax: implement `delete` as proper operatorJo-Philipp Wich
Turn `delete` into a proper operator mimicking ECMAScript semantics. Also ensure to transparently turn deprecated `delete(obj, propname)` function calls into `delete obj.propname` expressions during compilation. When strict mode is active, legacy delete() calls throw a syntax error instead. Finally drop the `delete()` function from the stdlib as it is shadowed by the delete operator syntax now. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2021-04-29compiler, lexer: improve lexical state handlingJo-Philipp Wich
- Instead of disambiguating division operator vs. regexp literal by looking at the preceeding token, raise a "no regexp" flag within the appropriate parser states to tell the lexer how to treat a forward slash when parsing the next token - Introduce another "no keyword" flag which disables parsing labels into keywords when reading the next token and set it in the appropriate parser states. This allows using reserved names in object declarations and property access expressions Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2021-04-27treewide: ISO C / pedantic complianceJo-Philipp Wich
- Shuffle typedefs to avoid need for non-compliant forward declarations - Fix non-compliant empty struct initializers - Remove use of braced expressions - Remove use of anonymous unions - Avoid `void *` pointer arithmetic - Fix several warnings reported by gcc -pedantic mode and clang 11 compilation Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2021-04-25treewide: rework internal data type systemJo-Philipp Wich
Instead of relying on json_object values internally, use custom types to represent the different ucode value types which brings a number of advantages compared to the previous approach: - Due to the use of tagged pointers, small integer, string and bool values can be stored directly in the pointer addresses, vastly reducing required heap memory - Ability to create circular data structures such as `let o; o = { test: o };` - Ability to register custom `tostring()` function through prototypes - Initial mark/sweep GC implementation to tear down circular object graphs on VM deinit The change also paves the way for possible future extensions such as constant variables and meta methods for custom ressource types. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2021-02-26lexer: improvementsJo-Philipp Wich
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2021-02-17treewide: rewrite ucode interpreterJo-Philipp Wich
Replace the former AST walking interpreter implementation with a single pass bytecode compiler and a corresponding virtual machine. The rewrite lays the groundwork for a couple of improvements with will be subsequently implemented: - Ability to precompile ucode sources into binary byte code - Strippable debug information - Reduced runtime memory usage Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-11-19treewide: rebrand to ucodeJo-Philipp Wich
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-11-05syntax: implement ES6-like arrow function syntaxJo-Philipp Wich
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-11-03syntax: implement ES6-like rest parameters for variadic functionsJo-Philipp Wich
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-11-02syntax: support `elif` clauses for alternative `if` syntaxJo-Philipp Wich
In the alternative `if` syntax mode, support a specific `elif` keyword instead of requiring an `else` branch followed by a disjunct `if` statement. The advantage is that templates do not require error-prone redundant `endif` keywords in else-if ladders. After this change, the following example: {% if (...): %} One condition {% else if (...): %} Another condition {% else if (...): %} A third condition {% else %} Final condition {% endif; endif; endif %} ... can be simplified into: {% if (...): %} One condition {% elif (...): %} Another condition {% elif (...): %} A third condition {% else %} Final condition {% endif %} Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-10-14lexer: rewriteJo-Philipp Wich
Rewrite the lexer into a restartable state machine to support parsing from file streams without the need to read the entire source text into memory first. As a side effect, the length of labels and strings is unlimited now. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-09-24syntax: add regular expression supportJo-Philipp Wich
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-09-22syntax: introduce case statement supportJo-Philipp Wich
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-09-21syntax: introduce try/catch blocksJo-Philipp Wich
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-09-11syntax: introduce !== and === operatorsJo-Philipp Wich
Also treat "in" as relational operator. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-09-06treewide: refactor internal AST structuresJo-Philipp Wich
- unify operand and value tag structures - use a contiguous array for storing opcodes - use relative offsets for next and children ops - defer function creation to runtime - rework "this" context handling by storing context pointer in scope tags Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-08-25treewide: add proper null value handlingJo-Philipp Wich
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-08-25treewide: introduce this keywordJo-Philipp Wich
Support a new keyword `this` which allows functions to access the context they're called upon. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-08-25lexer.h: add T_RESSOURCE op typeJo-Philipp Wich
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-08-25treewide: add dedicated T_CFUNC typeJo-Philipp Wich
Tag C function values with a dedicated T_CFUNC type to clearly differentiate them from runtime declared functions. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-08-25lexer.c, eval.c: move T_EXCEPTION definition to lexer headerJo-Philipp Wich
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2020-08-21Initial commitJo-Philipp Wich
Signed-off-by: Jo-Philipp Wich <jo@mein.io>