summaryrefslogtreecommitdiffhomepage
path: root/docs
diff options
context:
space:
mode:
authorJo-Philipp Wich <jo@mein.io>2023-07-12 16:45:12 +0200
committerJo-Philipp Wich <jo@mein.io>2023-07-13 10:41:24 +0200
commitf1190ef2f2719582efcad4109f1215b02baebfdf (patch)
treed9a2644de3ba6c1032a4a3b5643fa78b26a84aa8 /docs
parentcba0c3cf5312cdaed0538bb3dbf87b31b3e89916 (diff)
docs: various improvements
- Switch JSDoc theme to "clean-jsdoc-theme" - Add some custom CSS and JS tweaks to the theme - Use a condensed README.md for the toplevel directory - Include a longer README.md in the documentation portal - Tweak JSDoc annotations for better output results - Register `ucode.mein.io` CNAME Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Diffstat (limited to 'docs')
-rw-r--r--docs/CNAME1
-rw-r--r--docs/README.md660
-rw-r--r--docs/ucode.css20
-rw-r--r--docs/ucode.js8
4 files changed, 686 insertions, 3 deletions
diff --git a/docs/CNAME b/docs/CNAME
new file mode 100644
index 0000000..deb43a3
--- /dev/null
+++ b/docs/CNAME
@@ -0,0 +1 @@
+ucode.mein.io
diff --git a/docs/README.md b/docs/README.md
index 17fffbe..660f175 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -1,5 +1,659 @@
-# ucode Documentation
+# The ucode Scripting Language
-## Module Reference
+The ucode language is a tiny general purpose scripting language featuring a
+syntax closely resembling ECMAScript. It can be used in a stand-alone manner
+by using the ucode command line interpreter or embedded into host applications
+by linking libucode and utilizing its C language API. Additionally, ucode can
+be invoked in template mode where control flow and expression logic statements
+are embedded in Jinja-like markup blocks.
- - [ucode module documentation](index.html)
+Besides aiming for small size, the major design goals of ucode are the ability
+to trivially read and write JSON data, good embeddability into C applications,
+template capabilities for output formatting, extensiblity through loadable
+native extension modules and a straightforward set of built-in functions
+mimicking those found in the Perl 5 language.
+
+## History and Motivation
+
+In spring 2021 it has been decided to rewrite the OpenWrt firewall framework on
+top of nftables with the goal to replace the then current C application with a
+kind of preprocessor generating nftables rulesets using a set of templates
+instead of relying on built-in hardcoded rules like its predecessor.
+
+That decision spurred the development of *ucode*, initially meant to be a
+simple template processor solely for the OpenWrt nftables firewall but quickly
+evolving into a general purpose scripting language suitable for a wider range
+of system scripting tasks.
+
+Despite OpenWrt predominantly relying on POSIX shell and Lua as system
+scripting languages already, a new solution was needed to accomodate the needs
+of the new firewall implementation; mainly the ability to efficiently deal with
+JSON data and complex data structures such as arrays and dictionaries and the
+ability to closely interface with OpenWrt's *ubus* message bus system.
+
+Throughout the design process of the new firewall and its template processor,
+the following design goals were defined for the *ucode* scripting language:
+
+ - Ability to embed code logic fragments such as control flow statements,
+ function calls or arithmetic expressions into plain text templates, using
+ a block syntax and functionality roughly inspired by Jinja templates
+ - Built-in support for JSON data parsing and serialization, without the need
+ for external libraries
+ - Distinct array and object types (compared to Lua's single table datatype)
+ - Distinct integer and float types and guaranteed 64bit integer range
+ - Built-in support for bit operations
+ - Built-in support for (POSIX) regular expressions
+ - A comprehensive set of built-in standard functions, inspired by the core
+ functions found in the Perl 5 interpreter
+ - Staying as close to ECMAScript syntax as possible due to higher developer
+ familiarity and to be able to reuse existing tooling such as editor syntax
+ highlighting
+ - Bindings for all relevant Linux and OpenWrt APIs, such as *ubus*, *uci*,
+ *uloop*, *netlink* etc.
+ - Procedural, synchronous programming flow
+ - Very small executable size (the interpreter and runtime is currently around
+ 64KB on ARM Cortex A9)
+ - Embeddability into C host applications
+
+Summarized, *ucode* can be described as synchronous ECMAScript without the
+object oriented standard library.
+
+
+## Installation
+
+### OpenWrt
+
+In OpenWrt 22.03 and later, *ucode* should already be preinstalled. If not,
+it can be installed via the package manager, using the `opkg install ucode`
+command.
+
+### MacOS
+
+To build on MacOS, first install *cmake* and *json-c* via
+[Homebrew](https://brew.sh/), then clone the ucode repository and execute
+*cmake* followed by *make*:
+
+ $ brew install cmake json-c
+ $ git clone https://github.com/jow-/ucode.git
+ $ cd ucode/
+ $ cmake -DUBUS_SUPPORT=OFF -DUCI_SUPPORT=OFF -DULOOP_SUPPORT=OFF .
+ $ make
+ $ sudo make install
+
+### Debian
+
+The ucode repository contains build recipes for Debian packages, to build .deb
+packages for local installation, first install required development packages,
+then clone the repository and invoke *dpkg-buildpackage* to produce the binary
+package files:
+
+ $ sudo apt-get install build-essential devscripts debhelper libjson-c-dev cmake pkg-config
+ $ git clone https://github.com/jow-/ucode.git
+ $ cd ucode/
+ $ dpkg-buildpackage -b -us -uc
+ $ sudo dpkg -i ../ucode*.deb ../libucode*.deb
+
+### Other Linux systems
+
+To install ucode from source on other systems, ensure that the json-c library
+and associated development headers are installed, then clone and compile the
+ucode repository:
+
+ $ git clone https://github.com/jow-/ucode.git
+ $ cd ucode/
+ $ cmake -DUBUS_SUPPORT=OFF -DUCI_SUPPORT=OFF -DULOOP_SUPPORT=OFF .
+ $ make
+ $ sudo make install
+
+
+## Syntax
+
+### Template mode
+
+By default, *ucode* is executed in *raw mode*, means it expects a given source
+file to only contain script code. By invoking the ucode interpreter with the
+`-T` flag or by using the `utpl` alias, the *ucode* interpreter is switched
+into *template mode* where the source file is expected to be a plaintext file
+containing *template blocks* containing ucode script expressions or comments.
+
+#### Block types
+
+There are three kinds of blocks; *expression blocks*, *statement blocks* and
+*comment blocks*. The former two embed code logic using ucode's JavaScript-like
+syntax while the latter comment block type is simply discarded during
+processing.
+
+
+##### 1. Statement block
+
+Statement blocks are enclosed in an opening `{%` and a closing `%}` tag and
+may contain any number of script code statements, even entire programs.
+
+It is allowed to omit the closing `%}` of a statement block to parse the
+entire remaining source text after the opening tag as ucode script.
+
+By default, statement blocks produce no output and the entire block is
+reduced to an empty string during template evaluation but contained script
+code might invoke functions such as `print()` to explicitly output contents.
+
+For example the following template would result in `The epoch is odd` or
+`The epoch is even`, depending on the current epoch value:
+
+`The epoch is {% if (time() % 2): %}odd{% else %}even{% endif %}!`
+
+
+##### 2. Expression block
+
+Expression blocks are enclosed in an opening `{{` and a closing `}}` tag and
+may only contain a single expression statement (multiple expressions may be
+chained with comma). The implicit result of the rightmost evaluated expression
+is used as output when processing the block.
+
+For example the template `Hello world, {{ getenv("USER") }}!` would result in
+the output "Hello world, user!" where `user` would correspond to the name of
+the current user executing the ucode interpreter.
+
+
+##### 3. Comment block
+
+Comment blocks, which are denoted with an opening `{#` and a closing `#}` tag
+may contain arbitrary text except the closing `#}` tag itself. Comments blocks
+are completely stripped during processing and are replaced with an empty string.
+
+The following example template would result in the output "Hello world":
+
+`Hello {# mad #}word`
+
+
+#### Whitespace handling
+
+Each block start tag may be suffixed with a dash to strip any whitespace
+before the block and likewise any block end tag may be prefixed with a dash
+to strip any whitespace following the block.
+
+Without using whitespace stripping, the following example:
+
+```
+This is a first line
+{% for (x in [1, 2, 3]): %}
+This is item {{ x }}.
+{% endfor %}
+This is the last line
+```
+
+Would result in the following output:
+
+```
+This is a first line
+
+This is item 1.
+This is item 2.
+This is item 3.
+
+This is the last line
+```
+
+By adding a trailing dash to apply whitespace stripping after the block, the
+empty lines can be eliminated:
+
+```
+This is a first line
+{% for (x in [1, 2, 3]): -%}
+This is item {{ x }}.
+{% endfor -%}
+This is the last line
+```
+
+Output:
+
+```
+This is a first line
+This is item 1.
+This is item 2.
+This is item 3.
+This is the last line
+```
+
+By applying whitespace stripping before the block, all lines can be joined
+into a single output line:
+
+```
+This is a first line
+{%- for (x in [1, 2, 3]): -%}
+This is item {{ x }}.
+{%- endfor -%}
+This is the last line
+```
+
+Output:
+
+```
+This is a first lineThis is item 1.This is item 2.This is item 3.This is the last line
+```
+
+### Script syntax
+
+The ucode script language - used either within statement and expression blocks
+or throughout the entire file in *raw mode*, uses untyped variables and employs
+a simplified JavaScript like syntax.
+
+The language implements function scoping and differentiates between local and
+global variables. Each function has its own private scope while executing and
+local variables declared inside a function are not accessible in the outer
+calling scope.
+
+#### 1. Data types
+
+Ucode supports seven different basic types as well as two additional special
+types; function values and ressource values. The supported types are:
+
+ - Boolean values (`true` or `false`)
+ - Integer values (`-9223372036854775808` to `+9223372036854775807`)
+ - Double values (`-1.7e308` to `+1.7e308`)
+ - String values (e.g. `'Hello world!'` or `"Sunshine \u2600!"`)
+ - Array values (e.g. `[1, false, "foo"]`)
+ - Object values (e.g. `{ foo: true, "bar": 123 }`)
+ - Null value (`null`)
+
+Ucode utilizes reference counting to manage memory used for variables and values
+and frees data automatically as soon as values go out of scope.
+
+Numeric values are either stored as signed 64bit integers or as IEEE 756 double
+value. Conversion between integer and double values can happen implicitly, e.g.
+through numeric operations, or explicitely, e.g. by invoking functions such as
+`int()`.
+
+#### 2. Variables
+
+Variable names must start with a letter or an underscore and may only contain
+the characters `A`..`Z`, `a`..`z`, `0`..`9` or `_`. By prefixing a variable
+name with the keyword `let`, it is declared in the local block scope only
+and not visible outside anymore.
+
+Variables may also be declared using the `const` keyword. Such variables follow
+the same scoping rules as `let` declared ones but they cannot be modified after
+they have been declared. Any attempt to do so will result in a syntax error
+during compilation.
+
+```javascript
+{%
+
+ a = 1; // global variable assignment
+
+ function test() {
+ let b = 2; // declare `b` as local variable
+ a = 2; // overwrite global a
+ }
+
+ test();
+
+ print(a, "\n"); // outputs "2"
+ print(b, "\n"); // outputs nothing
+
+ const c = 3;
+ print(c, "\n"); // outputs "3"
+
+ c = 4; // raises syntax error
+ c++; // raises syntax error
+
+ const d; // raises syntax error, const variables must
+ // be initialized at declaration time
+
+%}
+```
+
+#### 3. Control statements
+
+Similar to JavaScript, ucode supports `if`, `for` and `while` statements to
+control execution flow.
+
+##### 3.1. Conditional statement
+
+If/else blocks can be used to execute statements depending on a condition.
+
+```javascript
+{%
+
+ user = getenv("USER");
+
+ if (user == "alice") {
+ print("Hello Alice!\n");
+ }
+ else if (user == "bob") {
+ print("Hello Bob!\n");
+ }
+ else {
+ print("Hello guest!\n");
+ }
+
+%}
+```
+
+If only a single statement is wrapped by an if or else branch, the enclosing
+curly braces may be omitted:
+
+```javascript
+{%
+
+ if (rand() == 3)
+ print("This is quite unlikely\n");
+
+%}
+```
+
+##### 3.2. Loop statements
+
+Ucode script supports three different flavors of loop control statements; a
+`while` loop that executes enclosed statements as long as the loop condition is
+fulfilled, a `for in` loop that iterates keys of objects or items of arrays and
+a counting `for` loop that is a variation of the `while` loop.
+
+```javascript
+{%
+
+ i = 0;
+ arr = [1, 2, 3];
+ obj = { Alice: 32, Bob: 54 };
+
+ // execute as long as condition is true
+ while (i < length(arr)) {
+ print(arr[i], "\n");
+ i++;
+ }
+
+ // execute for each item in arr
+ for (n in arr) {
+ print(n, "\n");
+ }
+
+ // execute for each key in obj
+ for (person in obj) {
+ print(person, " is ", obj[person], " years old.\n");
+ }
+
+ // execute initialization statement (j = 0) once
+ // execute as long as condition (j < length(arr)) is true
+ // execute step statement (j++) after each iteration
+ for (j = 0; j < length(arr); j++) {
+ print(arr[j], "\n");
+ }
+
+%}
+```
+
+##### 3.3. Alternative syntax
+
+Since conditional statements and loops are often used for template formatting
+purposes, e.g. to repeat a specific markup for each item of a list, ucode
+supports an alternative syntax that does not require curly braces to group
+statements but that uses explicit end keywords to denote the end of the control
+statement body for better readability instead.
+
+The following two examples first illustrate the normal syntax, followed by the
+alternative syntax that is more suitable for statement blocks:
+
+```
+Printing a list:
+{% for (n in [1, 2, 3]) { -%}
+ - Item #{{ n }}
+{% } %}
+```
+
+The alternative syntax replaces the opening curly brace (`{`) with a colon
+(`:`) and the closing curly brace (`}`) with an explicit `endfor` keyword:
+
+```
+Printing a list:
+{% for (n in [1, 2, 3]): -%}
+ - Item #{{ n }}
+{% endfor %}
+```
+
+For each control statement type, a corresponding alternative end keyword is defined:
+
+ - `if (...): ... endif`
+ - `for (...): ... endfor`
+ - `while (...): ... endwhile`
+
+
+#### 4. Functions
+
+Ucode scripts may define functions to group repeating operations into reusable
+operations. Functions can be both declared with a name, in which case they're
+automatically registered in the current scope, or anonymously which allows
+assigning the resulting value to a variable, e.g. to build arrays or objects of
+functions:
+
+```javascript
+{%
+
+ function duplicate(n) {
+ return n * 2;
+ }
+
+ let utilities = {
+ concat: function(a, b) {
+ return "" + a + b;
+ },
+ greeting: function() {
+ return "Hello, " + getenv("USER") + "!";
+ }
+ };
+
+-%}
+
+The duplicate of 2 is {{ duplicate(2) }}.
+The concatenation of 'abc' and 123 is {{ utilities.concat("abc", 123) }}.
+Your personal greeting is: {{ utilities.greeting() }}.
+```
+
+##### 4.1. Alternative syntax
+
+Function declarations support the same kind of alternative syntax as defined
+for control statements (3.3.)
+
+The alternative syntax replaces the opening curly brace (`{`) with a colon
+(`:`) and the closing curly brace (`}`) with an explicit `endfunction`
+keyword:
+
+```
+{% function printgreeting(name): -%}
+ Hallo {{ name }}, nice to meet you.
+{% endfunction -%}
+
+<h1>{{ printgreeting("Alice") }}</h1>
+```
+
+
+#### 5. Operators
+
+Similar to JavaScript and C, ucode scripts support a range of different
+operators to manipulate values and variables.
+
+##### 5.1. Arithmetic operations
+
+The operators `+`, `-`, `*`, `/`, `%`, `++` and `--` allow to perform
+additions, substractions, multiplications, divisions, modulo, increment or
+decrement operations respectively where the result depends on the type of
+involved values.
+
+The `++` and `--` operators are unary, means that they only apply to one
+operand. The `+` and `-` operators may be used in unary context to either
+convert a given value to a numeric value or to negate a given value.
+
+If either operand of the `+` operator is a string, the other one is converted
+to a string value as well and a concatenated string is returned.
+
+All other arithmetic operators coerce their operands into numeric values.
+Fractional values are converted to doubles, other numeric values to integers.
+
+If either operand is a double, the other one is converted to a double value as
+well and a double result is returned.
+
+Divisions by zero result in the special double value `Infinity`. If an operand
+cannot be converted to a numeric value, the result of the operation is the
+special double value `NaN`.
+
+```javascript
+{%
+ a = 2;
+ b = 5.2;
+ s1 = "125";
+ s2 = "Hello world";
+
+ print(+s1); // 125
+ print(+s2); // NaN
+ print(-s1); // -125
+ print(-s2); // NaN
+ print(-a); // -2
+
+ print(a++); // 2 (Return value of a, then increment by 1)
+ print(++a); // 4 (Increment by 1, then return value of a)
+
+ print(b--); // 5.2 (Return value of b, then decrement by 1)
+ print(--b); // 3.2 (Decrement by 1, then return value of b)
+
+ print(4 + 8); // 12
+ print(7 - 4); // 3
+ print(3 * 3); // 9
+
+ print(10 / 4); // 2 (Integer division)
+ print(10 / 4.0); // 2.5 (Double division)
+ print(10 / 0); // Infinity
+
+ print(10 % 7); // 3
+ print(10 % 7.0); // NaN (Modulo is undefined for non-integers)
+%}
+```
+
+##### 5.2. Bitwise operations
+
+The operators `&`, `|`, `^`, `<<`, `>>` and `~` allow to perform bitwise and,
+or, xor, left shift, right shift and complement operations respectively.
+
+The `~` operator is unary, means that is only applies to one operand.
+
+```javascript
+{%
+ print(0 & 0, 0 & 1, 1 & 1); // 001
+ print(0 | 0, 0 | 1, 1 | 1); // 011
+ print(0 ^ 0, 0 ^ 1, 1 ^ 1); // 010
+ print(10 << 2); // 40
+ print(10 >> 2); // 2
+ print(~15); // -16 (0xFFFFFFFFFFFFFFF0)
+%}
+```
+
+An important property of bitwise operators is that they're coercing their
+operand values to whole integers:
+
+```javascript
+{%
+ print(12.34 >> 0); // 12
+ print(~(~12.34)); // 12
+%}
+```
+
+##### 5.3. Relational operations
+
+The operators `==`, `!=`, `<`, `<=`, `>` and `>=` test whether their operands
+are equal, inequal, lower than, lower than/equal to, higher than or higher
+than/equal to each other respectively.
+
+If both operands are strings, their respective byte values are compared, if
+both are objects or arrays, their underlying memory addresses are compared.
+
+In all other cases, both operands are coerced into numeric values and the
+resulting values are compared with each other.
+
+This means that comparing values of different types will coerce them both to
+numbers.
+
+The result of the relational operation is a boolean indicating truishness.
+
+```javascript
+{%
+ print(123 == 123); // true
+ print(123 == "123"); // true!
+ print(123 < 456); // true
+ print(123 > 456); // false
+ print(123 != 456); // true
+ print(123 != "123"); // false!
+ print({} == {}); // false (two different anonymous objects)
+ a = {}; print(a == a); // true (same object)
+%}
+```
+
+##### 5.4. Logical operations
+
+The operators `&&`, `||`, `??` and `!` test whether their operands are all true,
+partially true, null or false respectively.
+
+In the case of `&&` the rightmost value is returned while `||` results in the
+first truish and `??` in the first non-null value.
+
+The unary `!` operator will result in `true` if the operand is not trueish,
+otherwise it will result in `false`.
+
+Operands are evaluated from left to right while testing truishness, which means
+that expressions with side effects, such as function calls, are only executed
+if the preceeding condition was satisifed.
+
+```javascript
+{%
+ print(1 && 2 && 3); // 3
+ print(1 || 2 || 3); // 1
+ print(2 > 1 && 3 < 4); // true
+ print(doesnotexist ?? null ?? 42); // 42
+ print(1 ?? 2 ?? 3); // 1
+ print(!false); // true
+ print(!true); // false
+
+ res = test1() && test2(); // test2() is only called if test1() returns true
+%}
+```
+
+##### 5.5. Assignment operations
+
+In addition to the basic assignment operator `=`, most other operators have a
+corresponding shortcut assignment operator which reads the specified variable,
+applies the operation and operand to it, and writes it back.
+
+The result of assignment expressions is the assigned value.
+
+```javascript
+{%
+ a = 1; // assign 1 to variable a
+ a += 2; // a = a + 2;
+ a -= 3; // a = a - 3;
+ a *= 4; // a = a * 4;
+ a /= 5; // a = a / 5;
+ a %= 6; // a = a % 6;
+ a &= 7; // a = a & 7;
+ a |= 8; // a = a | 8;
+ a ^= 9; // a = a ^ 9;
+ a <<= 10; // a = a << 10;
+ a >>= 11; // a = a >> 11;
+ a &&= 12; // a = a && 12;
+ a ||= 13; // a = a || 13;
+ a ??= 14; // a = a ?? 14;
+
+ print(a = 2); // 2
+%}
+```
+
+##### 5.6. Miscellaneous operators
+
+Besides the operators described so far, ucode script also supports a `delete`
+operator which removes a property from an object value.
+
+```javascript
+{%
+ a = { test: true };
+
+ delete a.test; // true
+ delete a.notexisting; // false
+
+ print(a); // { }
+%}
+```
diff --git a/docs/ucode.css b/docs/ucode.css
new file mode 100644
index 0000000..a40f478
--- /dev/null
+++ b/docs/ucode.css
@@ -0,0 +1,20 @@
+#sidebar > .sidebar-items-container {
+ margin-top: 1rem;
+}
+
+h1 {
+ display: flex;
+}
+
+article .description > ul,
+article .description > ol {
+ list-style: initial;
+}
+
+.navbar-item.github-home a {
+ background-image: url("data:image/svg+xml,%3Csvg width='98' height='96' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath fill-rule='evenodd' clip-rule='evenodd' d='M48.854 0C21.839 0 0 22 0 49.217c0 21.756 13.993 40.172 33.405 46.69 2.427.49 3.316-1.059 3.316-2.362 0-1.141-.08-5.052-.08-9.127-13.59 2.934-16.42-5.867-16.42-5.867-2.184-5.704-5.42-7.17-5.42-7.17-4.448-3.015.324-3.015.324-3.015 4.934.326 7.523 5.052 7.523 5.052 4.367 7.496 11.404 5.378 14.235 4.074.404-3.178 1.699-5.378 3.074-6.6-10.839-1.141-22.243-5.378-22.243-24.283 0-5.378 1.94-9.778 5.014-13.2-.485-1.222-2.184-6.275.486-13.038 0 0 4.125-1.304 13.426 5.052a46.97 46.97 0 0 1 12.214-1.63c4.125 0 8.33.571 12.213 1.63 9.302-6.356 13.427-5.052 13.427-5.052 2.67 6.763.97 11.816.485 13.038 3.155 3.422 5.015 7.822 5.015 13.2 0 18.905-11.404 23.06-22.324 24.283 1.78 1.548 3.316 4.481 3.316 9.126 0 6.6-.08 11.897-.08 13.526 0 1.304.89 2.853 3.316 2.364 19.412-6.52 33.405-24.935 33.405-46.691C97.707 22 75.788 0 48.854 0z' fill='%2324292f'/%3E%3C/svg%3E");
+ background-repeat: no-repeat;
+ background-size: 1.5em;
+ background-position: 1rem center;
+ padding-left: 3rem;
+}
diff --git a/docs/ucode.js b/docs/ucode.js
new file mode 100644
index 0000000..35d2d23
--- /dev/null
+++ b/docs/ucode.js
@@ -0,0 +1,8 @@
+document.addEventListener('DOMContentLoaded', (ev) => {
+ const accordionState = window.localStorage.getItem('accordion-id');
+
+ if (accordionState == null || accordionState == '{}')
+ document
+ .querySelector('[data-isopen="false"]')
+ .setAttribute('data-isopen', 'true');
+});