Parsing and the AST

Tokens are flat; programs are nested. The parser builds that nesting: multiplication belongs inside a binding, the binding belongs inside a function, and the indented returns belong to different branches.

The AST preserves what was written

The nodes are arena-owned, untyped, and source-spanned. A call node still carries the written positional and named arguments. A class node still carries fields, methods, init, and deinit. The parser does not know whether any name resolves or any call matches.

How Pratt parsing handles expressions

1. Read a prefixA literal, name, grouped expression, lambda, unary operator, or collection literal starts the left side.
2. Read postfix formsCalls, field access, indexing, and slicing bind most tightly and extend the current expression.
3. Compare binding powerAn infix operator joins only when its precedence is high enough for the current parse.
4. Recurse for the right sideAssociativity becomes the next minimum binding power, not another grammar function.

Two places the grammar refuses to guess

Luce rejects a chained comparison and rejects not directly in front of a comparison. Python-like and C-like readers can assign different meanings to those shapes while both parses remain plausible. Parentheses make the intended tree visible.

Recovery follows the reader

A broken construct reports at the offending token, then resumes at the next line in the same block. If a broken header owns an indented suite, recovery swallows that orphaned body rather than reinterpreting it one level out. An unclosed list blames its opening bracket, and a missing comma names the item it separates.

Some unambiguous mistakes are repaired only in the recovery stream: if x = 1: can be read as the intended comparison so the body is still checked, while the diagnostic tells the author to write ==. The compiler still returns failure; recovery exists to improve the rest of the report, not to run corrected code.

The small amount of early sugar

Most source sugar survives into typed HIR. Two historical exceptions are expanded while syntax is still untyped: an f-string becomes concatenation through str(...), and an elif chain becomes nested conditionals. The parser contract calls this a wart because lowering sugar belongs after meaning is settled.

Robustness is part of the grammar

Statements, conditions, expression nesting, prefix chains, and type arguments share a recursion budget. Diagnostic volume is capped. A lexer that truncated its stream after a structural bound causes the parser to stay silent about the synthetic tail. Production parsing includes how malformed input ends, not only how valid input begins.