Lexing and layout
The lexer is the first simplifier. It turns characters into a flat row of meaningful pieces: func, a name, parentheses, a colon, a newline, and an indent. It does not decide whether the program makes sense.
From source to layout tokens
func main():
let answer = 40 + 2
print(str(answer))The colon announces a possible suite. Exactly four additional spaces open it. Both body statements remain at that level. End-of-file closes the open suite with dedent.
Indentation is a language rule
The step is exactly four columns. Tabs are rejected. Blank and comment-only lines produce no layout. Inside parentheses or map-literal braces, a newline is spacing rather than a statement terminator. Nesting is bounded so hostile generated input reports an error instead of exhausting the native stack.
Accepted
A block opens exactly four spaces deeper than its parent and returns to an earlier stack entry.
Refused
Two-space taste, tab stops, or a dedent to a column that was never an open block.
Source must read the way it runs
Bidirectional controls are rejected everywhere, including comments and string literals. Raw controls inside text are rejected. Unicode look-alikes for language punctuation are diagnosed with the ASCII character to write. These checks live at the lexical boundary because letting a deceptive spelling become an ordinary token would erase the evidence.
Recovery without pretending success
The lexer does not stop at the first malformed byte. It emits a luce.lex.* diagnostic and the closest usable stream, including a recovery token where a value was clearly intended. Identical diagnostic runs collapse, total reports are capped, and a structural truncation tells the parser not to complain about the synthetic closing tokens.
One mistake, one owner. If the lexer already named a stray character or unterminated string, the parser suppresses the secondary complaint that would merely describe the hole left behind.
What lexing does not decide
name and TypeName are both identifier tokens; meaning comes later. A minus sign and an integer are separate tokens; contextual numeric typing comes later. Whether a call exists, whether a field is private, and whether a value is sendable are all invisible here.