context_sensitive_lexer_scaling

Katz 0.097

Context-sensitive token classification for structural / prose braces and top-level import keywords must remain linear in document size.
The lexer records brace-depth changes sparsely ( not per-character tables ) and looks them up in log time while tokenising, so workspace language-server initialization and editor re-lexes stay bounded.
Code fences are skipped during the brace scan so example braces cannot inflate work or desync depth.
Bounded work still assumes the lexer / parser *returns*; unbounded hangs are cut by parse_budget_timeout.

implementation

reqlan-token-builder.ts

status

done

tests

parsing.test.ts:parses a large import prelude without repeated full-document brace scans, parsing.test.ts:treats fenced TypeScript braces as opaque for surrounding structure

Interactive page: context_sensitive_lexer_scaling