File detail
rust_port.rq
reqlan rq/core_analysis/rust_port.rq
| Idea | Status | Tags | Summary | Katz |
|---|---|---|---|---|
| analytical_rust_port | in-progress | Port ["../core-architecture.rq".analytical_package] to Rust: lexer, parser ( index path ), extractor, SQLite index, search, and export. `@reqlan/analytical` remains the published JS facade ( `createAnalysisRuntime` / `AnalysisApi` / Zustand host state ). Native work lives in a `crates/` Cargo workspace and ships as per-architecture binaries per [distribution.rust_binary_distribution]. The editor language server is [lsp_rust.language_server_rust]. This analytical port does not replace that process. Component plans: [rust_crate_layout], [lexer_rust], [parser_rust], [parser_align], [indexer.indexer_rust], [search_rust], [export_rust], [native_bridge], [cutover]. | 0.243 | |
| rust_crate_layout | done | Cargo workspace at repo-root `crates/` ( not inside `packages/analytical` ), crates: - `reqlan-parse` — lexer, parser, AST, recovery, parse budget - `reqlan-index` — extract, rusqlite store, mtime sync, diagnostics - `reqlan-search` — fuzzy / contextual / semantic ranking - `reqlan-export` — snapshot + html / md / json / csv writers; embed existing HTML JS assets - `reqlan-analytical` — facade matching `AnalysisApi` + `WorkspaceIndex` operations - `reqlan-napi` — Node binding for the core engine ( [native_bridge] ) - `reqlan-lsp` — editor language server ( [lsp_rust.language_server_rust] ) Mature crates only ( rusqlite, napi ): no experimental parser generators for the lexer. Golden tests compare Rust extract against Langium on the frozen [golden_corpus], plus the language-test fixtures ported into `reqlan-parse` / `reqlan-index` tests. Format: ["../development/commit.rq".rust_fmt]. Root `pnpm test` runs `test:js` then `test:rust` from ["../../package.json"]. `test:rust` is `cargo test --workspace --exclude reqlan-napi`. GitHub CI compiles crates once, then runs `cargo test` as its own step so crate tests reuse that build ( ["../distribution/distribution.rq".ci_gate] ). | 0.138 | |
| golden_corpus | done | Parser / extractor alignment uses a frozen `.rq` tree under `testdata/golden-corpus/`, not live product requirements in `reqlan rq/`. Production idea lists may change without regenerating the dump. Copy a representative file into the corpus and refresh `crates/reqlan-index/tests/golden/langium-corpus-names.json` only when grammar or extract behaviour needs a new sample. Langium and Rust both extract name + kind from that tree and must match the committed dump. Live parse alignment ( elements and refs, both engines in one test ): [parser_align]. Comment character ranges on the same tree: [comment_span_align]. | 0.104 | |
| comment_span_align | done | On every frozen [golden_corpus] `.rq` file, `//` and `/* … */` comment character ranges must be identical across: - the Langium lexer ( hidden `SL_COMMENT` / `ML_COMMENT` ) - `findCommentSpansInText` C-style spans ( host `#` / triple-quote spans are not `.rq` comments ) - TextMate `comment.line.reqlan` and `comment.block.reqlan` scopes Same-line fences, glob `/**/`, URL `://`, and fenced ` ` ` bodies must not open comments. Mid-line triple backticks are prose. They must not open a fence or hide later ideas. Example: `@reqlan/*` in inline code must not open a block comment. Related: [parser_align], ["../language/syntax.rq".code_snippets], ["../language/syntax.rq".comments], ["../language/syntax-edge-cases.rq".fencing_comments], ["../language/syntax.rq".inline_code]. | 0.141 | |
| lexer_rust | done | Hand-written Rust lexer that ports ["../../packages/language/src/reqlan-token-builder.ts"]: structural vs prose `{` / `}`, fence opacity, quote-aware comments, `@` only at line start, glob `/**/` not a block comment. Linear in document size with sparse brace-depth lookup ( [syntax_edges.context_sensitive_lexer_scaling] ). Owned by `reqlan-parse`; consumed by [parser_rust] and by [lsp_rust.language_server_rust]. The Langium token builder remains only for the transitional `node` engine fallback and dual-parser tests. | 0.105 | |
| parser_rust | done | Recursive-descent parser matching ["../../packages/language/src/reqlan.langium"] productions into a Rust AST the extractor can walk. Recovery: nameless `{...}` and local import mistakes still yield a usable document ( [parser_lexer.recovery_vs_budget] ). Wall-clock [parser_lexer.parse_budget_timeout] via native timeout / cancel — no Node worker process required on the index path. Open `.rq` buffers and CLI `parse` use `reqlan-parse` ( [lsp_rust.language_server_rust] ). Langium remains a dual-parser test oracle and the transitional `node` engine fallback — not the default editor LSP. Alignment contract: [parser_align]. | 0.217 | |
| parser_align | done | Langium and `reqlan-parse` stay two engines as a dual-parser **test oracle**. Grammar truth is [syntax] / ["../../packages/language/src/reqlan.langium"]. The same source must yield the same parse snapshot: top-level elements, live bracket and wiki refs, inline-code count, and fence count. One Node test runs Langium and native `parseReqlanSource` ( `parseAlignSnapshot` ) together. A committed name dump is not the only check. The default editor LSP already uses Rust ( [lsp_rust.language_server_rust] ); this idea does not gate that cutover. Related: [golden_corpus], [parser_rust], [comment_span_align]. | 0.150 | |
| search_rust | done | Port [search_reqs.fuzzy_search] ( separator-insensitive ranking ), [search_reqs.contextual_search] ( hop-distance rerank ), and [search_reqs.semantic_search] into `reqlan-search`. Same `AnalysisApi.searchRequirements` / `--context` contract ( [cli.commands] ). Activity-bar idea search uses `WorkspaceIndexRuntime.fuzzy_search` / napi `NativeWorkspaceIndex.fuzzySearch` so ranking stays next to rusqlite and only capped hits cross into JS. Mixed list includes indexed `.rq` file-name hits ( `kind` `file` ) and pages with `offset` / `limit` ( `truncated` ). Client-side highlight helpers used by webviews may stay in the TS `./fuzzy-search` export until a shared WASM / JS copy is needed. | 0.137 | |
| export_rust | done | Port [export_reqs.export_pipeline]: `buildExportSnapshot` then HTML / markdown / JSON / CSV writers in `reqlan-export`. Interactive HTML JS ( graph canvas, list filters, search bars ) stays static assets bundled into the export — not rewritten in Rust ( [html_export.html_export], [html_export.html_export_graph_animation] ). Hosts still call `NativeAnalysisApi.exportHtml` / `exportMarkdown` / `exportJson` / `exportCsv`; they do not grow format writers. PDF remains print-HTML ( [export_reqs.pdf_export] ); [html_export.html_export_print_order_by] stays a product follow-up, not a port blocker. | 0.137 | |
| native_bridge | in-progress | The Rust engine is the analytical core ( parse, index, search, export, typed queries, git-dates, overview coverage, barrel plan ) — not an optional addon. `@reqlan/analytical` is a typed napi facade ( specta `generated.ts` ) plus thin Node helpers ( `createBase`, path utils, `SqliteIndexStore` wrappers over typed napi ). It does not depend on Langium, `@reqlan/language`, or Zustand. Webview-only helpers ( search highlight, filter labels, file-treatment, context signals / model, graph physics ) live in the extension package. Interactive HTML JS stays in `crates/reqlan-export/assets`. Native owns: parse-for-index, persist, query, search, export execution, IndexService / WorkspaceIndex sync, and the index lifecycle FSM + status snapshot via `NativeWorkspaceIndex` + `NativeSqlDb`. The FSM ( `opening` / `syncing` / `ready` / `error` ), counts, in-memory sync progress / cancel, last error, file-index issues ( cap 500 ), and the document-update / workspace-change rings ( cap 50 ) live in `WorkspaceIndexRuntime`; TS reads them through `statusSnapshot()`. Durable diagnostics persist in `index-diagnostics.sqlite`. Consumers: [cli.cli_package] and MCP import `@reqlan/analytical/core` so they never load Langium; the extension host uses the per-target VSIX ( [distribution.vsix_export], [distribution.extension_host_target] ). The host-matching `.node` is required at runtime; there is no sql. js fallback. | 0.290 | |
| cutover | done | Native engine is required for CLI, MCP, and the extension IndexService path ( no `REQLAN_ANALYTICAL_ENGINE=js` escape hatch ). Order: golden parse / extract → headless CLI `analyse` / `search` / `export` → MCP → extension via platform VSIX ( host-matching napi `.node` required ). Shared [bases.base] `.reqlan` DB is schema v4 rusqlite; fixtures verify reopen compatibility. The default editor LSP is [lsp_rust.language_server_rust]. Langium remains only as the transitional `node` engine fallback and dual-parser oracle ( [parser_align] ). | 0.118 | |
| port_phases | Ideaset (port_phases) | 0.040 | ||
| ts_interface | done | The TS interface over the Rust functionality stays fully typed via specta ( `crates/reqlan-analytical` `export-ts` → `packages/analytical/src/native/generated.ts` ). Specta copies Rust doc comments. `export-ts` rewrites those `rq:` paths from crate-src depth to `packages/analytical/src/native` depth. | 0.040 |
Related clusters
community
folder
file
status
status
status
tag
Local graph
Initialising graph…