Code reference detail
native-packaging.test.ts
packages/analytical/test/native-packaging.test.ts
| Idea | Status | Tags | Summary | Katz |
|---|---|---|---|---|
| analytical_rust_port | in-progress | Port ["../core-architecture.rq".analytical_package] to Rust: lexer, parser ( index path ), extractor, SQLite index, search, and export. `@reqlan/analytical` remains the published JS facade ( `createAnalysisRuntime` / `AnalysisApi` / Zustand host state ). Native work lives in a `crates/` Cargo workspace and ships as per-architecture binaries per [distribution.rust_binary_distribution]. The editor language server is [lsp_rust.language_server_rust]. This analytical port does not replace that process. Component plans: [rust_crate_layout], [lexer_rust], [parser_rust], [parser_align], [indexer.indexer_rust], [search_rust], [export_rust], [native_bridge], [cutover]. | 0.243 | |
| native_bridge | in-progress | The Rust engine is the analytical core ( parse, index, search, export, typed queries, git-dates, overview coverage, barrel plan ) — not an optional addon. `@reqlan/analytical` is a typed napi facade ( specta `generated.ts` ) plus thin Node helpers ( `createBase`, path utils, `SqliteIndexStore` wrappers over typed napi ). It does not depend on Langium, `@reqlan/language`, or Zustand. Webview-only helpers ( search highlight, filter labels, file-treatment, context signals / model, graph physics ) live in the extension package. Interactive HTML JS stays in `crates/reqlan-export/assets`. Native owns: parse-for-index, persist, query, search, export execution, IndexService / WorkspaceIndex sync, and the index lifecycle FSM + status snapshot via `NativeWorkspaceIndex` + `NativeSqlDb`. The FSM ( `opening` / `syncing` / `ready` / `error` ), counts, in-memory sync progress / cancel, last error, file-index issues ( cap 500 ), and the document-update / workspace-change rings ( cap 50 ) live in `WorkspaceIndexRuntime`; TS reads them through `statusSnapshot()`. Durable diagnostics persist in `index-diagnostics.sqlite`. Consumers: [cli.cli_package] and MCP import `@reqlan/analytical/core` so they never load Langium; the extension host uses the per-target VSIX ( [distribution.vsix_export], [distribution.extension_host_target] ). The host-matching `.node` is required at runtime; there is no sql. js fallback. | 0.290 | |
| extension_host_native_stage | done | Local F5 and `pnpm run build` / `build:ext` stage the host-matching core engine `.node` into `packages/extension/native/` so activation loads the same layout as a per-target VSIX ( [distribution.vsix_export], ["../distribution/native_host_binary.rq".native_host_binary_development] ). The same build copies the `.node` into `packages/analytical-native/<napiSuffix>/` so the workspace optionalDependency matches a production npm install ( [distribution.rust_binary_distribution] ). The orchestrator builds `cargo build -p reqlan-napi -p reqlan-lsp --release` when needed, then copies `reqlan_napi.node` beside the extension. It is not a separate manual install. That command writes `reqlan-lsp` into `crates/target` ( ["../extension/language/support/language-server-rust.rq".language_server_rust] ). Azure compiles the [distribution.native_targets] matrix, then stages those binaries with `prepare-native-packages.mjs --binary-dir` before tests and VSIX packaging. Versions come from git at the `ext/v*` tag. Do not fetch platform packages from npm on Azure ( [distribution.extension_distribution] ). Implementation: ["../../scripts/stage-host-native.mjs"], ["../../scripts/ensure-host-native.mjs"], ["../../packages/extension/scripts/build.mjs"], ["../../azure-pipelines.yml"]. | 0.140 | |
| incremental_extension_build | done | Extension builds fingerprint inputs and outputs per independently cacheable artifact: - Langium generated sources / TextMate grammar - logo media - generated marketplace README - each of the five Vite webviews separately - extension-host / language-server esbuild outputs Host napi staging ( [extension_host_native_stage] ) always runs: cargo is incremental locally; Azure stages matrix artifacts and skips a second cargo compile in the package job. A step is skipped only when its command / input fingerprint and current output-content fingerprint both match the cache under `packages/extension/out/`. Missing, changed, or manually edited output invalidates that step. Fingerprint walks skip `node_modules` so Vite scratch files under `.vite-temp` and installed deps do not race or inflate hashes; dependency identity stays in `pnpm-lock.yaml` via shared inputs. Webview output fingerprints cover each complete emitted directory, including future lazy chunks. Host-bundle inputs include bundled phonebook data as well as workspace package output. TypeScript retains its native `tsc -b` project-reference incrementality. TextMate syntax copies are cached by content; Cursor-skill sync still runs because it targets workspace-external state. `pnpm run build:force` / `node scripts/build.mjs --force` bypasses all cache entries. `vscode:prepublish` uses forced mode with `--skip-native` so `vsce package --target` cannot replace a staged per-target `native/` with the packaging agent's host binary ( [distribution.vsix_export] ). `clean` removes `out/`, including the cache, so clean builds remain cold. Root `pnpm run build:ext` delegates only to the extension package. The extension build generates Langium artifacts when their fingerprints change, then its `tsc -b` builds language / analytical project references before downstream fingerprints are evaluated. It does not rebuild unrelated CLI, MCP, site, or test projects. | 0.045 | |
| scope | extension build steps for local development and CI | 0.040 | ||
| extension_host_target | in-progress | The extension host must load the native core engine for its machine. The selection strategy is ["./native_host_binary.rq".native_host_binary]. IndexService and WorkspaceIndex use `NativeWorkspaceIndex` ( ["../core_analysis/rust_port.rq".native_bridge] ). If the engine is missing, activation fails. Loader: ["../../packages/analytical/src/native/load-native.ts"]. The extension registers search folders from ["../../packages/extension/src/analytical_submodule/index.ts"]. | 0.268 | |
| extension_package | done | The VS Code extension lives in packages / extension with npm package name `reqlan-extension` ( Marketplace / Open VSX id `reqlan.reqlan-extension` ). Its package. json version is the Marketplace and Open VSX release version. That version is independent of `@reqlan/analytical` ( [version_management] ). | 0.088 | |
| extension_pipeline_cache | done | Azure `build_native` mirrors GitHub `Swatinem/rust-cache` via `Cache@2`: cache rustup + cargo home under `$(Pipeline.Workspace)/rust-cache` ( `CARGO_HOME` / `RUSTUP_HOME` ) and `crates/target`, keyed by OS, `rustTarget`, and `crates/Cargo.lock`. Install rustup only when `$CARGO_HOME/bin/rustc` is missing after restore; otherwise reuse the cached toolchain ( `--no-modify-path` ). Azure `build_test_package` caches the pnpm store keyed by OS and `pnpm-lock.yaml` ( same role as `cache: pnpm` on GitHub ). Do not cache VSIX or napi pipeline artifacts across tags; git at `ext/v*` stays canonical ( [vsix_native_not_from_npm] ). Implementation: ["../../azure-pipelines.yml"]. Related: [extension_distribution], ["./distribution.rq".ci_gate] ( GitHub rust-cache + pnpm ). | 0.072 | |
| extension_pipeline_publish | done | Marketplace and Open VSX publish every per-target VSIX in parallel ( `ForEach-Object -ThrottleLimit 6 -Parallel` ), not one after another. Open VSX uses the workspace `ovsx` package ( `pnpm --filter./packages/extension exec ovsx` ); do not `pnpm dlx ovsx` per file. Local `--openvsx` in ["../../scripts/package-extension-targets.mjs"] uses the same `pnpm exec ovsx` path. Implementation: ["../../azure-pipelines.yml"], ["../../packages/extension/package.json"], ["../../scripts/package-extension-targets.mjs"]. Related: [extension_distribution], [openvsx_registry], [vsix_export]. | 0.094 | |
| npm_distribution | in-progress | npm publishing runs only when a `language/v*`, `analytical/v*`, `cli/v*`, or `mcp/v*` tag is pushed. Tags are created by ["./distribution.rq".deployment_flow]; the npm workflow does not publish on ordinary commits. The workflow builds platform binaries when `@reqlan/analytical` was bumped ( `analytical/v*` tag, or `analytical_tag` from the release workflow ). It does not use `github.event_name == 'workflow_call'` for that gate: in a reusable workflow that name is the caller event ( `push` ), and `npm_ref` is the first language / analytical / cli / mcp tag, so a language bump would skip natives. It then publishes each package in sequence with `pnpm --filter <pkg> publish --access public --no-git-checks`. The sequence is language, then all platform packages for [native_targets], then `@reqlan/analytical`, then cli, then mcp. The workflow does not publish `@reqlan/analytical` until every platform package of that `@reqlan/analytical` version is on the npm registry or was published in the same job ( [rust_binary_distribution] ). ["../../scripts/require-natives-published.mjs"] enforces that gate from the analytical package. json version ( not Changesets ). Before each package publishes, ["../../scripts/skip-already-published.mjs"] with `--filter <pkg>` checks the public npm registry and skips that step when the version is already published ( sets `should_publish=false` ). That avoids re-attempting already-published versions under OIDC. Auth uses npm trusted publishing ( OIDC ) from GitHub Actions — no long-lived `NPM_TOKEN`. Workflow requirements: `permissions.id-token: write`, Node ≥ 22. 14 with npm ≥ 11. 5. 1, registry `https://registry.npmjs.org`. Each of `@reqlan/language`, `@reqlan/analytical`, `@reqlan/cli`, `@reqlan/mcp`, and every `@reqlan/analytical-<platform>-<arch>` platform package must have a Trusted Publisher on npmjs. com for GitHub Actions with org / user `littletuna4`, repository `reqlan`, workflow filename `deploy-npm.yml`, allowed action `npm publish`. Package `repository.url` must match `https://github.com/littletuna4/reqlan.git`. Provenance attestations are generated automatically for public packages from this public repo. Bootstrap: first publish of each scoped package can be done locally ( you must be logged in to npm as an `@reqlan` org member ) with `pnpm --filter @reqlan/language publish --access public --no-git-checks`, then platform packages, then analytical, then cli, then mcp — or wait for the trusted-publisher CI path after packages exist and Trusted Publisher is configured. Note: Packages must be `private: true` to skip local `changeset publish` ( `pnpm run release:npm` ), as with `reqlan-extension`. Platform packages are in the Changesets `ignore` list ( `@reqlan/analytical-*` ). Git `@reqlan/analytical` does not list them as `optionalDependencies`. ["../../scripts/prepare-native-packages.mjs"] `--publish-versions` writes pinned optionalDependencies for the npm checkout only. CI publishes platform packages per-package after binaries are built and does not use `changeset publish` for those tarballs. Published packages declare `publishConfig.access: public`. Trusted Publisher is configured on npmjs. com for each `@reqlan/analytical-<platform>-<arch>` package | 0.266 | |
| rust_binary_distribution | in-progress | One platform-specific npm package per [native_targets] tuple: `@reqlan/analytical-<platform>-<arch>` ( napi suffixes: `linux-x64-gnu`, `linux-arm64-gnu`, `darwin-x64`, `darwin-arm64`, `win32-x64-msvc`, `win32-arm64-msvc` ). Each package declares `os` and `cpu`, contains the matching core engine `.node`, and uses the `@reqlan/analytical` package. json version as the single source of truth. ["../../scripts/prepare-native-packages.mjs"] writes that version. Changesets ignores `@reqlan/analytical-*` in ["../../.changeset/config.json"]. Git `@reqlan/analytical` does not list platform packages as `optionalDependencies`; this workspace links them through `packageExtensions` in ["../../pnpm-workspace.yaml"]. The published tarball lists pinned optionalDependencies ( `--publish-versions` ) ( [version_management] ). CLI and MCP install the host-matching binary from those published optionalDependencies. The published tarball runs ["./native_host_binary.rq".published_host_native_install] so a host miss prints a warning. Published on the existing `analytical/v*` channel ( [npm_distribution] / [release_tagging] ); not a separate tag channel. The napi build runs when this channel moved: a direct `analytical/v*` tag, or `analytical_tag` from [deployment_flow]. The first npm tag ( `npm_ref` ) can be `language/v*` in the same release, so that ref is not the native-build gate. Website download listing and GitHub-release postinstall are later follow-ups ( out of scope here ). Engine port: ["../core_analysis/rust_port.rq".analytical_rust_port] / ["../core_analysis/rust_port.rq".native_bridge]. Per-target VSIX packaging uses the same package layout under ["../../packages/analytical-native/"], filled by Azure from this git commit ( [vsix_export], [vsix_native_not_from_npm] ). npm tarballs of those packages are for CLI / MCP only. `collect-napi-binary.mjs` copies `reqlan-lsp` only with `--with-lsp`. The npm workflow does not pass that flag. Azure does, because the VSIX ships `bin/reqlan-lsp` ( [vsix_export] ). Implementation: ["../../packages/analytical-native/"], ["../../scripts/prepare-native-packages.mjs"], ["../../scripts/require-natives-published.mjs"], ["../../packages/analytical/scripts/check-host-native.mjs"], ["../../.changeset/config.json"], ["../../.github/workflows/deploy-npm.yml"] build-native job, ["../../azure-pipelines.yml"] build_native job. | 0.380 | |
| site_distribution | GitHub Pages deployment runs only when an `site/v*` tag is pushed. Tags are created by ["./distribution.rq".deployment_flow]; the site workflow does not deploy on ordinary commits. The static build embeds `/spec` via ["../site/site.rq".spec_html_export], which needs the host `reqlan-napi` binary. The site workflow therefore builds and stages the host native engine ( `scripts/stage-host-native.mjs` with `REQLAN_FORCE_NATIVE_BUILD=1` ) before `pnpm run build`. The implementation lives in ["../../.github/workflows/deploy-site.yml"]. | 0.118 | ||
| stopgap | Stopgap distribution is at https://marketplace.visualstudio.com/manage/publishers/reqlan | 0.040 | ||
| version_management | done | Package versions are managed with Changesets via ["../../.changeset/config.json"]. Publishable channels ( each tag-driven ): - Marketplace / Open VSX: ["./distribution.rq".extension_package] ( private npm package; tagged `ext/v*` ) - GitHub Pages: ["./distribution.rq".site_package] ( tagged `site/v*` ) - npm: `@reqlan/language`, `@reqlan/analytical`, `@reqlan/cli`, and `@reqlan/mcp` ( tagged `language/v*`, `analytical/v*`, `cli/v*`, `mcp/v*` ) `reqlan-extension` has its own semver. It is not version-locked to `@reqlan/analytical`. The extension can update independently of the analytical surfaces it uses ( [extension_package] ). The version of `@reqlan/analytical` is the single source of truth for every `@reqlan/analytical-<platform>-<arch>` package ( [rust_binary_distribution] ). Platform packages are in the Changesets `ignore` list ( `@reqlan/analytical-*` ). Git `@reqlan/analytical` does not list them as `optionalDependencies`, so `changeset version` can run. This workspace still links the checkout packages during `pnpm install` via `packageExtensions` in ["../../pnpm-workspace.yaml"] ( `workspace:*`, not npm ). After `changeset version`, ["../../scripts/prepare-native-packages.mjs"] copies the analytical version onto each platform package. The published tarball lists pinned `optionalDependencies` ( `--publish-versions` ) so CLI and MCP customers get the host binary. Platform native packages are not a separate tag channel; they ride `analytical/v*` ( [rust_binary_distribution] ). VSIX packaging uses the package. json versions in that git commit ( [vsix_native_not_from_npm] ). `@reqlan/language`, `@reqlan/cli`, `@reqlan/mcp`, and the site keep their own versions. Contributors record release intent by adding a changeset file when a publishable package changes. | 0.233 | |
| vsix_native_not_from_npm | done | Azure must not use the npm deploy path as an input when it packs VSIXes. Git at the `ext/v*` tag is canonical. Azure compiles the [native_targets] matrix from that commit ( [vsix_export], [extension_distribution] ). npm stays the install path for CLI and MCP only ( [npm_distribution], [rust_binary_distribution] ). | 0.241 | |
| native_host_binary | done | Select the native core-engine binary from the extension host. The host is the Node process that runs the extension. Read `process.platform` and `process.arch` from that process. Do not use the platform of the VS Code UI client. Examples: - UI on Windows and host on Linux SSH: use a linux tuple. - UI on Windows and host on WSL: use a linux tuple. - UI and host on macOS: use a darwin tuple. Map the platform and the arch to one tuple in ["./distribution.rq".native_targets]. Load only the binary that matches that tuple. Do not load a binary for a different tuple. If the platform and the arch are not in ["./distribution.rq".native_targets], fail. If the binary is missing, fail. Do not use sql. js. Keep the loader map the same as ["../../scripts/native-targets.mjs"]. Local development: [native_host_binary_development]. Production distribution: [native_host_binary_distributed]. Implementation: ["../../packages/analytical/src/native/load-native.ts"]. | 0.144 | |
| native_host_binary_development | done | Use this procedure on a developer machine ( F5 and `pnpm run build` ). Identify the host from `process.platform` and `process.arch` of the build process. The build process and the extension host must be the same machine. Copy the matching `.node` file to `packages/extension/native/reqlan_napi.node`. Write `packages/extension/native/target.json` with the host tuple. If the binary is not present, build it with `cargo build -p reqlan-napi -p reqlan-lsp --release`. At run time, search in this order: 1. The `native/` folder of the extension, if `target.json` records the host tuple. 2. The resolved `main` of the host npm package ( `createRequire` from `@reqlan/analytical/package.json` ). 3. The host file in `packages/analytical-native/<napiSuffix>/` ( workspace package directory, same layout as a published package ). 4. The host npm package name. 5. `crates/target/<rustTarget>/release` and `crates/target/release`. 6. The debug folders with the same names. Do not search packages for other platforms. If `target.json` records a different tuple, do not load `native/reqlan_napi.node`. Windows cargo emits `reqlan_napi.dll`; the loader uses `process.dlopen` for `.dll` and `.node` ( Node `require` only loads `.node` ). Windows UI with a WSL extension host uses the linux tuple ( [native_host_binary] ). Windows UI with a Windows extension host cannot load a `.node` staged from WSL; restage on Windows or open the folder in a WSL remote window. Before tests, the workspace host package must contain the `.node` so loaders behave like production ( [native_host_binary_distributed] ). `scripts/ensure-host-native.mjs` uses an existing matrix binary, or cargo locally. Azure compiles the ["./distribution.rq".native_targets] matrix from git and stages those artifacts; it does not fetch binaries from npm. Detail: [development_build.extension_host_native_stage]. Implementation: ["../../scripts/stage-host-native.mjs"], ["../../scripts/ensure-host-native.mjs"], ["../../packages/extension/src/analytical_submodule/index.ts"]. | 0.078 | |
| native_host_binary_distributed | in-progress | Use this procedure for a published VSIX and for published npm packages. VS Code extension: The registry installs the VSIX that has `targetPlatform` equal to the extension host. Each VSIX contains one `.node` file for one ["./distribution.rq".native_targets] tuple. The file name is `native/reqlan_napi.node`. The VSIX also contains `native/target.json`. At run time, read `process.platform` and `process.arch` on the extension host. If `target.json` records a different tuple, do not load `native/reqlan_napi.node`. Do not put binaries for other platforms in that VSIX. The Azure pipeline compiles each platform binary from this git commit ( `build_native` + `collect-napi-binary.mjs` + `prepare-native-packages.mjs --binary-dir` ). Package versions in git are canonical. The pipeline does not wait for npm. Related: ["./distribution.rq".vsix_export], ["./distribution.rq".extension_distribution], ["./distribution.rq".vsix_native_not_from_npm]. CLI and MCP: npm installs the host package from `optionalDependencies` on the published `@reqlan/analytical` tarball. Git does not list those entries. ["../../scripts/prepare-native-packages.mjs"] `--publish-versions` writes them at npm publish. The loader resolves that package from `@reqlan/analytical/package.json` ( `createRequire` ), which is where the optional dependency is attached after `npm install`. Published installs also run [published_host_native_install]. Related: ["./distribution.rq".rust_binary_distribution], [rust_port.native_bridge]. Implementation: ["../../scripts/package-extension-targets.mjs"], ["../../scripts/prepare-native-packages.mjs"], ["../../packages/analytical/src/native/load-native.ts"], ["../../packages/analytical/scripts/check-host-native.mjs"], ["../../azure-pipelines.yml"]. | 0.129 | |
| published_host_native_install | done | After a published install of `@reqlan/analytical`, the host platform package must resolve. The host is `process.platform` and `process.arch` of the install process ( [native_host_binary] ). Git `@reqlan/analytical` does not list `optionalDependencies`. The check exits 0 in this workspace. The published tarball lists pinned optionalDependencies ( ["./distribution.rq".rust_binary_distribution] ). That tarball runs ["../../packages/analytical/scripts/check-host-native.mjs"] as `postinstall`. If the host package name is missing from `optionalDependencies`, or `createRequire` from `@reqlan/analytical/package.json` cannot resolve that package to a file, the script prints a warning and exits 0. The warning text tells the user how to enable optional dependencies and how to limit pnpm to the host OS and CPU. Do not use `--force`. `--force` installs every optional platform package. Do not skip the script with `--ignore-scripts` on a published install. CLI install documentation: ["../../packages/cli/README.template.md"]. Implementation: ["../../packages/analytical/scripts/check-host-native.mjs"], ["../../packages/analytical/package.json"], ["../../packages/cli/README.template.md"], ["../../packages/analytical/README.template.md"]. | 0.114 | |
| site | the / site folder contains a static Next. js App Router landing page for reqlan. should align with [brand] Copy modules: [copy] Document head: [meta_tags] | 0.040 | ||
| spec_html_export | done | The site static build exports the workspace requirement graph ( excluding `*.secret.rq` ) under `out/spec/` via `reqlan export`. It must pass [html_export.html_export_url_base] for `/spec` ( honouring `SITE_BASE_PATH`, e. g. `/reqlan/spec` in production ) so `/spec` loads with or without a trailing slash. It must pass [html_export.html_export_header_link] pointing at the marketing site home so readers can leave the embedded spec and return to the parent site. Embedded `/spec` graph pages inherit [html_export.html_export_graph_background_load] so large workspace graphs do not freeze the page while settling. Pipeline: ["../../site/scripts/build-static.mts"] → ["../../site/scripts/export-spec.mts"] → ["../../site/scripts/verify-static-export.mts"]. Host options come from [html_export.html_export] / CLI [cli_package.commands], including native CLI export via ["../core_analysis/rust_port.rq".native_bridge]. CI must build / stage host `reqlan-napi` before this pipeline ( [distribution_reqs.site_distribution] ); the loader probes repo-root `crates/target`, not `packages/crates`. | 0.142 |
Reference labels
"../../packages/analytical/test/native-packaging.test.ts", "../../packages/analytical/test/native-packaging.test.ts:@reqlan/analytical optionalDependencies are publish-only, not in git", "../../packages/analytical/test/native-packaging.test.ts:Azure extension pipeline caches rust and pnpm and publishes in parallel", "../../packages/analytical/test/native-packaging.test.ts:analytical version is SSOT for platform packages; Changesets ignores them", "../../packages/analytical/test/native-packaging.test.ts:deploy-npm builds natives when analytical was bumped, not when npm_ref is language", "../../packages/analytical/test/native-packaging.test.ts:deploy-npm publishes platform packages before @reqlan/analytical", ../../packages/analytical/test/native-packaging.test.ts, ../../packages/analytical/test/native-packaging.test.ts:@reqlan/analytical optionalDependencies are publish-only, not in git, ../../packages/analytical/test/native-packaging.test.ts:Azure extension pipeline caches rust and pnpm and publishes in parallel, ../../packages/analytical/test/native-packaging.test.ts:analytical version is SSOT for platform packages; Changesets ignores them, ../../packages/analytical/test/native-packaging.test.ts:deploy-npm builds natives when analytical was bumped, not when npm_ref is language, ../../packages/analytical/test/native-packaging.test.ts:deploy-npm publishes platform packages before @reqlan/analytical