diff --git a/.claude/SCHEMA_DECISIONS.md b/.claude/SCHEMA_DECISIONS.md index 800ffd9..ebd14e2 100644 --- a/.claude/SCHEMA_DECISIONS.md +++ b/.claude/SCHEMA_DECISIONS.md @@ -183,3 +183,16 @@ an unresolved read is `reason: "non-literal"` (key never closes on one literal) | L3 | `def → actual_in:k` binds when the reaching variable's head is a whole word in argument k's TEXT at the call site | derived at emit time from intra ddg + `PARAM_IN` + the callable's own `call_sites` (python's assembler produces these as `extra_edges`; ours does not) | no per-argument AST at this stage; the ceiling is a local/property name clash (`f(o.v)` with local `v`), which over-binds | | L4 | `actual_out → callsite` (python's class), not `actual_out → use stmt` | the existing `L → use` edges carry the value onward | derivable without guessing which defined variable is the return | | L5 | ddg deduped on `(src, dst, var, prov)` and sorted | a binding can be reached from more than one sdg edge | determinism | + +## Neo4j bindings, parameters, unresolved config reads (2026-09-07, #182 — python `PY_IMPORTS`/`parameters_json`/`PY_READS_CONFIG_UNRESOLVED` parity) + +Spec: `docs/design/specs/neo4j-bindings-parameters-config.md`. Additive on contract 2.0.0. + +| # | Concept | Decision | Rationale | +|---|---|---|---| +| D1 | **`resolved_module`** on `TSImport` / re-export `TSExport` | project-relative file key from `ts.resolveModuleName` under the importer's program (`moduleResolution.ts`), kept whether or not the target module was emitted this run; absent for external/builtin/unresolvable; a PER-RUN stamp over every module, cached ones included (set or deleted each run) | python `PyImport.resolved_module` re-resolved per run by `resolve_imports`; the compiler already handles `paths`, index files, `.js`→`.ts` — no second resolver; per-run because tsconfig and file existence are state the content-hash cache cannot see (a build-time value goes stale or wrong while the importer is byte-identical). Incremental Bolt ceiling: an edge flipping absent→present under an unchanged importer waits for `--eager` (same class as `TS_CALLS`) | +| D2 | **`TS_IMPORTS`** | `:TSModule → :TSModule \| :TSExternal`, one edge per (importer, target): `spellings[]`, `imported_names[]`, `aliases[]`, `type_only_names[]`; external target = `/@external/` ghost; unresolved relative spellings dropped from the graph | python's aggregation rule (MERGE overwrites a second row for the same pair); ghost on the package root joins `TS_PROVIDES`; `type_only_names[]` is TS's one structural difference | +| D3 | **Exports** | `exports_json` on `:TSModule` (lossless) + `TS_RE_EXPORTS` aggregated like D2 (`exported_names[]`); local exports stay `is_exported` on the declaration node | no reference precedent — coined once here; the edge makes barrel chains walkable, the property keeps `export { x as y }` and spans | +| D4 | **`parameters_json`** on `:TSCallable` | `JSON.stringify(c.parameters)` verbatim, `null` when empty | python's property and encoding; SDK already decodes it; 1.4 % of graph on cants self; a property on the existing node does not worsen #177 | +| D5 | **`TS_READS_CONFIG_UNRESOLVED`** | `:TSApplication → :TSExternal \| :TSCallable`, `key`/`reason`/`prov`, `_k = key\|reason`, no `site`; env-root reads ghost under `@external/`, call-rule reads target the resolved callee id | python shape verbatim incl. its documented per-site collapse; retires the #101 "config_reads stay JSON-only" note (python overturned it in #162) | +| D6 | **Version / tracking** | contract `2.0.0`, analyzer 1.4.0, one PR closing #182 | every addition optional-with-absent; the SDK pins `analyzer_version` | diff --git a/CLAUDE.md b/CLAUDE.md index bd76fcd..1f63085 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -118,8 +118,14 @@ literal→dataflow-intra→dataflow-interproc tier (`src/semantic_analysis/confi `src/dataflow/configUse.ts`); `config_reads` deliberately SHRINKS as `-a` rises — the layer's one non-monotonic section. `src/artifacts/`. Neo4j contract 2.1.0 (SCHEMA_VERSION unmoved — every analyzer re-baselines together later): NEUTRAL :Artifact/:Package/:ConfigKey (purl) — sanctioned -prefix exception — plus TS_PROVIDES/TS_UNRESOLVED_IMPORT into :TSExternal ghosts and -TS_USES_CONFIG into :ConfigKey. Consumer query skill: `docs/skills/analyzing-cants-graphs/`. +prefix exception — plus TS_PROVIDES/TS_UNRESOLVED_IMPORT into :TSExternal ghosts, TS_USES_CONFIG +into :ConfigKey, and (#182, python parity) TS_READS_CONFIG_UNRESOLVED (app → ghost/callee, +`_k=key|reason`), TS_IMPORTS/TS_RE_EXPORTS (one per module pair, aggregated names; externals on +the same ghosts), `exports_json` on :TSModule, `parameters_json` on :TSCallable. `resolved_module` +on TSImport/TSExport = the compiler's answer (`ts.resolveModuleName`), a PER-RUN stamp over +cached modules too (`syntactic_analysis/moduleResolution.ts`) — tsconfig/file existence is +state the content-hash cache cannot see. +Consumer query skill: `docs/skills/analyzing-cants-graphs/`. ## Commands diff --git a/docs/design/specs/neo4j-bindings-parameters-config.md b/docs/design/specs/neo4j-bindings-parameters-config.md new file mode 100644 index 0000000..0ca2be1 --- /dev/null +++ b/docs/design/specs/neo4j-bindings-parameters-config.md @@ -0,0 +1,82 @@ +# Neo4j projection: import/export bindings, callable parameters, unresolved config reads + +Tracking: codeanalyzer-typescript #182. Ships in 1.4.0, one PR, Neo4j contract unchanged at +2.0.0 (additive). Sibling references: codeanalyzer-python `neo4j/schema.py` (`PY_IMPORTS`, +`parameters_json`, `PY_READS_CONFIG_UNRESOLVED`), python-sdk leg 2.5 spec +(`docs/design/specs/2026-09-06-leg-2.5-typescript.md`), codeanalyzer-java #231. + +## Problem + +Three python-sdk accessors refuse on a TypeScript graph and answer on the in-process backend, and +every one of them is recoverable from data `analysis.json` already carries. Measured on +`main @ 2064b6b` (cants on itself: 139 modules, 1008 callables): + +| gap | `analysis.json` | graph | +|---|---|---| +| import / export bindings | 801 imports, 67 exports | none | +| callable parameters | 1546 parameters on 970 callables | none | +| unresolved config reads | `config_reads` 10 (artifacts-app, `-a 2`) | none — `TS_USES_CONFIG` (resolved reads) IS projected; only the unresolved counterpart is missing | + +The third row corrects the issue as filed: TypeScript does not emit `DEFINES_CONFIG` alone. + +## Contract-impact triage + +- **Schema v2 output moves.** `analysis.json`: `resolved_module` on `TSImport` and `TSExport`. + Neo4j: relationship types `TS_IMPORTS`, `TS_RE_EXPORTS`, `TS_READS_CONFIG_UNRESOLVED`; + properties `:TSCallable.parameters_json`, `:TSModule.exports_json`. Ids, levels, the containment + spine, and every existing label/type are untouched. Additive → `SCHEMA_VERSION` stays `2.0.0` + (decision D1 of the v1.3.0-parity block: every analyzer re-baselines together). +- **Repos.** codeanalyzer-typescript emits (this PR). python-sdk consumes in leg 2.5b, which + re-points its TypeScript Neo4j backend from the v1 vocabulary and pins the release carrying this + work. codeanalyzer-java #231 must spell the unresolved-config edge the same way + (`JAVA_READS_CONFIG_UNRESOLVED`) — a naming constraint, not a child. Docs: the consumer skill + `docs/skills/analyzing-cants-graphs/` and the regenerated `schema.neo4j.json`. + +## Decisions + +| # | Concept | Decision | Rationale | +|---|---|---|---| +| D1 | **Import resolution** | `resolved_module?: string` on `TSImport` and on re-export `TSExport`s: the project-relative **file key** the specifier resolves to under the importer's own program, from `ts.resolveModuleName` (`syntactic_analysis/moduleResolution.ts`), keyed the same way the symbol table keys modules — **whether or not that module was emitted this run** (`--skip-tests` / `--program` can exclude the target; the graph gates on presence, the JSON keeps the key). Absent when the specifier is external (`isExternalLibraryImport`, outside the root, `node_modules`), a builtin, or unresolvable. **A per-run stamp over every module, cached ones included** — set or deleted each run, never a build-time fact. | Python parity (`PyImport.resolved_module`, re-resolved per run by `resolve_imports`; canonical keystone `import.path`). The compiler already resolved it — `tsconfig` `paths`, `index.ts`, `.js`→`.ts`, `.d.ts` — so the analyzer neither re-implements a resolver nor guesses; `ts.resolveModuleName` rather than ts-morph's symbol route because a side-effect-imported script with no exports has no module symbol. Per-run because the answer depends on the tsconfig and on which files exist, neither of which the per-file content hash sees: a build-time value would go silently stale (a target added later) or wrong (a `paths` edit) while the importer stays byte-identical. | +| D2 | **`TS_IMPORTS`** | `:TSModule → :TSModule \| :TSExternal`, **one edge per (importer, target)** aggregating every binding: `spellings[]` (the raw specifiers), `imported_names[]`, `aliases[]`, plus the TS-only `type_only_names[]` (names imported `import type` / `{ type X }`). Target: the resolved module's node when `resolved_module` is set; otherwise the existing import ghost `/@external/` (`binding.ts::specifierRoot`; `node:`-prefixed and bare builtins ghost under their own spelling, e.g. `@external/node:fs`). Relative spellings that failed to resolve are dropped from the graph (they survive in JSON). | Python's exact shape and aggregation rule (a second row for the same pair would overwrite the first under MERGE). Ghosting on the package root joins the edge to `TS_PROVIDES` / `TS_UNRESOLVED_IMPORT`, closing `:Package → :TSExternal ← :TSModule`. `type_only_names[]` lets a consumer drop type-only edges from a runtime dependency graph — TypeScript's one structural difference here. | +| D3 | **Exports** | Two carriers. `exports_json` on `:TSModule`: `JSON.stringify(mod.exports)` verbatim, `null` when empty — lossless, answers `get_exports`. `TS_RE_EXPORTS`: `:TSModule → :TSModule \| :TSExternal`, aggregated per (module, target) like D2 with `spellings[]`, `exported_names[]` (`"*"` for `export * from`), `aliases[]`, `type_only_names[]`; targets resolved as in D2. Local exports need no edge — `is_exported` already sits on every declaration node. | No reference precedent (ES modules only; python and java have none), so the shape is coined here once. The edge is what makes barrel chains (`index.ts` re-exporting a tree) walkable in Cypher; the property is what keeps `export { x as y }` locals and per-binding spans recoverable. The 1.x graph carried `RE_EXPORTS`; the name is prefixed like every other TS claim. | +| D4 | **`parameters_json`** on `:TSCallable` (and the `:TSAnonymousCallable` twin) | `JSON.stringify(c.parameters)` verbatim — name, `id` (`@formal_in:N`), type, default, `is_optional`/`is_rest`/`is_readonly`, accessibility, decorators, span — `null` when the list is empty. | Python's property, python's encoding; the SDK's `reconstruct.callable_` already decodes `parameters_json` for TypeScript, so the accessor answers with no SDK change. Cost measured at 430 KB on cants self = 1.4 % of `graph.cypher` (body nodes dominate); a minimal shape (148 KB) would be a second parameter vocabulary to keep in sync for 0.9 %. Property on the existing node, so declaration merging (#177) is not made worse: the collapsed node keeps the last writer's list exactly as it keeps the last writer's `kind`. | +| D5 | **`TS_READS_CONFIG_UNRESOLVED`** | `:TSApplication → :TSExternal \| :TSCallable`, one edge per `config_reads` record with `key`, `reason`, `prov`, discriminant `_k = key\|reason`. Target: for env-root reads (`callee` = `process.env`, `import.meta.env`, `Bun.env`) the ghost `/@external/`; for detector-table call rules the resolved callee id as-is (an `:TSExternal` ghost or an in-project `:TSCallable`). `site` is not carried. | Python's shape verbatim, including its documented ceiling: several sites reading the same (callee, key, reason) collapse into one edge, so counts differ from `analysis.json` while presence/absence agrees — the python-sdk comment on `get_unresolved_config_reads` applies unchanged and the shared reconstruct needs no TS branch. Retires the #101 note "config_reads stay JSON-only"; python overturned that in its #162. | +| D6 | **Version and tracking** | Contract `2.0.0`, analyzer 1.4.0 minor. One work item (#182), one PR, spec in this repo. | Every addition is optional-with-absent; a graph from 1.3.0 differs only by missing rows. The SDK's generation probe keys on `analyzer_version`, which is what it pins. | + +## Projection rules (what the backend rung implements) + +- `project.ts::projectModule`-side: after the module node, aggregate `mod.imports` by target + (`resolved_module` → `moduleIdOf(app, key)`; else `specifierRoot(spec) ?? spec` → import ghost), + emit one `TS_IMPORTS` per bucket; same for re-export `TSExport`s into `TS_RE_EXPORTS`. + `exports_json` rides `moduleProps`. Sorted arrays, so the snapshot is byte-stable. +- `callableProps` gains `parameters_json`. `prune` keeps the python `null`-when-empty rule. +- After the `config_uses` loop: `config_reads` → `TS_READS_CONFIG_UNRESOLVED` with key + `${key ?? ""}|${reason}`. +- `schema.ts`: three `RelType`s, two properties, `_k` declared on the config edge. `bun run + gen:schema` re-baselines `schema.neo4j.json`; the conformance test enforces it. +- Incremental Bolt (#140): every new edge's owner is its source module (`TS_IMPORTS`/`TS_RE_EXPORTS`) + or the application (`TS_READS_CONFIG_UNRESOLVED`). The per-module purge covers the removal + direction (a target that vanishes takes its incoming edges with it under `--eager`; a default push + leaves them, as it leaves the target's nodes). **Known ceiling, same class as `TS_CALLS`:** a + binding edge's presence depends on the *target* existing, so an edge that flips absent → present + while the importer is byte-identical (target file added, shard widened) is computed but filtered + by the "shared or changed-owner" rule and not written until the importer changes or the push is + `--eager`. Ghost targets are shared nodes, never pruned — unchanged. + +## Definition of done + +- `analysis.json`: `resolved_module` present on every relative import/re-export of the sample-app + fixture that names a module in the symbol table; absent on externals. L1 ⊆ L2 ⊆ L3 ⊆ L4 gate + still green (the field is level-free). +- Snapshot on `test/fixtures/sample-app`: `TS_IMPORTS` to `:TSModule` for `./controllers`, + `./models`, `./services`, `./util`, to ghosts for `commander`, `neo4j-driver`, `node:crypto`, + `node:fs`, `node:path`; `parameters_json` on every callable with parameters; `exports_json` + where `exports[]` is non-empty. On `artifacts-app`: `TS_READS_CONFIG_UNRESOLVED` count = number + of distinct (callee, key, reason) triples in `config_reads`. +- Conformance test green against the regenerated `schema.neo4j.json`; edge-identity test covers + the `_k` discriminant on the new config edge; bolt container test exercises an incremental + re-push where an import target module vanishes. +- `.claude/SCHEMA_DECISIONS.md` carries D1–D6; `docs/skills/analyzing-cants-graphs/` documents + the three query shapes. +- Propagation: comment on codeanalyzer-java #231 naming `JAVA_READS_CONFIG_UNRESOLVED` and the + `_k` rule; python-sdk #55 / leg 2.5b pointed at the release. diff --git a/docs/skills/analyzing-cants-graphs/SKILL.md b/docs/skills/analyzing-cants-graphs/SKILL.md index b70cbc9..9145eb6 100644 --- a/docs/skills/analyzing-cants-graphs/SKILL.md +++ b/docs/skills/analyzing-cants-graphs/SKILL.md @@ -28,7 +28,10 @@ query answers the wrong question. The repository-artifact layer — `Artifact`/`Package`/`ConfigKey` and every edge among them (`HAS_ARTIFACT`, `DECLARES_DEPENDENCY`, `LOCKS`, `DEFINES_CONFIG`, `TS_PROVIDES`, -`TS_UNRESOLVED_IMPORT`) — is **L1 data, level-free**: identical at every `-a`. Neo4j is always +`TS_UNRESOLVED_IMPORT`) — is **L1 data, level-free**: identical at every `-a`. So are the module +binding edges `TS_IMPORTS` / `TS_RE_EXPORTS`, `exports_json` on `TSModule`, and `parameters_json` on +`TSCallable`. `TS_READS_CONFIG_UNRESOLVED` exists from `-a 2` and SHRINKS as the level rises (a read +resolved at a higher tier moves to `TS_USES_CONFIG`). Neo4j is always projected **full-depth** for the level actually analyzed (`--emit neo4j` + `-a`/`--graphs` together is a CLI error, not a partial graph). @@ -76,9 +79,10 @@ section (`config_uses` is the opposite: asserted superset-monotonic, L2 ⊆ L3 this branch's fixture at 21/25/29 uses and 10/9/8 reads across L2/L3/L4). A read unresolved at the literal tier can close at a higher dataflow tier, so it *moves* from `config_reads` into `config_uses` as the level climbs. Diffing two levels and seeing a `config_reads` record vanish -means "resolved at the higher tier," never "fixed in the code." Pair the two: `config_uses` is the -graph edge (`TS_USES_CONFIG`); `config_reads` never became an edge — it is JSON-only, a record of -absence, not a graph fact (analyses.md §7). +means "resolved at the higher tier," never "fixed in the code." Pair the two: `config_uses` is +`TS_USES_CONFIG` (body node → key); `config_reads` is `TS_READS_CONFIG_UNRESOLVED` (application → +the read root's ghost or the resolved callee), one edge per distinct (target, key, reason) — several +sites collapse onto it, so its count is a floor on the JSON list's, never equal (analyses.md §7). **`--app-name` is the cross-analyzer join precondition.** Artifact ids are language-neutral specifically so a TS and a Python analysis of one repository MERGE onto the same `:Artifact` node @@ -114,11 +118,16 @@ from the body node straight to the `ConfigKey` instead. no name segment) sit on the same label but different ids — join them by the `module` property when a query needs both (`references/vocabulary.md`, "External ghosts"). -**No entrypoint or import-graph vocabulary exists yet.** `TSCallable` carries no `is_entrypoint`; a -"reachable from the entrypoints" query needs a root set you supply yourself (analyses.md §2). A -module's `imports[]`/`exports[]` (specifiers, aliases, type-only flags) live only in -`analysis.json`'s `TSModule` — `TS_UNRESOLVED_IMPORT`/`TS_PROVIDES` cover the dependency-hygiene -case only, not a general per-module import graph. +**The import graph is aggregated, the export list is not.** `TS_IMPORTS` / `TS_RE_EXPORTS` carry ONE +relationship per (module, target) with every binding folded into `spellings[]` / +`imported_names[]` (`exported_names[]`) / `aliases[]` / `type_only_names[]` — per-binding spans and +kinds are not on the edge. `exports_json` on `TSModule` is the verbatim `exports[]` list (including +`export { x as y }` locals, which are no edge); imports have no such property — read +`analysis.json` for per-binding detail. An unresolved *relative* import (`./missing`) has no node to +land on and is absent from the graph while present in JSON; externals land on the same +`@external/` ghost `TS_PROVIDES` / `TS_UNRESOLVED_IMPORT` address, builtins under +their own spelling (`@external/node:fs`). A "reachable from the entrypoints" query still needs a +root set: `is_entrypoint` on `TSCallable`/`TSClass` (analyses.md §2). Every trap above is a silent-empty-result failure, not an error: Cypher does not reject a query naming a nonexistent label, property, or relationship — it just returns nothing. Cross-check diff --git a/docs/skills/analyzing-cants-graphs/references/analyses.md b/docs/skills/analyzing-cants-graphs/references/analyses.md index c742ead..e21fd94 100644 --- a/docs/skills/analyzing-cants-graphs/references/analyses.md +++ b/docs/skills/analyzing-cants-graphs/references/analyses.md @@ -18,6 +18,20 @@ ORDER BY m.name LIMIT 25 // locate a callable — the graph has no source text, only file + line span (see vocabulary.md) MATCH (c:TSCallable {name: "analyze"}) RETURN c.signature, c._module, c.start_line, c.end_line, c.cyclomatic_complexity + +// module import graph, in-project only (drop type-only-only edges for a runtime view) +MATCH (m:TSModule)-[i:TS_IMPORTS]->(t:TSModule) +WHERE i.imported_names IS NULL OR size(i.imported_names) > size(coalesce(i.type_only_names, [])) +RETURN m.name AS importer, t.name AS target, i.imported_names, i.spellings + +// who imports a package (external), with the names they take +MATCH (m:TSModule)-[i:TS_IMPORTS]->(x:TSExternal {module: "commander"}) RETURN m.name, i.imported_names + +// barrel chains: what a module re-exports, transitively +MATCH p = (m:TSModule {name: "src/index.ts"})-[:TS_RE_EXPORTS*1..5]->(t:TSModule) RETURN [n IN nodes(p) | n.name] + +// a callable's parameters, decoded client-side (JSON string; absent when there are none) +MATCH (c:TSCallable {name: "create"}) RETURN c.signature, c.parameters_json ``` ## 2. Call graph (L2) @@ -224,12 +238,21 @@ RETURN k.key, // literal-tier-only view (drop the dataflow-widened edges) MATCH ()-[u:TS_USES_CONFIG]->() WHERE u.prov = ["literal"] RETURN count(u) + +// reads that closed on NO declared key: which root/callee, which key (if literal), why +MATCH (a:TSApplication)-[r:TS_READS_CONFIG_UNRESOLVED]->(t) +RETURN coalesce(t.module, t.signature) AS via, r.key, r.reason ORDER BY r.reason, r.key + +// undefined-key reads = literal keys nobody declares (a config-drift signal) +MATCH ()-[r:TS_READS_CONFIG_UNRESOLVED {reason: "undefined-key"}]->() RETURN collect(DISTINCT r.key) ``` Edges never guess: the literal tier (`-a 2`) needs a statically-known key at a recognized env root or detector-listed call; the dataflow tiers (`-a 3` intra, `-a 4` interprocedural) resolve only chains that close over exactly one string literal. Everything else lands in `config_reads` with a -reason — see SKILL.md's standing traps for why that list shrinks as `-a` rises. +reason, projected as `TS_READS_CONFIG_UNRESOLVED` — one edge per distinct (target, key, reason), so +the graph count is a floor on the JSON count — see SKILL.md's standing traps for why that list +shrinks as `-a` rises. ## 8. Health metrics (any level) diff --git a/docs/skills/analyzing-cants-graphs/references/vocabulary.md b/docs/skills/analyzing-cants-graphs/references/vocabulary.md index 7bbcd00..bdbdf51 100644 --- a/docs/skills/analyzing-cants-graphs/references/vocabulary.md +++ b/docs/skills/analyzing-cants-graphs/references/vocabulary.md @@ -87,14 +87,14 @@ package" — `references/analyses.md` §6). | `TS_PROVIDES` | Package → TSExternal (module-level ghost) | — | correlatable with the call-graph grain via `module` only (see "External ghosts" above), not the same id; `TS_`-prefixed because the claim is this analyzer's own | | `TS_UNRESOLVED_IMPORT` | TSApplication → TSExternal (module-level ghost) | prov[] | undeclared-import hygiene signal | | `TS_USES_CONFIG` | TSBodyNode → ConfigKey | prov[] | which read joins which key; prov ⊆ {literal (L2+), dataflow (L3 intra, L4 interproc — same tag both tiers)}; superset-monotonic `-a 2 ⊆ 3 ⊆ 4` | - -There is **no relationship for unresolved config reads** — `config_reads` (JSON: `site`, `callee`, -`key?`, `reason`, `prov[]`) is not projected; it records an absence, not a graph fact. There is also -**no import-graph relationship** — a module's `imports[]`/`exports[]` (with specifiers, aliases, -type-only flags) exist only in `analysis.json`'s `TSModule`; `TS_UNRESOLVED_IMPORT`/`TS_PROVIDES` -cover the dependency-hygiene case only, not a general per-module import graph. There is also **no -entrypoint vocabulary** — `TSCallable` carries no `is_entrypoint`/`entrypoint_frameworks`; this -analyzer does not (yet) detect framework entrypoints. +| `TS_READS_CONFIG_UNRESOLVED` | TSApplication → TSExternal (read-root ghost `@external/process.env`) / TSCallable (call-rule callee) | key?, reason, prov[], `_k` | a recognized read that closed on no declared key (`config_reads`); `_k` = `"\|"`, NO per-site identity — several sites collapse onto one edge (python's documented ceiling); SHRINKS as `-a` rises | +| `TS_IMPORTS` | TSModule → TSModule (resolved) / TSExternal (`@external/`, builtins `@external/node:fs`) | spellings[], imported_names[], aliases[], type_only_names[] | ONE per (module, target), every binding folded in; `imported_names` has `"*"` for a namespace import and nothing for a side-effect import; unresolved *relative* spellings are absent (JSON keeps them); level-free | +| `TS_RE_EXPORTS` | TSModule → TSModule / TSExternal | spellings[], exported_names[], aliases[], type_only_names[] | `export … from` only, same aggregation as `TS_IMPORTS`; `exported_names` has `"*"` for `export * from`; a local `export { x as y }` is in `TSModule.exports_json`, never an edge; level-free | + +`TSModule.exports_json` is the verbatim `exports[]` list (absent when empty); there is no +`imports_json` — per-binding import spans and kinds are `analysis.json`-only. `TSCallable.parameters_json` +is the verbatim `parameters[]` list (absent when empty), python's encoding. Entrypoints: +`is_entrypoint` / `entrypoint_frameworks` on `TSCallable` and `TSClass`, the report on `TSApplication`. All dataflow relationships are stored src→dst in the forward direction. diff --git a/schema.neo4j.json b/schema.neo4j.json index 34e6502..33728d4 100644 --- a/schema.neo4j.json +++ b/schema.neo4j.json @@ -80,6 +80,7 @@ "is_tsx": "boolean", "is_declaration_file": "boolean", "content_hash": "string", + "exports_json": "string", "start_line": "integer", "end_line": "integer" } @@ -196,7 +197,8 @@ "start_line": "integer", "end_line": "integer", "is_entrypoint": "boolean", - "entrypoint_frameworks": "string[]" + "entrypoint_frameworks": "string[]", + "parameters_json": "string" } }, { @@ -357,6 +359,54 @@ "prov": "string[]" } }, + { + "type": "TS_READS_CONFIG_UNRESOLVED", + "from": [ + "TSApplication" + ], + "to": [ + "TSExternal", + "TSCallable" + ], + "properties": { + "key": "string", + "reason": "string", + "prov": "string[]", + "_k": "string" + } + }, + { + "type": "TS_IMPORTS", + "from": [ + "TSModule" + ], + "to": [ + "TSModule", + "TSExternal" + ], + "properties": { + "spellings": "string[]", + "imported_names": "string[]", + "aliases": "string[]", + "type_only_names": "string[]" + } + }, + { + "type": "TS_RE_EXPORTS", + "from": [ + "TSModule" + ], + "to": [ + "TSModule", + "TSExternal" + ], + "properties": { + "spellings": "string[]", + "exported_names": "string[]", + "aliases": "string[]", + "type_only_names": "string[]" + } + }, { "type": "TS_DECLARES", "from": [ diff --git a/src/build/neo4j/project.ts b/src/build/neo4j/project.ts index 33ffdca..d788f1f 100644 --- a/src/build/neo4j/project.ts +++ b/src/build/neo4j/project.ts @@ -11,6 +11,7 @@ * for the incremental writer's per-module isolation); shared nodes (External) carry none. */ +import { specifierRoot } from "../../artifacts/binding"; import type { TSAnalysis, TSApplication, TSBodyNode, TSCallable, TSDecorator, TSEntrypoint, TSEntrypointReport, TSField, TSModule, TSType } from "../../schema"; import { globalOrdinal, purlNpm } from "../../schema/ids"; import { SCHEMA_VERSION } from "./schema"; @@ -133,12 +134,37 @@ export function project(app: TSAnalysis, _appName?: string): GraphRows { b.edge("TS_UNRESOLVED_IMPORT", appRef, importGhost(u.module), prune({ prov: u.prov.length ? u.prov : null })); } } + // Module-level binding edges (#182): after the artifact layer, so externals share its ghosts. + for (const mod of Object.values(root.symbol_table)) projectBindings(b, root, mod, importGhost); + // config_use literal/dataflow tier (#101 unit C2/C3): src is a body-node ordinal id already // projected as a :CanNode above; dst is a :ConfigKey id, already projected in the artifact - // loop. config_reads stay JSON-only — they record absence, not an edge. + // loop. for (const u of root.config_uses ?? []) { b.edge("TS_USES_CONFIG", ref(u.src), { label: "ConfigKey", keyProp: "id", value: u.dst }, prune({ prov: u.prov })); } + // config_reads (#182, python PY_READS_CONFIG_UNRESOLVED): application → the read's target. An + // env-root read (`process.env`, `import.meta.env`, `Bun.env`) has no callee node, so it ghosts + // under the root's spelling; a call-rule read names its resolved callee id — an external ghost + // or an in-project callable — and is gated on that node existing this run. `_k = key|reason`; + // one row per distinct (target, key, reason), since several sites collapse onto it anyway. + { + // Fold the collapsed sites' `prov` together, so a triple read at both the literal and a + // dataflow tier keeps both tags rather than whichever site came first. + const folded = new Map }>(); + for (const r of root.config_reads ?? []) { + const k = `${r.key ?? ""}|${r.reason}`; + const id = `${r.callee}\0${k}`; + const f = folded.get(id) ?? { r, k, prov: new Set() }; + for (const p of r.prov) f.prov.add(p); + folded.set(id, f); + } + for (const { r, k, prov } of folded.values()) { + const props = prune({ key: r.key ?? null, reason: r.reason, prov: prov.size ? [...prov].sort() : null }); + if (r.callee.startsWith("can://")) b.edgeToSymbol("TS_READS_CONFIG_UNRESOLVED", appRef, r.callee, props, k); + else b.edge("TS_READS_CONFIG_UNRESOLVED", appRef, importGhost(r.callee), props, k); + } + } // External library targets (shared nodes — no _module). for (const ext of Object.values(root.external_symbols ?? {})) { @@ -246,6 +272,49 @@ function projectDecorator(b: RowBuilder, on: NodeRef, d: TSDecorator): void { }); } +/** + * TS_IMPORTS / TS_RE_EXPORTS (#182, python `_project_imports` parity): ONE edge per (module, + * target), every binding of that pair folded into sorted arrays — the writers MERGE on the endpoint + * pair and SET props, so a second row for the same pair would silently overwrite the first. + * A resolved spelling lands on the real :TSModule (only if this run emitted it — a skipped test + * file is not a node); an external one on the `@external/` ghost the dependency + * layer already addresses (builtins under their own spelling, `node:fs`); a relative spelling that + * resolved to nothing has no node to land on and is dropped here — the JSON keeps it. + */ +function projectBindings(b: RowBuilder, root: TSApplication, mod: TSModule, ghost: (name: string) => NodeRef): void { + const from = ref(mod.id); + const targetOf = (spec: string, resolved: string | undefined): NodeRef | null => { + if (resolved !== undefined) { + const t = root.symbol_table[resolved]; + return t ? ref(t.id) : null; + } + if (/^[./#]/.test(spec)) return null; + return ghost(specifierRoot(spec) ?? spec); + }; + type Binding = { module?: string; resolved_module?: string; name: string; alias?: string; is_type_only: boolean }; + type Bucket = { to: NodeRef; spellings: Set; names: Set; aliases: Set; typeOnly: Set }; + const aggregate = (items: Binding[]): Bucket[] => { + const buckets = new Map(); + for (const it of items) { + if (it.module === undefined) continue; // a local `export { x as y }` binds nothing to another module + const to = targetOf(it.module, it.resolved_module); + if (!to) continue; + let bk = buckets.get(to.value); + if (!bk) buckets.set(to.value, (bk = { to, spellings: new Set(), names: new Set(), aliases: new Set(), typeOnly: new Set() })); + bk.spellings.add(it.module); + if (it.name) bk.names.add(it.name); // "" = side-effect import: the edge alone records the dependency + if (it.alias) bk.aliases.add(it.alias); + if (it.is_type_only && it.name) bk.typeOnly.add(it.name); + } + return [...buckets.values()]; + }; + const list = (s: Set): string[] | null => (s.size ? [...s].sort() : null); + for (const bk of aggregate(mod.imports ?? [])) + b.edge("TS_IMPORTS", from, bk.to, prune({ spellings: list(bk.spellings), imported_names: list(bk.names), aliases: list(bk.aliases), type_only_names: list(bk.typeOnly) })); + for (const bk of aggregate(mod.exports ?? [])) + b.edge("TS_RE_EXPORTS", from, bk.to, prune({ spellings: list(bk.spellings), exported_names: list(bk.names), aliases: list(bk.aliases), type_only_names: list(bk.typeOnly) })); +} + /** Key-sorted shallow copy, so the encoded JSON is stable across runs (python sorts too). */ function sortedKeys(o: Record): Record { const out: Record = {}; @@ -290,6 +359,9 @@ function moduleProps(mod: TSModule, fileKey: string): Props { return prune({ id: mod.id, kind: "module", name: fileKey, content_hash: mod.content_hash ?? null, is_tsx: mod.is_tsx, is_declaration_file: mod.is_declaration_file, + // #182: the lossless export list (per-binding spans, `export { x as y }` locals) — the + // TS_RE_EXPORTS edges carry only the aggregated cross-module part. Absent when empty. + exports_json: mod.exports?.length ? JSON.stringify(mod.exports) : null, ...span(mod), _module: fileKey, }); } @@ -330,6 +402,9 @@ function callableProps(c: TSCallable, fileKey: string, source: string): Props { is_async: c.is_async, is_generator: c.is_generator, is_exported: c.is_exported, is_ambient: c.is_ambient, is_implicit: c.is_implicit, code: spanCode(source, c.span), ...span(c), _module: fileKey, + // #182 (python `parameters_json`): the parameter list verbatim as a JSON string — Neo4j has no + // nested-map property — and absent, not "[]", when there are none. The SDK decodes it as-is. + parameters_json: c.parameters?.length ? JSON.stringify(c.parameters) : null, }); } diff --git a/src/build/neo4j/rows.ts b/src/build/neo4j/rows.ts index f1c6b95..223e188 100644 --- a/src/build/neo4j/rows.ts +++ b/src/build/neo4j/rows.ts @@ -152,12 +152,13 @@ export class RowBuilder { * `:CanNode` this run — so EXTENDS / IMPLEMENTS never dangle (the id is already resolved-only by * the time it reaches here; this is the defense-in-depth gate, not the primary resolution step). */ - edgeToSymbol(type: string, from: NodeRef, targetId: string, props: Props = {}): void { + edgeToSymbol(type: string, from: NodeRef, targetId: string, props: Props = {}, key?: string): void { this.deferred.push({ type, from, to: { label: CAN_NODE, keyProp: "id", value: targetId }, props, + key, }); } diff --git a/src/build/neo4j/schema.ts b/src/build/neo4j/schema.ts index fc4d569..355503c 100644 --- a/src/build/neo4j/schema.ts +++ b/src/build/neo4j/schema.ts @@ -113,7 +113,9 @@ export const NODE_LABELS: NodeLabel[] = [ label: "TSModule", mergeLabel: CAN, key: "id", - properties: { ...COMMON, name: "string", is_tsx: "boolean", is_declaration_file: "boolean", content_hash: "string", ...SPAN }, + // `exports_json` (#182): the module's export list as a JSON string, absent when empty; the + // cross-module part is also TS_RE_EXPORTS edges. Imports are edges only (TS_IMPORTS). + properties: { ...COMMON, name: "string", is_tsx: "boolean", is_declaration_file: "boolean", content_hash: "string", exports_json: "string", ...SPAN }, }, { label: "TSClass", @@ -159,6 +161,7 @@ export const NODE_LABELS: NodeLabel[] = [ is_async: "boolean", is_generator: "boolean", is_exported: "boolean", is_ambient: "boolean", is_implicit: "boolean", code: "string", ...SPAN, is_entrypoint: "boolean", entrypoint_frameworks: "string[]", // #72 (python PyCallable parity) + parameters_json: "string", // #182 (python PyCallable parity): the parameter list, absent when empty }, }, { label: "TSField", mergeLabel: CAN, key: "id", properties: { ...COMMON, name: "string", type: "string", ...SPAN } }, @@ -204,6 +207,35 @@ export const REL_TYPES: RelType[] = [ { type: "TS_UNRESOLVED_IMPORT", from: ["TSApplication"], to: ["TSExternal"], properties: { prov: "string[]" } }, { type: "DEFINES_CONFIG", from: ["Artifact"], to: ["ConfigKey"], properties: {} }, { type: "TS_USES_CONFIG", from: ["TSBodyNode"], to: ["ConfigKey"], properties: { prov: "string[]" } }, + // A detector-matched config read that never closed on exactly one declared key (#182, python + // PY_READS_CONFIG_UNRESOLVED): application → the read root's ghost (`@external/process.env`) or + // the resolved callee of a call-rule read. `_k` discriminates by (key, reason): one callee + // legitimately reads several distinct undeclared/dynamic keys, and a plain endpoint-pair MERGE + // would keep only the last. Per-site identity is NOT carried (python's documented ceiling). + { + type: "TS_READS_CONFIG_UNRESOLVED", + from: ["TSApplication"], + to: ["TSExternal", "TSCallable"], + properties: { key: "string", reason: "string", prov: "string[]", _k: "string" }, + }, + // Module-level binding edges (#182, python PY_IMPORTS parity): ONE relationship per + // (module, target) aggregating every binding of that pair — the writers MERGE on the endpoint + // pair and SET props, so a second row for the same pair would overwrite the first. Resolved + // targets are the real :TSModule; externals land on the `/@external/` ghost + // that TS_PROVIDES / TS_UNRESOLVED_IMPORT already address. `type_only_names[]` is the TS-only + // addition: the bindings that are `import type` / `{ type X }` / `export type`. + { + type: "TS_IMPORTS", + from: ["TSModule"], + to: ["TSModule", "TSExternal"], + properties: { spellings: "string[]", imported_names: "string[]", aliases: "string[]", type_only_names: "string[]" }, + }, + { + type: "TS_RE_EXPORTS", + from: ["TSModule"], + to: ["TSModule", "TSExternal"], + properties: { spellings: "string[]", exported_names: "string[]", aliases: "string[]", type_only_names: "string[]" }, + }, { type: "TS_DECLARES", from: ["TSModule", "TSNamespace", "TSCallable"], diff --git a/src/schema/schema.ts b/src/schema/schema.ts index e517786..de643a6 100644 --- a/src/schema/schema.ts +++ b/src/schema/schema.ts @@ -44,6 +44,13 @@ export interface TSImport { module: string; // the module specifier, e.g. "./user" or "@nestjs/common" name: string; // the imported binding (or "" for side-effect imports / "*" for namespace) alias?: string; + // #182 (python `PyImport.resolved_module` parity): the project-relative file key the specifier + // resolves to under the importer's own tsconfig (`paths`, directory index, `.js` → `.ts`) — + // whether or not that module was emitted this run (`--skip-tests` / `--program` can exclude + // the target; the graph gates on presence, the JSON keeps the key). Re-stamped EVERY run, + // cached modules included (moduleResolution.ts). ABSENT for externals, builtins, and spellings + // that resolve to nothing. + resolved_module?: string; is_type_only: boolean; // `import type { X } ...` import_kind: "named" | "default" | "namespace" | "side_effect"; start_line: number; @@ -56,6 +63,7 @@ export interface TSExport { module?: string; // re-export source, e.g. "./user"; absent for `export { x }` name: string; // exported name ("*" for `export * from`) alias?: string; + resolved_module?: string; // #182: as on TSImport; only a re-export (`module` set) can carry one is_type_only: boolean; export_kind: "named" | "default" | "namespace" | "re_export"; start_line: number; diff --git a/src/syntactic_analysis/builders.ts b/src/syntactic_analysis/builders.ts index 3ec5287..4a000b7 100644 --- a/src/syntactic_analysis/builders.ts +++ b/src/syntactic_analysis/builders.ts @@ -988,6 +988,8 @@ export function buildNamespace(ns: Node, root: string): { sig: string; ns: TSTyp // imports / exports / comments // ---------------------------------------------------------------------------------------------- +// `resolved_module` is NOT stamped here: it is a per-run pass (moduleResolution.ts, #182) because +// the answer depends on the tsconfig and on which files exist — state the module cache cannot see. function buildImports(sf: Node): TSImport[] { const out: TSImport[] = []; const decls = (sf as unknown as { getImportDeclarations: () => Node[] }).getImportDeclarations(); @@ -1053,13 +1055,14 @@ function buildExports(sf: Node): TSExport[] { }); } for (const ne of named) { - const n = ne as unknown as { getName: () => string; getAliasNode?: () => { getText: () => string } | undefined }; + const n = ne as unknown as { getName: () => string; getAliasNode?: () => { getText: () => string } | undefined; isTypeOnly?: () => boolean }; const alias = n.getAliasNode?.()?.getText(); out.push({ ...(module != null ? { module } : {}), name: n.getName(), ...(alias != null ? { alias } : {}), - is_type_only: typeOnly, + // per-specifier `export { type X }` counts as much as the `export type { }` list form + is_type_only: typeOnly || (n.isTypeOnly?.() ?? false), export_kind: module ? "re_export" : "named", ...s, }); diff --git a/src/syntactic_analysis/moduleResolution.ts b/src/syntactic_analysis/moduleResolution.ts new file mode 100644 index 0000000..1a1bbf4 --- /dev/null +++ b/src/syntactic_analysis/moduleResolution.ts @@ -0,0 +1,62 @@ +/** + * `resolved_module` on every import / re-export binding (#182; python `resolve_imports` parity). + * + * A PER-RUN stamp, never a build-time fact: the answer depends on state the content-hash cache + * cannot see — the owning tsconfig (`paths`, `baseUrl`, `moduleResolution`) and whether the target + * file exists — so an importer that is byte-identical across runs still needs re-resolving. Every + * module, cached or freshly built, is re-stamped here: set when the specifier resolves to a file + * inside the project root, DELETED otherwise, so a value from an older run never survives. + * + * `ts.resolveModuleName` rather than ts-morph's `getModuleSpecifierSourceFile()`: the latter goes + * through the module SYMBOL, which a side-effect-imported script with no import/export of its own + * never has (`import "./polyfill"` would come back unresolved). The compiler's resolver answers + * every specifier the way tsc itself does, under the importer's own program. + */ +import type { Project } from "ts-morph"; +import { ts } from "ts-morph"; +import { type TSModule, fileKeyOf } from "../schema"; + +interface Binding { + module?: string; + resolved_module?: string; +} + +export function stampResolvedModules( + symbol_table: Record, + files: Array<{ absPath: string; fileKey: string }>, + projectOf: (absPath: string) => Project, + root: string, +): void { + // One resolution cache per program (a repository's imports repeat the same few hundred + // specifiers from the same few directories, and every miss is several stat calls). + const caches = new Map(); + for (const f of files) { + const mod = symbol_table[f.fileKey]; + if (!mod) continue; + const project = projectOf(f.absPath); + let cache = caches.get(project); + if (!cache) { + const host = project.getModuleResolutionHost(); + cache = ts.createModuleResolutionCache(host.getCurrentDirectory?.() ?? root, (x) => x, project.getCompilerOptions()); + caches.set(project, cache); + } + const stamp = (b: Binding): void => { + const key = b.module === undefined ? undefined : resolve(b.module, f.absPath, project, cache!, root); + if (key === undefined) delete b.resolved_module; + else b.resolved_module = key; + }; + for (const im of mod.imports ?? []) stamp(im); + for (const ex of mod.exports ?? []) stamp(ex); + } +} + +/** The project-relative file key the specifier resolves to, or undefined for anything external. */ +function resolve(spec: string, importer: string, project: Project, cache: ts.ModuleResolutionCache, root: string): string | undefined { + const hit = ts.resolveModuleName(spec, importer, project.getCompilerOptions(), project.getModuleResolutionHost(), cache).resolvedModule; + if (!hit || hit.isExternalLibraryImport) return undefined; + const key = fileKeyOf(hit.resolvedFileName, root).fileKey; + // Outside the root, or inside node_modules without tsc flagging it: an external, addressed by + // the dependency layer (TS_PROVIDES / TS_UNRESOLVED_IMPORT), never a symbol-table key. + if (key.startsWith("../") || key.startsWith("/") || /(^|\/)node_modules\//.test(key)) return undefined; + return key; +} diff --git a/src/syntactic_analysis/symbolTable.ts b/src/syntactic_analysis/symbolTable.ts index f6296f2..0a385b5 100644 --- a/src/syntactic_analysis/symbolTable.ts +++ b/src/syntactic_analysis/symbolTable.ts @@ -1,6 +1,7 @@ import * as path from "node:path"; import { Project, ts } from "ts-morph"; import { buildModule } from "./builders"; +import { stampResolvedModules } from "./moduleResolution"; import { fileMeta, fileUnchanged } from "../utils"; import { discoverSourceFiles, resolveTargetFiles, type DiscoveredFile } from "./discovery"; import type { Materialization, ProgramSpec } from "../build"; @@ -155,6 +156,11 @@ export function buildSymbolTable( } log.info(`symbol table: ${built} built, ${fromCache} cached, ${Object.keys(symbol_table).length} modules`); + // #182: `resolved_module` is re-stamped on EVERY module each run, cached ones included — the + // answer depends on the tsconfig and on which files exist, neither of which the per-file + // content hash sees (python re-resolves per run for the same reason). + stampResolvedModules(symbol_table, buildFiles, (abs) => projectOf.get(ownerProgram(abs, specs))!, root); + // The root program is always last; its Project is the one legacy single-program consumers expect. // Under --program the root may not be selected, so fall back to the shallowest SELECTED program // (the list is deepest-first, so that is its last entry). diff --git a/test/neo4j-bindings.test.ts b/test/neo4j-bindings.test.ts new file mode 100644 index 0000000..977bbd9 --- /dev/null +++ b/test/neo4j-bindings.test.ts @@ -0,0 +1,206 @@ +/** + * #182 — import/export bindings, callable parameters and unresolved config reads reach the graph + * (spec: docs/design/specs/neo4j-bindings-parameters-config.md, decisions D1–D5). + * + * D1: `resolved_module` is the checker's answer (tsconfig `paths`, directory index, `.js` → `.ts`), + * absent for externals, builtins and spellings that resolve to nothing. + * D2/D3: one TS_IMPORTS / TS_RE_EXPORTS edge per (module, target) aggregating every binding; + * externals land on the `/@external/` ghost; unresolved relative spellings are + * dropped from the graph (they survive in JSON); `exports_json` is the lossless carrier. + * D4: `parameters_json` is python's encoding — the list verbatim, absent when empty. + * D5: TS_READS_CONFIG_UNRESOLVED mirrors PY_READS_CONFIG_UNRESOLVED, `_k = key|reason`. + */ +import { describe, expect, test } from "bun:test"; +import * as fs from "node:fs"; +import * as os from "node:os"; +import * as path from "node:path"; +import { project } from "../src/build/neo4j"; +import type { EdgeRow, GraphRows } from "../src/build/neo4j/rows"; +import { analyze } from "../src/core"; +import type { AnalysisOptions } from "../src/options"; +import type { TSAnalysis, TSCallable, TSModule } from "../src/schema"; + +const dir = fs.mkdtempSync(path.join(os.tmpdir(), "cants-bindings-")); +const w = (rel: string, lines: string[]) => { + fs.mkdirSync(path.dirname(path.join(dir, rel)), { recursive: true }); + fs.writeFileSync(path.join(dir, rel), lines.join("\n")); +}; +w("tsconfig.json", [ + JSON.stringify({ + compilerOptions: { target: "ES2020", module: "ESNext", moduleResolution: "node", baseUrl: ".", paths: { "@lib/*": ["src/lib/*"] } }, + include: ["src/**/*.ts"], + }), +]); +w("src/lib/util.ts", [ + "export function helper(x: number, y?: string, ...rest: unknown[]): number { return x; }", + "export const a = 1;", +]); +w("src/models/index.ts", ["export interface User { id: string }", "export type UserId = string;", "export class Robot {}"]); +w("src/x.ts", ["export const x = 1;", "export function noargs(): void {}"]); +w("src/side.ts", ['console.log("side");']); +w("src/index.ts", [ + 'import { helper, a as aa } from "@lib/util";', + 'import { Robot, type User } from "./models";', + 'import type { UserId } from "./models";', + 'import { x } from "./x.js";', + 'import "./side";', + 'import { z } from "zod";', + 'import * as fs from "node:fs";', + 'import { gone } from "./missing";', + 'export * from "./models";', + 'export { helper as help, a } from "@lib/util";', + 'export { type User as U } from "./models";', + "const local = 1;", + "export { local as renamed };", + "export function main(id: UserId, u?: User): number { return helper(x, undefined, id, u, fs, z, gone, aa, new Robot()); }", +]); +const opts = { input: dir, appName: "bd", analysisLevel: 1, noBuild: true, emit: "json" } as unknown as AnalysisOptions; + +const res = await analyze(opts); +const app = (res.application as TSAnalysis).application; +const mod = (key: string): TSModule => app.symbol_table[key] as TSModule; +const index = mod("src/index.ts"); +const rows: GraphRows = project(res.application as TSAnalysis); +const edgesOf = (type: string, from: string): EdgeRow[] => rows.edges.filter((e) => e.type === type && e.from.value === from); +const toMap = (edges: EdgeRow[]) => new Map(edges.map((e) => [e.to.value, e])); + +describe("D1 resolved_module (JSON)", () => { + const byModule = new Map(index.imports.map((i) => [i.module, i])); + test("tsconfig paths alias, directory index and .js → .ts all resolve to the symbol-table key", () => { + expect(byModule.get("@lib/util")?.resolved_module).toBe("src/lib/util.ts"); + expect(byModule.get("./models")?.resolved_module).toBe("src/models/index.ts"); + expect(byModule.get("./x.js")?.resolved_module).toBe("src/x.ts"); + expect(byModule.get("./side")?.resolved_module).toBe("src/side.ts"); + }); + test("externals, builtins and a missing relative target carry no resolved_module", () => { + expect(byModule.get("zod")?.resolved_module).toBeUndefined(); + expect(byModule.get("node:fs")?.resolved_module).toBeUndefined(); + expect(byModule.get("./missing")?.resolved_module).toBeUndefined(); + expect(byModule.has("./missing")).toBe(true); // the spelling itself survives + }); + test("re-exports resolve; a local export list has no module and no resolution", () => { + const star = index.exports.find((e) => e.name === "*"); + expect(star?.module).toBe("./models"); + expect(star?.resolved_module).toBe("src/models/index.ts"); + expect(index.exports.find((e) => e.name === "helper")?.resolved_module).toBe("src/lib/util.ts"); + const renamed = index.exports.find((e) => e.alias === "renamed"); + expect(renamed?.module).toBeUndefined(); + expect(renamed?.resolved_module).toBeUndefined(); + }); +}); + +describe("D2 TS_IMPORTS", () => { + const imports = toMap(edgesOf("TS_IMPORTS", index.id)); + test("one edge per (importer, resolved module), aggregating every binding of that pair", () => { + const models = imports.get(mod("src/models/index.ts").id); + expect(models?.props).toEqual({ spellings: ["./models"], imported_names: ["Robot", "User", "UserId"], type_only_names: ["User", "UserId"] }); + const util = imports.get(mod("src/lib/util.ts").id); + expect(util?.props).toEqual({ spellings: ["@lib/util"], imported_names: ["a", "helper"], aliases: ["aa"] }); + expect(imports.get(mod("src/x.ts").id)?.props).toEqual({ spellings: ["./x.js"], imported_names: ["x"] }); + // a side-effect import has no names: the edge still records the dependency + expect(imports.get(mod("src/side.ts").id)?.props).toEqual({ spellings: ["./side"] }); + }); + test("externals and builtins land on the @external ghost; an unresolved relative spelling is dropped", () => { + expect(imports.get(`${app.id}/@external/zod`)?.props).toEqual({ spellings: ["zod"], imported_names: ["z"] }); + expect(imports.get(`${app.id}/@external/node:fs`)?.props).toEqual({ spellings: ["node:fs"], imported_names: ["*"], aliases: ["fs"] }); + expect([...imports.keys()].some((k) => k.includes("missing"))).toBe(false); + expect(imports.size).toBe(6); + const ghost = rows.nodes.find((n) => n.value === `${app.id}/@external/node:fs`); + expect(ghost?.labels).toContain("TSExternal"); + expect(ghost?.props.module).toBe("node:fs"); + }); +}); + +describe("D3 exports", () => { + test("TS_RE_EXPORTS aggregates per target, star included", () => { + const re = toMap(edgesOf("TS_RE_EXPORTS", index.id)); + expect(re.get(mod("src/models/index.ts").id)?.props).toEqual({ spellings: ["./models"], exported_names: ["*", "User"], aliases: ["U"], type_only_names: ["User"] }); + expect(re.get(mod("src/lib/util.ts").id)?.props).toEqual({ spellings: ["@lib/util"], exported_names: ["a", "helper"], aliases: ["help"] }); + expect(re.size).toBe(2); + }); + test("exports_json is the module's exports list verbatim, absent when empty", () => { + const node = rows.nodes.find((n) => n.value === index.id); + const parsed = JSON.parse(node?.props.exports_json as string); + expect(parsed).toEqual(index.exports); + expect(parsed).toHaveLength(5); + expect(rows.nodes.find((n) => n.value === mod("src/side.ts").id)?.props.exports_json).toBeUndefined(); + }); +}); + +describe("D4 parameters_json", () => { + const callable = (m: TSModule, sig: string): TSCallable => Object.values(m.functions).find((c) => c.signature.endsWith(sig)) as TSCallable; + test("python's encoding: the parameter list verbatim", () => { + const helper = callable(mod("src/lib/util.ts"), ".helper"); + const node = rows.nodes.find((n) => n.value === helper.id); + const parsed = JSON.parse(node?.props.parameters_json as string); + expect(parsed).toEqual(helper.parameters); + expect(parsed.map((p: { name: string }) => p.name)).toEqual(["x", "y", "rest"]); + expect(parsed[1].is_optional).toBe(true); + expect(parsed[2].is_rest).toBe(true); + }); + test("absent, not '[]', on a callable without parameters", () => { + const noargs = callable(mod("src/x.ts"), ".noargs"); + expect(rows.nodes.find((n) => n.value === noargs.id)?.props.parameters_json).toBeUndefined(); + }); +}); + +describe("D5 TS_READS_CONFIG_UNRESOLVED", () => { + const FIXTURE = path.resolve(import.meta.dir, "fixtures/artifacts-app"); + test("one edge per distinct (callee, key, reason), keyed key|reason, from the application", async () => { + const cacheDir = fs.mkdtempSync(path.join(os.tmpdir(), "cants-bindings-cfg-")); + const r = await analyze({ input: FIXTURE, appName: "art", analysisLevel: 2, noBuild: true, emit: "json", eager: true, cacheDir } as unknown as AnalysisOptions); + fs.rmSync(cacheDir, { recursive: true, force: true }); + const a = (r.application as TSAnalysis).application; + expect(a.config_reads.length).toBeGreaterThan(0); + const g = project(r.application as TSAnalysis); + const edges = g.edges.filter((e) => e.type === "TS_READS_CONFIG_UNRESOLVED"); + const triples = new Set(a.config_reads.map((c) => `${c.callee}\0${c.key ?? ""}|${c.reason}`)); + expect(edges.length).toBe(triples.size); + for (const e of edges) { + expect(e.from.value).toBe(a.id); + expect(e.key).toBe(`${e.props.key ?? ""}|${e.props.reason}`); + expect(e.props.prov).toEqual(["literal"]); + } + // an env-root read targets the root's ghost; the ghost is a real node this run + const envRead = a.config_reads.find((c) => c.callee === "process.env"); + expect(envRead).toBeDefined(); + const ghostId = `${a.id}/@external/process.env`; + expect(edges.some((e) => e.to.value === ghostId)).toBe(true); + expect(g.nodes.find((n) => n.value === ghostId)?.labels).toContain("TSExternal"); + }); +}); + +describe("D1 is a per-run stamp, not a cached fact", () => { + // The importer never changes between runs, so its module comes back from the content-hash + // cache each time; only the world around it moves. A build-time value would be stale (run 2) + // and then WRONG (run 3), silently. + const proj = fs.mkdtempSync(path.join(os.tmpdir(), "cants-bindings-cache-")); + const cacheDir = fs.mkdtempSync(path.join(os.tmpdir(), "cants-bindings-cache-dir-")); + const write = (rel: string, text: string) => { + fs.mkdirSync(path.dirname(path.join(proj, rel)), { recursive: true }); + fs.writeFileSync(path.join(proj, rel), text); + }; + const tsconfig = (libDir: string) => + JSON.stringify({ compilerOptions: { target: "ES2020", module: "ESNext", moduleResolution: "node", baseUrl: ".", paths: { "@lib/*": [`src/${libDir}/*`] } }, include: ["src/**/*.ts"] }); + const run = async () => { + const r = await analyze({ input: proj, appName: "cc", analysisLevel: 1, noBuild: true, emit: "json", cacheDir } as unknown as AnalysisOptions); + const m = (r.application as TSAnalysis).application.symbol_table["src/index.ts"] as TSModule; + return Object.fromEntries(m.imports.map((i) => [i.module, i.resolved_module])); + }; + test("a target added later resolves; a tsconfig paths edit re-points; the importer is cached throughout", async () => { + write("tsconfig.json", tsconfig("a")); + write("src/a/util.ts", "export const u = 1;"); + write("src/index.ts", ['import { u } from "@lib/util";', 'import { later } from "./later";', "export const v = u + later;"].join("\n")); + expect(await run()).toEqual({ "@lib/util": "src/a/util.ts", "./later": undefined }); + + write("src/later.ts", "export const later = 2;"); + expect(await run()).toEqual({ "@lib/util": "src/a/util.ts", "./later": "src/later.ts" }); + + write("tsconfig.json", tsconfig("b")); + write("src/b/util.ts", "export const u = 3;"); + expect(await run()).toEqual({ "@lib/util": "src/b/util.ts", "./later": "src/later.ts" }); + + fs.rmSync(path.join(proj, "src/later.ts")); + expect(await run()).toEqual({ "@lib/util": "src/b/util.ts", "./later": undefined }); + }); +}); diff --git a/test/neo4j-bolt.test.ts b/test/neo4j-bolt.test.ts index 0d25c7b..819e21d 100644 --- a/test/neo4j-bolt.test.ts +++ b/test/neo4j-bolt.test.ts @@ -140,6 +140,46 @@ containerSuite("neo4j bolt writer", () => { 120_000, ); + test( + "binding edges land, and an import edge to a vanished target goes with the target under --eager (#182)", + async () => { + const opts = optsFor(); + const result = await analyze(opts); + const full = project(finalizeAnalysis(result.internal, result.program_graphs ?? null, opts).application); + await boltWriter(full, cfg, log, true, true); + const count = (t: string) => full.edges.filter((e) => e.type === t).length; + expect(count("TS_IMPORTS")).toBeGreaterThan(0); + expect(await num("MATCH (:TSModule)-[r:TS_IMPORTS]->() RETURN count(r)")).toBe(count("TS_IMPORTS")); + // index.ts imports ./models: a resolved edge lands on the real module, names aggregated + expect( + await num("MATCH (m:TSModule {name:'src/index.ts'})-[r:TS_IMPORTS]->(t:TSModule {name:'src/models.ts'}) WHERE 'User' IN r.imported_names RETURN count(r)"), + ).toBe(1); + // an external lands on the dependency layer's ghost, under the application prefix + expect(await num("MATCH (:TSModule)-[:TS_IMPORTS]->(x:TSExternal {module:'commander'}) RETURN count(x)")).toBe(1); + expect(await num("MATCH (c:TSCallable {name:'create'}) WHERE c.parameters_json STARTS WITH '[{' RETURN count(c)")).toBeGreaterThan(0); + + // models.ts vanishes. Its importers (index.ts, services.ts) are UNCHANGED modules, so the + // incremental diff never rewrites their edges: the stale TS_IMPORTS edge to the victim + // survives a default push exactly like the victim's own nodes do (#116's rule) — that + // `> 0` is the load-bearing assertion. Under --eager the application is wiped and rebuilt + // from the reduced rows, so the edge is gone with the victim; the final count pins that the + // rebuilt import graph is exactly the reduced projection's — nothing dangles. + const app = result.internal; + delete app.symbol_table["src/models.ts"]; + const reduced = project(finalizeAnalysis(app, result.program_graphs ?? null, opts).application); + const appId = full.nodes.find((n) => n.labels[0] === "Application")!.value; + const victimId = `${appId}/src/models.ts`; + const intoVictim = () => num("MATCH ()-[r:TS_IMPORTS]->(t {id:$id}) RETURN count(r)", { id: victimId }); + expect(reduced.edges.filter((e) => e.type === "TS_IMPORTS" && e.to.value === victimId).length).toBe(0); + await boltWriter(reduced, cfg, log, true, false); + expect(await intoVictim()).toBeGreaterThan(0); + await boltWriter(reduced, cfg, log, true, true); + expect(await intoVictim()).toBe(0); + expect(await num("MATCH (:TSModule)-[r:TS_IMPORTS]->() RETURN count(r)")).toBe(reduced.edges.filter((e) => e.type === "TS_IMPORTS").length); + }, + 120_000, + ); + test( "a vanished module is pruned only under --eager (#116)", async () => { diff --git a/test/schema-v2.test.ts b/test/schema-v2.test.ts index 5a25c34..1c35d0a 100644 --- a/test/schema-v2.test.ts +++ b/test/schema-v2.test.ts @@ -15,6 +15,7 @@ import type { AnalysisOptions } from "../src/options"; import { forEachCallable, forEachType, type GraphSelector } from "../src/schema"; import type { AnalysisResult } from "../src/core"; import { type GraphRows, project } from "../src/build/neo4j"; +import { specifierRoot } from "../src/artifacts/binding"; import { tscProvider } from "../src/semantic_analysis"; const FIXTURE = path.resolve(import.meta.dir, "fixtures/sample-app"); @@ -713,6 +714,18 @@ function canNodeIds(app: TSAnalysis): Set { } for (const id of Object.keys(app.application.external_symbols ?? {})) ids.add(id); for (const id of Object.keys(app.application.synthesized_callables ?? {})) ids.add(id); + // #182: TS_IMPORTS / TS_RE_EXPORTS ghost every non-relative spelling that did not resolve + // in-project (package root, or the builtin's own spelling), and TS_READS_CONFIG_UNRESOLVED + // ghosts an env-root read's root — the same `@external/` id space as the artifact layer. + for (const m of Object.values(app.application.symbol_table)) { + for (const b of [...(m.imports ?? []), ...(m.exports ?? [])]) { + if (b.module === undefined || b.resolved_module !== undefined || /^[./#]/.test(b.module)) continue; + ids.add(`${app.application.id}/@external/${specifierRoot(b.module) ?? b.module}`); + } + } + for (const r of app.application.config_reads ?? []) { + if (!r.callee.startsWith("can://")) ids.add(`${app.application.id}/@external/${r.callee}`); + } // Repository-artifact layer (#101/PR-160 shape): artifacts/packages are NOT CanNodes (own // neutral merge labels) — but TS_PROVIDES / TS_UNRESOLVED_IMPORT mint module-level // :TSExternal ghosts in the CanNode id space. @@ -810,8 +823,35 @@ describe("neo4j ↔ json count parity — full depth (issue #27)", () => { ].reduce((n, t) => n + relCount(monoRows, t), 0); const resolvesTo = relCount(monoRows, "TS_RESOLVES_TO"); const heritage = relCount(monoRows, "TS_EXTENDS") + relCount(monoRows, "TS_IMPLEMENTS"); - expect(resolvesTo).toBe(resolvesToCount(monoApp4)); - expect(typedOverlay + containment + artifactLayer + resolvesTo + heritage).toBe(monoRows.edges.length); + // #182: module bindings and unresolved config reads are AGGREGATED families — one relationship + // per distinct (module, target) / (target, key, reason) — so their JSON source is a distinct + // count, not a list length. The aggregation rule is restated here on purpose (python's rule: + // resolved → the module, external → the package-root ghost, unresolved relative → dropped), + // so this is a CARDINALITY gate only — a wrong target with the right count passes here and is + // caught by neo4j-bindings.test.ts, which asserts endpoints and props. dataflow-app has + // imports but no re-exports and no unresolved config reads, so those two lines pin 0 === 0 + // on this fixture; their non-vacuous coverage also lives in neo4j-bindings.test.ts. + const app = monoApp4.application; + const targetKey = (spec: string, resolved: string | undefined): string | null => { + if (resolved !== undefined) return app.symbol_table[resolved] ? `mod:${resolved}` : null; + if (/^[./#]/.test(spec)) return null; + return `ext:${specifierRoot(spec) ?? spec}`; + }; + const pairs = (pick: (m: (typeof app.symbol_table)[string]) => Array<{ module?: string; resolved_module?: string }>) => { + const s = new Set(); + for (const [key, m] of Object.entries(app.symbol_table)) + for (const b of pick(m)) { + const t = b.module === undefined ? null : targetKey(b.module, b.resolved_module); + if (t) s.add(`${key}\0${t}`); + } + return s.size; + }; + expect(relCount(monoRows, "TS_IMPORTS")).toBe(pairs((m) => m.imports)); + expect(relCount(monoRows, "TS_RE_EXPORTS")).toBe(pairs((m) => m.exports)); + const unresolvedReads = new Set(app.config_reads.map((r) => `${r.callee}\0${r.key ?? ""}|${r.reason}`)).size; + expect(relCount(monoRows, "TS_READS_CONFIG_UNRESOLVED")).toBe(unresolvedReads); + const bindings = relCount(monoRows, "TS_IMPORTS") + relCount(monoRows, "TS_RE_EXPORTS") + relCount(monoRows, "TS_READS_CONFIG_UNRESOLVED"); + expect(typedOverlay + containment + artifactLayer + resolvesTo + heritage + bindings).toBe(monoRows.edges.length); }); test("DDG/CFG_NEXT parity survives the writers: every row keyed, keys fully discriminate (issue #70)", () => {