feat(java): leg 3b — the query surface for Java, addressing through interprocedural flow - #358
Merged
Merged
Conversation
…test (#337) The inject step downloaded codeanalyzer-java's latest release regardless of the pin in pyproject.toml, so 2.0.0rc2 shipped both codeanalyzer-2.4.1.jar (checked in) and codeanalyzer-3.0.1.jar (injected); JCodeanalyzer._locate_jar picked 2.4.1 only by lexical order. Read the pin, fetch codeanalyzer-<pin>.jar from releases/tags/v<pin>, replace whatever is under jar/, and make the bundle check require exactly that one jar in the wheel and the sdist. A pin without a matching asset now fails the release instead of silently bundling something else. Closes #336
Decisions J-1..J-12: string call-graph node keys, path key + package dotted vocabulary, name-or-signature callable resolution with overloads raising, CRUD raising on v2 until codeanalyzer-java #187, annotation markers, addressability of initializers/implicit/anonymous callables, leaf accessors, rich v1 fields as computed views, floor 3.0.1 with a generation-aware probe, source_code removal, two work items (#310 = 3a, #311 = 3b) in 2.0.0-rc.4.
Generated by the released codeanalyzer-3.0.1.jar on Temurin 21.0.5+11, never hand-edited. a1 is the whole application at -a 1 (138 units). a4 is -a 4 for the two beans, TradeDirect and TradeServletAction; --schema v2 rejects -t, so the restriction is a source-pruned copy of the tree analyzed with --no-build. The README carries the exact command lines and the measured contents.
….3.7 ones analysis_json now yields the v2/a1 (L1, whole application) text and a new analysis_json_a4 fixture yields v2/a4 (L4, four units). The two 2.3.7 fixtures go with them. Expected red until T1 lands the v2 models: 83 failed, 52 passed, 14 skipped -- every one the v1 JApplication refusing the v2 envelope (symbol_table Field required). Plan Task 0 ticked with the measured floors and the deviations (no -t at --schema v2; fixture sizes).
The analyzer pretty-prints; the raw a1 + a4 pair is 18.7 MB. gzip -9 brings them to 1.3 MB with no loss, and conftest reads them with gzip.open.
…es kept as views cldk/models/java is now an extra="forbid" pydantic mirror of the 3.0.1 envelope (JAnalysis -> JApplication -> JCompilationUnit -> JType -> JCallable, with body nodes, cfg/cdg/ddg/summary edges, param edges, artifacts and dependencies). The v1 field names callers read stay as computed views: code is the span slice of the unit source, call_sites are built from the call body nodes, thrown_exceptions/cyclomatic_complexity/ variable_declarations/referenced_types/accessed_fields read error_channel/ metrics/local_variables/refs, the JType predicates read kind and the owner chain threaded in model_post_init. JGraphEdges is now the wire JCallGraphEdge; the module-global callable lookup table is gone. Both 3.0.1 fixtures round-trip byte-equal through model_dump(exclude_unset, by_alias). tests/analysis/java is red at the backend sites until Task 2.
- code slices by UTF-8 byte offsets (span.bytes are getBytes(UTF_8) prefix sums), decoded; plain-index fast path for ASCII units - JCallable.code is the v1 value: the body block (body_span), falling back to span only without a body - node identity is the id on JType/JCallable/JField/JCompilationUnit so parses compare without walking the owner back-references - nested_type_declarations returns qualified names; annotations return the v1 source spelling (@name(args)) - a spanned node never threaded into a unit raises instead of returning ""; file_path raises when no JApplication stamped it - JBodyNode is spanned; one helper each for decorator names / first comment
…3.0.1, string call-graph keys, source_code removed JavaAnalysisBackend now parameterises AnalysisBackend[JApplication, JCompilationUnit, JType, JCallable, JField, JCallableParameter] with P = N = "J", so a query added to the shared shape has to land in Java too or the backends stop instantiating. JCodeanalyzer drives 3.0.1 at L1-L4 (the level reaches the analyzer as -a via analyzer_level, so 3 and 4 are newly requestable for Java), validates the schema-v2 envelope with JAnalysis.model_validate_json, and keeps it on .analysis alongside .application. Two sanctioned public shifts (spec leg 3): - J-1: get_call_graph() nodes are "<type fqn>.<signature>" strings, not (signature, klass) tuples. Node attrs method_detail/kind; edge attrs type/weight/calling_lines unchanged. Nested and local classes are spelled in source form off JType.qualified_name (Outer.Inner, Outer.$anon$0). @external/ endpoints are dropped, keeping 1.x's callable-only graph; get_external_symbols arrives in 3b. An endpoint homed on no callable raises naming it rather than becoming a node keyed by a raw id. - J-10: source_code single-file mode is removed from JavaAnalysis, CLDK.java, the compat shim's Java branch and JCodeanalyzer. get_test_methods reads every unit's source; remove_all_comments raises, which is the honest v2 behaviour for the one accessor that only ever worked in source mode. The ten skip-gated witnesses retire. Also: CRUD accessors raise from one module-level constant on both backends, naming codeanalyzer-java#187 (J-4); a cached analysis.json without schema_version is refused with a re-run message and one below the requested max_level triggers a re-run (J-9, local half); the artifact/config five return the shared Py* models, with get_config_uses and get_unresolved_config_reads empty because the Java wire carries neither; JGraphEdgesST is deleted with its importers and get_system_dependency_graph returns the wire call graph; get_method_parameters is annotated List[JCallableParameter], which is what it always returned. The public surface is frozen by test, derived from 1375b55 minus source_code. The pin moves to 3.0.1 but no jar is committed: python-sdk#339 removes committed jars in this leg, so _locate_jar gains a documented test/dev seam, $CLDK_CODEANALYZER_JAVA_JAR, and the e2e runs the release jar through it. The jar the e2e exercised is codeanalyzer-3.0.1.jar, sha256 c4e7d4b0b3aa4750d6783b370b6b2aea011df6607d9ee4dd6319cd776ab7a4dd. JNeo4jBackend stays abstract until Task 3 implements the artifact five; its 17 red tests are that one cause, as TSNeo4jBackend's were at the equivalent TypeScript step.
…scoped by can:// prefix, with a schema probe
Re-points every statement `JNeo4jBackend` issues at what 3.0.1 projects, and
refuses at attach anything else. The anchor is `:JApplication {name}`; scope is
the single id prefix `can://java/<app>/` from one helper, `_scoped`, so the
spelling cannot drift; `_module`, `J_HAS_UNIT`, `:JCompilationUnit`,
`J_HAS_CALLABLE`, `:JParameter`, `:JCallSite`, `:JComment` and the CRUD labels
are gone (the last six still have index definitions on a 3.0.1 database and
hold no nodes, which is why reading them was a silent empty, not an error).
Eight statements rebuild the canonical JApplication: one anchored module fetch,
one `*0..` containment traversal that replaces every per-parent fetch, one
prefix-scoped call-site fetch, one anchored import fetch, one doubly-scoped
J_CALLS fetch and the anchored artifact/dependency pair. Every accessor then
answers from the same models the in-memory backend walks, so the two agree by
construction rather than by two implementations of the same query.
Seek labels are measured, not assumed (table in the plan): every keyed label
already owns a uniqueness constraint on `id`, so each statement anchors on the
bare specific label and `:JCanNode` is used nowhere -- it loses the
whole-application prefix (118 ms against 24) and the point lookup (1.40 against
1.13 on `:JSymbol`), and wins only a per-module prefix, which no statement here
issues.
Two accessors raise instead of answering, because the projection cannot serve
them and an empty list would read as an answer: get_all_comments and
get_comment_in_file (there are no comment nodes at all -- a declaration keeps
only its javadoc, in a `docstring` property, and a file-level comment is not
projected). get_method_parameters does *not* raise: `parameters_json` is the
analyzer's own serialisation of the list, so parameters round-trip exactly.
Live parity on daytrader8 with ThingsBoard in the same database: identical
symbol-table keys (138), classes (149), JType/JField/JCallable fields,
parameters, call sites (4,006), wire call graph (1,862 edges, prov and weight
included), nx node/edge sets, callers/callees and class call graphs in both
modes, entrypoints (66/133), artifacts (235), dependencies (4), config keys
(336). The projection forces external calls -- 10,705 J_CALLS out of daytrader8
callables, 8,843 to `:JExternal` -- and 3a keeps the 1.x callable-only graph, so
1,862 remain, exactly the JSON's count. Documented tolerances, each measured:
a module's `source` is not projected; `code` is the declaration slice, not the
body block, so it ends with the reference's on all 1,117 callables with a body
and `calling_lines` shift on 563 of 1,862 edges; order within one source line is
not recoverable for fields (1 class), local variables (12 callables) or
annotations (30 declarations); imports are aggregated per target; docstrings are
declaration-level, not file-level; `param_in`/`param_out` and the L3/L4 graphs
are not rebuilt in 3a.
…est span sentinels, and review corrections A local or anonymous class's `JType.qualified_name` now carries the signature of the callable that declares it -- `p.Outer.m(int).$anon$0`, mirroring the id grammar the analyzer already writes -- and with it every `get_class(...)`, `get_all_classes()` and `get_call_graph()` node key on both backends. `$anon$N` is numbered per declaring callable, so without that segment two sibling callables of one type both spell `$anon$0`: on ThingsBoard 97 colliding names shadowing 381 of 5,102 type declarations, and on `JNeo4jBackend` the duplicate made the first index-routed accessor raise -- the whole backend was dead on the scale corpus, while daytrader8's four non-colliding local classes hid it. Fixed once on the model, so both indexes inherit it and neither is patched. Member (nested) types are unchanged: no collision in the 97 involves only type-declared anons, because those share a per-type counter. Recorded as the J-1 erratum in the spec and in CHANGELOG.md with a migration; witnessed offline by a hand-built payload driven through both backends, and live by a new ThingsBoard suite. Spans stop reporting a plausible wrong number. The projection carries `start_line`/`end_line` and nothing else, so every reconstructed column and both byte offsets are the model's own -1 on types, callables, fields, local variables, call sites and units -- the `0` columns and the `bytes=(0, len(code))` offset into a `source` that is `""` are gone. Frozen by assertion in the type, callable and field parity tests. J-16, a new spec decision that overrides Task 3's ruling: the two *file-keyed* comment accessors keep the raise (the graph holds nothing file-level, so any answer is fabricated), the three *declaration-keyed* ones keep the javadoc-only subset -- and the split is now stated where a caller reads it, with a `Raises:` clause on the ABC and the narrowing documented on the ABC and the facade both. `get_all_docstrings` also harvests enum-constant and record-component javadoc, which `reconstruct` had always rebuilt and the loop had always skipped (ThingsBoard: 1,248 constants, 29 documented; daytrader8 has none, so the parity test could not see it). The Neo4j module docstring's seek rationale was factually wrong -- `:JCallable` owns no id index at all, only a range index on `name` and a fulltext, so bare `:JCallable` plans a label scan. The conclusion survives re-measurement and stands; the argument is now the one the measurements support: the two prefix-scoped statements are traversal-dominated (under 1% wall clock either way, `:JCanNode` a quarter again the db hits) and everything else walks out from the `:JApplication` anchor. The multi-application audit was accepting a statement that scopes one endpoint of two, and was blind to any statement not passed to `self._run(`. It now parses the node variables out of each pattern and requires every one to be prefixed or reachable from the application anchor over a containment or shared-vocabulary relationship (`J_CALLS` is deliberately neither), with three negative and three positive cases as its own net; and it allow-lists every attribute touched on `self._driver` / `self._session_obj`, so `execute_query` cannot slip past. Also: `application` is a read-only property over a private `_application` cache, because rebinding it left `_idx` and `_call_graph` stale; `reconstruct` reads `props["source"]`, `edge["spec"]` and `edge["kind"]` rather than defaulting (`"runtime"` would have silently reclassified every dependency); `_facet` is deleted (its result was discarded, its `"callable"` branch unreachable, and `JType.kind` is already a `Literal`); the two backends' twin defect messages are one shared id-free text each and `_node_of` names the endpoint by signature and module key; the `code`/`body` divergence is documented on the model, the ABC and the facade; the probe's substring assertions are replaced by behaviour, the seek test renamed to say it greps a ruling rather than measuring one, and the dead `_MATCHES_BY_ID` branch removed. Five measured numbers corrected in the plan and the docstrings that quoted them: body-less callables are 99, not 129 (99 + 1,117 = 1,216); local-variable order differs on 6 callables, not 12; the body-key column delta reaches 110 with a mode of 4, not "up to 12"; attach plus first use is nine round trips (3 + 6), not eight statements. `calling_lines` re-measures at 1,299 of 1,862 -- the plan was right -- and the shared-contract line's `26 passed` was the correct total of the three files it named; both are now labelled rather than dropped. Java offline: 304 passed, 29 skipped (from 294/25 -- +10 tests, +4 live-gated skips). Live parity on 7691 with ThingsBoard in the same database: 19 passed. New ThingsBoard suite: 4 passed.
…anctioned shifts CHANGELOG [Unreleased] gains the Java block: the 2.4.1 -> 3.0.1 pin with the extra="forbid" v2 model mirror and the refusal of a v1 cache; analysis levels 3 and 4 becoming reachable; the two sanctioned breaks (string call-graph node keys with a migration that survives dotted parameter types and local classes, and the removal of source_code single-file mode); JGraphEdges becoming JCallGraphEdge; the CRUD raise (codeanalyzer-java#187); the J-16 comment split; the declaration vs body-block divergence of JCallable.code over Neo4j (codeanalyzer-java#176); the 3.0.1 graph vocabulary and probe floor; the generic-ABC artifact accessors; and the get_method_parameters annotation fix. docs/agent-api-reference.md gains a Java section: what attaches today, the 3.0.1 floor as an attach table, the accessors that still raise, and the note that the leg-1.5/1.6 query surface reaches Java in 3b. CLAUDE.md's Java row records the level range, the probed graph and the jar that #339 still has to move. The leg-3 spec gains a J-8 erratum: 3.0.1 emits declaration for every callable that has one (1,115 of daytrader8's 1,216); it is None only on the 99 implicit callables and the two <clinit>$N() initializers, and codeanalyzer-java#215 is about storing the text once per module, not about a missing field. The plan's Task 4 is ticked with the three runs, the grep and the refusal matrix. Also here, because the verification found them: a live refusal test attaching the Java Neo4j backend to a real codeanalyzer-python graph, and the two prose mentions of the retired 2.4.1 pin reworded to name schema v1 instead.
…d drop the bundled jar and JDK download The Java backend now gets both the analyzer and the JVM it runs on from the codeanalyzer-java PyPI wheel, pinned at 3.0.2 behind a new optional `java` extra. 3.0.2 reads its primordial scope from `jrt:/` in the running JVM, so nothing needs `JAVA_HOME` or a `jmods/` directory any more: * `JCodeanalyzer._get_codeanalyzer_exec` returns `codeanalyzer_java.command()`. The import is lazy, so `import cldk` and `import cldk.analysis.java` work without the extra; running the backend without it raises CodeanalyzerExecutionException naming the distribution and `pip install "cldk[java]"`. * Deleted `cldk/analysis/java/codeanalyzer/_jdk.py` (`ensure_jdk`, the pinned Temurin download, the ~200 MB per-project JDK cache), the checked-in `codeanalyzer-2.4.1.jar` and its directory, the `[tool.hatch.build]` force-include, the root `.gitignore` `*.jar` rule and the `CLDK_CODEANALYZER_JAVA_JAR` seam the e2e grew while the tree's jar was stale. * release.yml no longer downloads the pinned jar from the codeanalyzer-java GitHub release and no longer verifies a jar is bundled in the wheel and sdist. This retires the mechanism #284, #336 and #337 patched: the wheel pin is now the single source of the analyzer version, and nothing is fetched at build time. * Extras take the thin-base shape 2.0 is heading for: `neo4j`, `java`, and `all = ["cldk[java]", "cldk[neo4j]"]`. codeanalyzer-python and codeanalyzer-typescript stay hard dependencies until #340. * Fixtures regenerated at 3.0.2 with the command lines the v2 README records; the only content delta against the 3.0.1 pair is `analyzer.version` (verified by recursive parsed-JSON compare: one differing leaf per file). Verified: the wheel path with `JAVA_HOME` unset and no JDK on PATH returns the identical daytrader8 `-a 4` call-graph edge set the 3.0.1 jar returned on the provisioned Temurin -- 1862 edges, 1378 `declared+rta` / 375 `rta` / 109 `declared`, symmetric difference 0 after normalising the app name. A wheel built from this tree contains no `.jar`, and neither does the sdist. Closes #339
…urning a declared-only graph silently codeanalyzer-java degrades rather than failing: when the build it needs for L3/L4 cannot run, it emits a declared-only call graph and an SDG with no points-to provenance, exits 0, and still stamps max_level with the level it was asked for. Measured on daytrader8 at -a 4, the only difference between Maven on PATH and not is 1862 edges (declared+rta) versus 1391 (all declared) and 1220 points-to ddg edges versus none -- the envelope, the exit code and the application keys are identical. The analyzer declares this itself, but only on its log, at WARN. So the log is the signal. The SDK now passes -v (only alongside -o, since without it the payload occupies the same stdout the log would), matches WARN lines on the shape the analyzer uses -- a capability "unavailable", "emitting ... only" -- rather than on the exception text inside the parentheses, strips the ANSI colour, and keeps the analyzer's own sentence as the message. The verdict is persisted beside analysis.json as <cache>/java/analyzer_diagnostics.json, never as a field inside the payload, so a cache hit -- which by design never re-invokes the analyzer, and therefore has no log -- still knows. Three states, distinguishable, on JCodeanalyzer.analyzer_diagnostics: a non-empty list of level_too_low Diagnostics (the analyzer declared a degradation), [] (it declared none), and None (no verdict is recorded, so it is unknown -- a pre-#341 cache, an analysis.json from elsewhere, or stdout-pipe mode). None is reported as unknown, never as clean. Each state is logged once per analysis at WARNING, and nothing raises: a declared-only call graph is still a real answer, and callers content with it keep working. The payload's proxies (no rta, no points-to) are not consulted at all -- a small project can legitimately have neither. No new accessor on JavaAnalysis; a public reader is a 3b candidate. Closes #341
…g, and close the leg review
`get_test_methods()` re-parsed each compilation unit's `source` with Tree-sitter, and the
Neo4j projection carries no module source, so it returned `{}` on every graph-backed
analysis — `{}` on ThingsBoard, where 3,354 callables carry `@Test`/`@ParameterizedTest`.
It now reads the analyzer's own annotations off the model, so both backends answer the
same, and is keyed by `"<type fqn>.<signature>"` rather than a bare method name that
collapses same-named tests across classes.
`CHANGELOG.md`'s `[Unreleased]` block was structurally corrupt and the release workflow
lifts it verbatim: a `### Removed` heading split the `### Changed` list, two bullets
appeared twice (the second copy superseded), the Java pin read 3.0.1 where the pin is
3.0.2, and the preamble named only leg 2.5a. One heading of each kind now, no bullet
repeated, and both legs and both release targets named.
Also from the review: `get_config_keys()` is keyed by the artifact-relative key rather
than a `can://` id stamped with `--app-name` (the two backends shared zero keys);
`calling_lines` are sorted absolute file lines rather than offsets into a `JCallable.code`
that differs by backend, which removes the divergence on 560 of daytrader8's 1,862 edges;
`get_call_graph()` parses each body once instead of once per edge (ThingsBoard 145.7s →
41.0s); the degradation sidecar is bound to its payload's sha256 and never raises; the
`Makefile` no longer wgets a jar into a deleted directory; and the docstrings that
described behaviour neither backend has now name the backend arm.
The rebase left two Changed and two Fixed headings and six duplicated bullets, at 417 lines. Same content, one section of each, ordered by what a reader needs first: what breaks and how to migrate, what is new, what changed, what is fixed, what is still missing.
…and signature-aware resolution Java's two addressing rules the shared helpers cannot express, and nothing else (leg 3b, task 0). J-2: a Java module's dotted name is its declared package, not a derivation of its path — `src/main/java/com/ibm/…/TradeDirect.java` dots to `src.main.java.com.ibm…`, which names nothing, silently. `java_module_dotted` reads the package and adds the `package.TypeName` spelling, which is what splits two files of one package. J-3: a callable is keyed by a signature carrying the erased parameter tail, so `java_callable_names` gives the resolver both spellings — the bare name finds the callable with the tail cut, two overloads raise listing the tail-carrying signatures, and writing the tail resolves exactly. The cut is at the last `(`: an anonymous class's qualified name carries its declaring callable's tail (the J-1 erratum), and `in_class=` accepts that spelling unchanged. Both rules reach the shared policy as parameters rather than a fork: `CallableCandidate` gains `match_names` / `module_names` (empty = the previous behaviour, so every Python and TypeScript call site is unchanged) and `resolve_callable_signature` gains `by_full_name`, because "narrow it by naming more of the dotted path" cannot split two overloads and an instruction that cannot work is the failure E8 keeps out of the error path.
Both legs parameterised resolve_callable_signature for the same reason and differently. The candidate fields subsume the injected function and express what it cannot: a Java module answers to several dotted spellings, not one. Writing the resolution down so the rebase does not pick one silently.
…e must be followable
Three findings from the Task 0 review, plus the reconciliation with leg 2.5b.
module_names becomes Optional: an empty tuple means the language supplied
none, None means derive it from the path. Conflating them on truthiness
handed a .java path to a derivation whose default suffix list is (".py",),
so nothing was stripped and the extension rode into the dotted name — a
path derivation J-2 forbids, presented as a successful match.
The ambiguity advice said to name the signature "with erased parameter
types". The analyzer does not erase them, so a caller following that
literally gets a not-found error. It now points at the candidates the
message already carries. The in_class= and in_module= clauses are pruned
per ambiguity, since neither can split two overloads of one class.
2.5b parameterised the same resolver with an injected dotted function.
The two compose rather than compete: explicit module_names win, and the
injected function is what derives when a language has none to give — so a
derivation is never made with another language•s suffix list.
A task report read the a1/a4 difference as the tail spelling being unstable across analysis levels. It is not: it is type resolution. a4 is a pruned copy, so most types cannot be resolved and the analyzer falls back to the spelling written in the source — unqualified, generics intact. a1 has 143 signatures containing '<' against a4's 8, and 34 of 34 qualified interface names against 4 of 5. The consequence for the remaining tasks is the point: a test that pins an a4 signature pins the pruning, not the analyzer, and will not match the same method in a complete project.
The seven accessors of leg 3b Task 1, in Python's signatures keyword-for-keyword: locate, locate_many, resolve_callable, resolve_value, get_source, describe and the has_resolution_edges property. Implemented once, on JavaAnalysisBackend, rather than per backend: leg 3a made JNeo4jBackend rebuild the canonical JApplication and answer from it, so a second implementation would have nothing extra to read and would only be a second place for "what does cancelOrder mean" to drift. Each backend supplies three facts — the body nodes of a callable, the text of one, and whether call sites resolve at all. What still differs is the data, not the code, and it is documented where a caller reads it: a callable's text is the body block off analysis.json and the whole declaration over Neo4j (codeanalyzer-java#176), a body node has text only locally, and a module-scope locate over the graph answers "" plus module_source_unavailable. J-6 in full: initializers resolve and behave like methods, implicit callables resolve with line=-1 and get_source refuses them by name, anonymous classes carry their declaring callable. The one statement this adds is the per-callable body-node fetch, anchored on the bare :JBodyNode label and measured on ThingsBoard rather than ported: 86.6 ms / 16,024 db hits against :JCanNode's 86.8 / 20,028 and a J_HAS_BODY_NODE hop's 204.2 / 503,741. Carrying the whole-application prefix as well, so the scope audit's existing regex would match, was measured at 10-15x slower — the audit learned the UNWIND spelling instead, and now judges the values bound to $prefixes.
…ates The thirteen dataflow accessors of leg 3b Task 2, on both backends and the facade, in PythonAnalysis's signatures with Java's edge models. The call-graph half (reaches, callers_of, callees_of, backward_cone, call_paths_between) is implemented once on JavaAnalysisBackend over the get_call_graph() both backends already build, so it issues no new Cypher; the body-node half reaches the v2 models locally and five prefix-scoped statements over Neo4j, anchored on the callable's id prefix rather than a doubled containment hop, which drops every self-loop (python-sdk#349 — Log.printCollection has 20 J_DDG edges, 11 of them self-loops, and the doubled spelling returns 9). codeanalyzer-java 3.0.1 emits the L4 port lattice disconnected from the statement dependence graph: no J_DDG or J_CDG edge touches a formal_in / actual_in / formal_out / actual_out vertex, so a parameter has out-degree zero. slice_forward, paths_between, flows_to_call and flows_to_argument would therefore return the same answer for every input, which is indistinguishable from a proved absence of flow, so they raise naming the gap through one data-driven guard. slice_backward and the call-graph half are unaffected.
…ry surface
Completes the leg-1.5/1.6 query surface for Java (leg 3b, Task 3): the entrypoint
trio, the four bulk projections, get_external_symbols, the artifact layer on the
facade, and the J-7 leaf accessors get_interfaces/get_enums/get_enum_members/
get_records — 18 accessors in PythonAnalysis's signatures, with Java's models.
Thirteen of them live once on JavaAnalysisBackend and neither backend overrides
one: 3a made JNeo4jBackend rebuild the canonical JApplication (decorators
included, since J_ANNOTATED_BY is one of the relationships its containment walk
collects), so a second implementation would only be a second place to drift.
The leg adds exactly one Cypher statement, _external_rows.
Three things Java says rather than answers, each measured:
* get_entrypoint_coverage reports entrypoint_report_unavailable. The
:JApplication anchor carries name/schema_version/analyzer_name/
analyzer_version and nothing else, and analysis.json has no report key —
unlike codeanalyzer-python 1.4.1 and codeanalyzer-typescript 1.3.0. A count
of syntactically-marked callables is not a coverage report, so none is
synthesised from the booleans (J-4). The marks themselves agree id-for-id
between the two sources: 133 callables and 66 types on daytrader8.
* get_external_symbols raises off a local run and answers over Neo4j.
codeanalyzer-java homes out-of-project call targets only under
--external-calls, which --emit neo4j forces on and the SDK's own run does
not pass, so the graph carries 1,195 (2,570 on ThingsBoard) and a plain -a
payload carries None. {} would read as "this project calls nothing outside
itself"; the raise names the flag instead.
* get_enum_members raises SelectorNotInGraph on a name that is not an enum,
where TypeScript returns []. "This enum declares no constant" and "no such
enum" are different answers.
get_decorated_callables matches a marker by simple name, with a leading @
ignored, or by fully-qualified name — one rule, since the Java wire carries the
simple name only (J-5). get_method_bodies and get_callsites_for are keyed by the
J-1 "<type fqn>.<signature>" name, matched exactly, and omit a miss rather than
mapping it to None. get_config_keys's artifact-relative key is untouched
(python-sdk#346).
The multi-application audit grows from 24 statements to 25 and judges the new
one unchanged; two behavioural leak tests cover the Task 3 surface and the
external ghosts.
Docs: the agent reference replaces 3a's "arrives in 3b" line with the two landed
sections and gains five lossiness bullets — the disconnected L4 port lattice
(codeanalyzer-java#227), the 87 dangling ddg endpoints (#228), the twelfth
body-node kind `switch`, get_external_symbols' asymmetry and get_callsites_for's
intra-line order. One number Task 2 published was wrong and is corrected: a
`switch` vertex's J_CDG fan-out is at most 154, not 1,498.
``_BODY_NODES`` returned id, kind and the two line numbers, so ``R.body_node`` minted ``callee=None`` on every row and every call site located over the graph read as unresolved -- while ``has_resolution_edges`` was telling the caller those nulls were per-site. The graph has the fact as a ``J_RESOLVES_TO`` edge; it is now read in the same round trip, scoped to the application prefix like every other endpoint, and an ``@external`` target is carried as the ``callee`` it is (``get_external_symbols`` is keyed by those very ids). The offline responder emitted the same three columns and so froze the divergence; it projects ``callee`` now. The live parity assertion is containment rather than equality, which is the relation that is actually true: ``--emit neo4j`` forces ``--external-calls`` and the reference run does not, so the graph resolves all 4,006 of daytrader8's call sites where the payload resolves 1,723 -- and must agree on every one of those.
…H it follows A free-standing WHERE deleted the null row an OPTIONAL MATCH exists to produce, and read a property off an unbound variable while doing it. Both are now what Cypher does: the clause filters the optional half, the null row survives, and a predicate on an unbound variable is null rather than a KeyError.
…nswer empty
The spec says an implicit callable resolves and that get_source, the three
per-callable graphs and the slices raise naming the reason. Only get_source
did: get_cfg/get_cdg/get_ddg returned total=0, edges=[], complete=True on both
backends -- a well-formed page reading "this callable has no control flow" for
99 of daytrader8's 1,216 callables -- and resolve_value blamed the value name
("value not in graph") for a callable that declares nothing at all.
One guard, _require_explicit, on the shared ABC beside the level and port
guards, reached by both backends' graph accessors and by resolve_value, which
is the single door every slice, path and flow predicate addresses a value
through. The premise is one message constant; only the tail differs.
_SLICE and _VALUE_REACHES bound only their seed and their far endpoint, so a walk could leave the application between them and come back -- and _VALUE_REACHES carried a comment claiming 'every hop is inside the application, as in _PATHS', which was the one thing it was not. Both now carry the same whole-path predicate _PATHS does, keeping the endpoint one alongside so the per-variable audit still reads them as scoped. The audit that exists for exactly this class selected on 'allShortestPaths' while its docstring said 'a shortest-path or variable-length pattern', so it skipped both of them; it now matches any variable-length quantifier and names _SUBTREE as the one exclusion, with the reason (it walks containment out of the application anchor, so its interior cannot leave). Two docstrings that described tests and scopes that did not exist are corrected. Cost, re-measured on the shipped statements against ThingsBoard's largest formal_in backward cone (604 nodes, depth 5, interleaved, median of 5, three sessions): 9.6/9.9/9.9 ms against 9.2/9.5/9.1, identical rows.
…he body map resolve_value composes <callable id>@formal_in:<n> from the declared parameters, which exist at every analysis level; the vertex that carries one exists only from level 3. _sources_for looked it up among the body nodes, so describe() raised KeyError -- documented as 'a stale or foreign address' -- on a ref resolve_value had just returned, at the default analysis_level of CLDK.java(...). It now answers present-with-no-text, which is true at every level on both backends. An index past the end of the parameter list is a stale address and still raises.
--emit neo4j forces --external-calls, so no :JExternal row means the run homed
them and found none -- a real answer about a project that calls nothing outside
itself. 'or None' turned it into the value get_external_symbols raises on,
whose message says the Neo4j backend answers this, and made the documented {}
unreachable on either backend.
The Neo4j spelling matches (b:JBodyNode)-[...]->(m:JBodyNode), so it sees an edge only when its target was emitted as a node; the in-memory one counted any outgoing edge of a formal_in. codeanalyzer-java 3.0.2 emitted 87 of daytrader8's 5,434 ddg edges naming an endpoint it never emitted (#228, fixed in 3.0.3), which is the shape that makes the two disagree -- and this one boolean decides whether four accessors raise or answer. The offline responder's stand-in for the graph's answer moves to the same definition.
describe() listed LocateResult among the shapes it accepts and then raised 'needs something carrying a ref ...; got LocateResult' whenever the position landed on no body node -- 187 of 300 random in-callable positions on daytrader8, so the common reading rather than an edge case. It now says which of the three things happened (inside a callable but on no emitted vertex, at module scope, or in an uncovered file) and ends with a call that runs on the value the caller already has.
AmbiguousName renders 'Narrow it with {narrow_with}.', so every clause has to
be a noun phrase. The shared by_full_name default and Java's override were
verb phrases, and Java's keyword pruning makes that clause the only one for
every overload ambiguity -- the commonest ambiguity in the language. Both are
now noun phrases; the Python and TypeScript renderings keep reading as before,
and a test over every clause this module can produce holds the sentence.
…ed for
Three facades document 'self-reachability is True only through a real cycle',
bounds.py's self-path refusal tells the caller to ask reaches(X, X), and both
Neo4j backends answer it that way -- their pattern is {1,depth} and lands back
on the source like any other node. The three in-memory backends asked
'b in descendants(a)', and descendants excludes the source, self-loop or not,
so the answer was False for every input: a wrong answer reached from the error
path, and a backend divergence besides.
One shared call_reaches in commons/graphs.py replaces the identical line in
all three: for a != b it is what it was, and for a == b it asks whether any
predecessor of the source is reachable from it -- one hop shorter when the
budget is finite, so the two halves add up to depth.
… either test_java_addressing_rules pinned the word out of the error messages and not out of the models, so JCallableOverview.signature still called it 'the erased parameter tail' -- the normalisation the fixtures disprove (a4 spells setTopLosers(Collection<QuoteDataBean>), a1 setTopLosers(java.util.Collection) for the same method). The docstring now says what the tail is, and the ruling covers the model modules.
check_distinct_endpoints, not refuse_self_path; daytrader8 has 13,436 body nodes, not 12,236 (counted on the reference graph).
…callables
Four places said the omitted set is "the 99 implicit constructors and the two
<clinit>$N() initializers", and the plan put the count at 1,115. Both halves
are wrong. The filter is `code`, not `declaration`: 99 callables have no code,
101 have no declaration, and the two initializers are in the second set only --
they carry a body block and come back.
Measured through the shipped accessor on the committed a1 fixture, on both
backends: get_method_bodies over the whole overview is 1,117 of 1,216, and the
omitted key set is exactly {o.key for o in overview if o.is_implicit}. The
offline test now pins the count, the omitted set and the two initializers'
presence instead of only asserting the values are strings; the plan already
said 1,117 at Task 1 Step 2 and the live suite already asserted
`compared == 1117`.
…etracted
Two published figures for a1 -- "143" signatures containing '<' and "34 of 34"
qualified interface names -- came from a walk over each unit's top-level
`type_declarations` only. That walk skips nested, local and anonymous types,
and it yields 1,177 callables, the number the leg-3a J-8 erratum already
retracted in favour of 1,216. Reproduced exactly (143 / 34 of 34 / 1,177), so
the walk is the cause.
Re-measured over `JCodeanalyzer._types`, the flattened index the SDK addresses
on: whole a1 is 149 types, 1,216 callables, 154 signatures containing '<' and
45 of 45 qualified interface names.
The row was also mislabelled "generic arguments". All 154 of a1's '<'
characters are in the callable's *name* (<init>/<clinit>); counting only the
parameter tail (`"<" in sig.partition("(")[2]`), a1 has 0 and a4 has 4. The
table now reports the four shared types on both sides -- 128 callables each,
0 vs 4 generics in the tail, 5 of 5 vs 4 of 5 qualified interfaces -- with the
old figures kept as a dated erratum. The conclusion is unchanged.
Two of them sit in files that implement the thing they promise: `get_external_symbols` is right there on both backends. The clause is deleted. The third promised a retirement that did not happen. The reference said the nine NotImplementedError raisers hold "until 3b"; the spec's section 4 erratum records that no task of the plan carries that work, and test_java_public_surface.py's RAISING deliberately keeps all nine. It now points at the erratum and at that record instead of at a release.
`resolve_callable_signature` gained keyword pruning in this leg: the "Narrow it with ..." sentence now offers in_class=/in_module= only when the caller has not already passed that keyword and the listed matches disagree on it, and the trailing clause is a noun phrase that differs per language. That changes the message Python's resolve_callable / callers_of / callees_of / resolve_within produce for the same input -- a pre-existing Python test had to be rewritten for it -- with no CHANGELOG line. Adds one, describing the message as the code renders it today (verified against both branches of the pruning). Also fixes a docstring in test_resolve.py that cited test_the_advice_drops_a_keyword_that_cannot_split_these_matches; the test is test_the_hint_drops_... A sweep of the branch for other unresolvable test and symbol cross-references found none.
It returns [] unconditionally, for the reason get_config_uses does -- the Java wire carries no code-to-config edges and no config-read detector -- but it was listed among the landed accessors with nothing caller-facing saying so, and the CHANGELOG's Known limitations had no bullet for any of the three. A caller reading [] as "no code reads this key" would be reading a fact about the analyzer as a fact about the program.
The plan claimed has_resolution_edges costs "no extra round trip" and then gave the parenthetical that contradicts it. What it actually costs is one statement rather than two, by reusing the relationship-type fingerprint the schema probe already reads. test_java_schema_probe.py's test_attach_does_not_reconstruct_the_application states it correctly and pins the four; the plan line now says the same thing.
J-3 says spelling `name(erased types)` "resolves exactly". Measured on the
committed a1 fixture, both backends:
resolve_callable("cancelOrder(java.lang.Integer, boolean)")
-> AmbiguousName, 4 matches (TradeDirect, TradeSLSBBean, DirectSLSBBean,
TradeServices)
An interface and its implementors declare the same signature by construction.
Over the whole fixture, 1,216 callables carry only 581 distinct tails, 200 of
those declared by more than one type, covering 835 callables -- so a bare tail
addresses 381 of 1,216 and no framework method at all.
The behaviour is right: J-1 already makes <type fqn>.<signature> the address,
the ambiguity lists the full keys, and in_class=/in_module= do split this one.
Only the sentence was wrong, in three places -- the J-3 row, java_callable_names'
docstring and the resolve_callable bullet of the agent reference. Same style as
the J-1, J-8, J-9 and section 4 errata.
SHOW CONSTRAINTS / SHOW INDEXES on 7691: it is not true that every keyed Java label owns an id constraint. :JCallable has only a range index on `name` plus the code/docstring fulltext, :JType only one on `name`, :JExternal nothing at all, and :JCanNode a non-constraint RANGE index over 615,329 nodes. Their nodes carry the merge label :JSymbol, whose id uniqueness constraint is the only way to seek one by id. :JBodyNode is the only anchor on this surface that owns an id uniqueness constraint directly -- and it is the label the hot statements use, so the seek rule holds for a different reason than the leg-3a plan assumed. The code is conformant; only the stated reason was wrong. The cfg/cdg/ddg keyset orders have no unique tie-break, unlike _PATHS, which appends elementId for exactly that reason -- they cannot, since the same key has to be computable from analysis.json. Two edges with an identical full sort key straddling a page boundary would lose the second while `total` still counted both. Measured: 0 fully-identical parallel J_CFG_NEXT/J_CDG/J_DDG edges across both applications of the reference graph, and 0 in analysis.json (a1, a4, and the whole 22.5 MB daytrader8 -a 4 payload: 6,984 cfg, 4,416 cdg, 5,434 ddg, every key distinct within its callable). It cannot fire today; the asymmetry is now recorded beside the elementId comment.
The port-lattice bullet said codeanalyzer-python and codeanalyzer-typescript "have both already fixed the same gap" and then cited a Python-only number. Measured on the TypeScript reference graph (superset-frontend, read-only): 16,774 TS_DDG edges leave a formal_in, from 7,704 of its 8,025 formal_in vertices -- against Java's zero of all of them.
…graph loses The write-gated Neo4j parity module has skipped through every leg since it was written, so all five of its comparisons had gone stale. No backend changes. Two of the differences were not lossiness at all, and the fixture now removes them instead of tolerating them: - The project is created in a directory named APP_NAME. The in-memory backend takes the can:// application segment from the project directory's name and the emitter takes it from --app-name; with different names every id disagreed for a reason unrelated to the projection. - The graph is emitted at analysis_level=2, matching the reference's "call_graph". AnalysisOptions defaults to 1, which emits no PY_RESOLVES_TO edges (every call site's callee_signature None) and no defuse-provenance call edges (greet -> greet._decorate missing). That was a level difference being read as a projection loss; at level 2 the call graph matches exactly. Also stale: the module-level scope key. A module's own module_name for pkg/models.py is the short "models", not "pkg.models" -- both backends agree, including on answering an unresolvable scope key with an empty result, so the #246 regression is asserted on both spellings. What remains is tolerated in _norm with its cause named in the module docstring: absolute vs repo-relative file_path (asserted as a relation, not skipped), unprojected PyModule.source and imports (asserted empty), id and span left at their model defaults, an attribute's initializer and a call site's arguments, argument_types and columns. callee_signature stays exact on get_callsites_for -- the one accessor that follows PY_RESOLVES_TO -- so its resolution cannot regress behind a blanket tolerance. Three of these (id, initializer, arguments) are recoverable from properties the graph already carries and are marked as gaps.
rahlk
force-pushed
the
feat/issue-311-java-query-surface
branch
from
September 7, 2026 10:38
c2fe79b to
9885042
Compare
This was referenced Sep 7, 2026
…-java-query-surface # Conflicts: # .github/workflows/release.yml # docs/design/specs/2026-09-06-leg-3-java.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #311.
Leg 3b of the CLDK 2.0 agent-facing query facade: the leg-1.5/1.6 query surface for Java, over
codeanalyzer-java3.0.2. Specdocs/design/specs/2026-09-06-leg-3-java.md(decisions J-1..J-17 and its errata), plandocs/design/plans/2026-09-06-leg-3b-java-query-surface.md.What it adds
Thirty-eight accessors in Python's own signatures: the addressing vocabulary (
locate,locate_many,resolve_callable,resolve_value,get_source,describe), the per-callable graphs with keyset pagination, slices and cones, the call-graph and value-flow predicates, the entrypoint and policy accessors, and the leaf accessors. Both backends answer identically, including on the miss paths — asserted programmatically rather than by inspection.Addressing
A callable is addressed by
<type fqn>.<signature>(J-1). Names resolve by simple name, by@-prefixed annotation, or by full signature; overloads raise with the candidates listed rather than guessing. There is no fuzzy matching anywhere, including in error paths — an ambiguity's advice names only spellings that genuinely matched, and drops a keyword that cannot split the matches it lists.Where it refuses instead of answering
Four interprocedural value accessors —
slice_forward,paths_between,flows_to_call,flows_to_argument— raise on a Java graph while the analyzer's port lattice carries no dependence edge. The refusal is measured from the data, never from an analyzer version:codeanalyzer-javaemitted the port vertices and the lattice between them but noddgedge touching a port, so every one of those four would return the same answer whatever the program does, which is indistinguishable from a proved absence of flow.This lifts by itself on
codeanalyzer-java3.0.3 (upstream #227 / #230), verified with no SDK change: the probe flips,slice_forwardreturns 45 nodes where it refused, andpaths_betweenreturns a 21-hop cross-callable path alternatingdata,argument,returnandcontrol. #354 carries the pin bump and the fixture regeneration.slice_backward,get_ddgand the whole call-graph half are unaffected and are not gated.CRUD accessors raise pending codeanalyzer-java#187;
get_external_symbolsraises on the in-process backend, which never passes--external-calls, and answers over Neo4j, which always does.Review findings fixed in this branch
Three independent adversarial reviews ran against the completed leg; all ten code findings and all ten documentation findings are fixed here, each with a test that fails without it. The ones worth calling out:
body.calleewas unconditionallyNoneon the graph backend and populated in process, so every located call site read as unresolved — whilehas_resolution_edgesreturnedTrue, which is the flag that tells a caller those nulls are per-site. The offline harness emitted the same three columns as the real query, so it could not have caught it.get_cfg/get_cdg/get_ddgreturnedtotal=0, complete=Truefor the 99 implicit constructors on daytrader8 — indistinguishable from "this callable has no control flow". One guard, reached by both backends and byresolve_value, so the slice and flow half is covered at the root.allShortestPathsand so skipped both.reaches(x, x)was a backend divergence, not just wrong advice — both graph backends already answered the cycle question, all three in-memory ones excluded the source. One shared implementation now.Published numbers were re-measured rather than trusted:
get_method_bodiesreturns 1,117 (not 1,115, and the two<clinit>$N()initializers are present, not omitted), and two fixture-README figures had been measured with a walk that skips nested, local and anonymous types — the same walk that produced the callable count the spec already retracted.Verification
Full release gate green: 1521 passed, 354 skipped, 84.91% coverage. Java live suite green against a graph holding two applications side by side, which is the live half of the multi-application scope audit. No test is flaky across repeated runs, including in randomized order.
Stack
#342(leg 2.5a) →#357(leg 2.5b) →#347(leg 3a) → this. Each targetsrelease/2.0and contains its predecessors, so merge in that order. This branch has been rebased onto 2.5b, so its tree is the reconciled one if any merge conflicts appear.