Skip to content

Write-gated Neo4j parity modules have been skipped through every leg; all ten tests now fail against current behaviour #360

Description

@rahlk

Problem

Two live parity modules are gated on CLDK_TEST_NEO4J_WRITE_*, which no leg has ever set. They have been skipped continuously through legs 1, 1.5, 1.6, 2.5a and 2.5b while the behaviour they pin was deliberately changed underneath them. Pointed at a disposable server for the first time, both fail:

module result
tests/analysis/typescript/test_typescript_neo4j_backend.py 5 failed
tests/analysis/python/test_python_neo4j_backend.py 5 failed, 1 passed

None of the ten is an SDK regression. Every one asserts behaviour that a later leg intentionally replaced, and each replacement is documented elsewhere and asserted by the read-only live suites. The defect is that a whole verification layer went quiet and nothing noticed, because a skip is not a failure.

What the ten actually assert

TypeScript (5)

  • test_fields_and_parameters, test_get_method_parameters_module_level_function — expect get_method_parameters to return parameters. It now raises CodeanalyzerExecutionException("...carries no parameters..."), which is deliberate and which test_typescript_e2e_neo4j_live.py asserts with pytest.raises(..., match="no parameters for").
  • test_call_graph_no_dangling_nodes, test_callers_and_callees — assert "tags" in edge_data. The edge carries {provenance, type, weight}. tags is set nowhere in cldk/ — its only surviving mention is a docstring at cldk/analysis/typescript/typescript_analysis.py:154 promising callers-of edges carry "provenance / tags". The docstring is wrong, not the edge.
  • test_call_sites — expects receiver_type == "UserService"; it is None. docs/agent-api-reference.md:128 lists "a call site's method_name, receiver and args" among the projection's documented losses.

Python (5)

All five compare an in-memory module against the Neo4j-rebuilt one with plain ==, where the projection is documented to be lossy. On one module the diff is exactly: file_path absolute vs repo-relative, id present vs '', source present vs ''. The Java live suite handles the same situation correctly — it carries an explicit, reasoned tolerance list rather than asserting equality.

Scope boundary

In scope: make all ten assert current, documented behaviour — with tolerances stated and justified the way test_java_neo4j_backend.py states its own — fix the tags docstring, and make the gate impossible to leave unnoticed again.

Out of scope: changing any SDK behaviour. Every current answer here is the intended one.

Goals

  • Both write-gated modules pass against a disposable server, asserting what the code does today
  • Each tolerance is stated with its cause, in the idiom test_java_neo4j_backend.py already uses — not skipped, not loosened to assert True
  • typescript_analysis.py:154 no longer promises a tags edge attribute that nothing sets
  • A skipped write-gated module is visible: the suite reports at the end that these modules were skipped and what to set, or CI runs them against a throwaway container

Caveats and known risks

  • The write target must be disposable. These modules emit a fixture project over Bolt and delete that application's subgraph afterwards. Pointing them at a reference graph would destroy it; that is why the gate exists and why it has no default (test_python_neo4j_backend.py writes to whatever CLDK_TEST_NEO4J_URI points at, and never cleans up #324 — a default of bolt://localhost:7687 is an ssh tunnel on at least one development machine). Keep the no-default rule.
  • Fixing the assertions is not the same as restoring the coverage. Two of the TypeScript tests will end up asserting a refusal, which is weaker than what they were written to check; say so where it is true rather than leaving a reader to assume parity is proven.

Definition of done

  • Ten green, each pinning behaviour that exists, and a mechanism that makes the next silent skip loud.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions