Skip to content

Extract a public JSONTransform library over the shared query engine - #65

Draft
ThorFuchs wants to merge 6 commits into
mainfrom
codex/1.7.0-json-transform
Draft

ThorFuchs wants to merge 6 commits into
mainfrom
codex/1.7.0-json-transform

Conversation

@ThorFuchs

@ThorFuchs ThorFuchs commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Extract the existing query compiler/VM behind the opt-in JSONTransform library. External packages compile immutable Sendable plans and pull exact native JSON through an invocation-owned guarded scope. swift-json query calls the same compiler/VM through package-only adapters on its existing cumulative parse/compile/execute/write ledger.

Copies of an execution handle share one owner and meter. Escaped handles invalidate, provider re-entry fails before mutable context access, cancellation and operational failures latch, and last-alias refunds drain after terminal errors. Native admission checks public outputs against the execution policy, including literals compiled under broader limits. Output admission preserves cumulative work/statistics and refunds temporary traversal frames on failure. The CLI retains its existing native writer and host semantics.

The candidate adds the tenth library to the actual development source. Existing nine public library roots and the regex runtime are unchanged from starting main. Nineteen private kernel files match after reversing type renames; portable Foundation utilities remain in that existing kernel. No second evaluator, numeric authority, dependency, dialect extension, platform change or fallback engine is introduced.

Current feature checkpoint: 21199f9af5bf50f1d473c6d3e67b91ae91f75089, tree 9f54309a5dbef38c7207d5783f2d90d7e0690eac.
Hosted PR execution: 1e7197f95bdad453fc50b542b0c143b43299637e, tree 426a77ef870dfce4f9d7a16a8602511d8d771e1f. Only main's sibling README/Developer Kit documentation differs from the branch.

  • Local Native: all 45 commands passed; Debug and Release each 802 functions/325 parameter invocations, all 49 native caps, all 16 CLI caps and pinned independent suites retained.
  • Local safety: ASan 802/325, TSan 52 shared-state functions, strict memory safety with warnings as errors and lifetime probes passed.
  • DocC: ten catalogs, 1,214 documented authored declarations, JSONTransform 42; compiled examples/site regressions passed.
  • Public module: 23 behavior functions, seven ownership compiler probes and twelve external public consumer witnesses passed. Fresh HTTPS exact-revision Debug/Release consumption also passed; this is not version-tag consumption.
  • Scoped WASI: 23 module functions and twelve consumer witnesses, Debug/Release on Node and WasmKit. No whole-root or CLI/I/O claim.
  • CLI: 16 baseline/candidate byte/exit comparisons passed. Pinned jq oracle: 154 evaluated passes, 14 explicit exclusions, zero failures.
  • Reproduction: deterministic branch archive and independent Git-free CLI/resource/public-consumer checks passed. The distinct hosted integration archive passed the full 45-command Native reproduction, Debug/Release each 802/325, with byte-equal extracted rebuild.

Hosted results for the current source: seven of eight product contexts passed. Native run 39142488867 passed native, safety, docs and archive. Linux run 39142488872 passed both normal architectures and ARM64 safety. Documentation publication was correctly skipped for this draft PR.

Open qualification gate: Linux x86_64 safety job 117719572404 passed ASan 786/325, TSan 51 and strict compilation, then swift-package crashed with signal 11 during pre-planning of the separate schema-example lifetime consumer. That process was not run. Root cause is unconfirmed; no library/compiler fix or common cause with the historical incident is claimed. Original bytes are preserved. No rerun, source workaround, gate weakening, flag/pin/timeout or leak-policy change was made.

Performance remains partial: loaded alternating baseline/candidate medians were −0.75% for 2,000 records and +0.17% for 20,000; paired logical retention was unchanged. These observations do not close controlled idle-host timing or isolated allocation qualification. The 10% review trigger and historical streaming disposition remain unchanged.

The predecessor 61125ed passed all eight contexts, but a separate public consumer then demonstrated its compiled-literal/tighter-execution-policy bug. This successor fixes it and adds inclusive-boundary, terminal-publication and cumulative-work/temporary-frame regressions. Earlier rejected ownership experiments, compiler crash and failed commands remain preserved as history, not final passes.

Status: PARTIAL, draft for source/API review and bounded qualification closure. No independent human approval or release readiness is claimed. Main advanced through sibling documentation work; this branch has not been rebased or merged with it. Keep ready conversion, merging, tagging and publication separate. Private evidence is retained locally and is not attached to this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant