Add POST /v1/systemone: typed decisions on Continuum GLM-5.3-Flash (MVP)

[AnthonyRonning]

Summary

First-pass MVP of a TypeSafe-compatible "System One" decisions endpoint, POST /v1/systemone, on OpenSecret.

A request carries a state (string or JSON) and typed questions (noul, choice, score, each with instructions and criteria, mirroring TypeSafe's SDK). The handler answers every question with one masked, single-token chat completion against Continuum's GLM-5.3-Flash: the prompt numbers the options, the assistant turn is prefilled with answer=, sampling is restricted to the option-number tokens (allowed_token_ids), and the distribution is read from their log-probabilities (logprob_token_ids, batched at Privatemode's 128-id cap and merged). The model never generates text. Answers carry type and come back TypeSafe-shaped: noul (P(true)); choice + probabilities + confidence; score (expected level index) + legend + distribution + confidence, both keyed by level number. Each answer also reports the calibration temperature and option_mass; usage reports the upstream request count.

Prompt layout, prefill and calibration follow Privatemode's reference implementation (edgelesssys/privatemode-decisions at 45f4273) so its benchmark and calibration numbers apply: question-first layout rendered byte-identically (Python json.dumps separators, state embedded as the caller's raw bytes), and a default temperature by option count (ln T = 0.962 − 0.076 ln n; "temperature": 1 in the request returns raw probabilities).

Deliberate MVP choices, per discussion:

• Provider and model are fixed: the registry's enabled Continuum route for glm-5-3-flash, the only deployment that honors the masked logprob read (Tinfoil's GLM-5.3-Flash returns HTTP 500 on logprob_token_ids). The completion router is bypassed on purpose instead of inheriting the caller's Router V2 bucket. Route health is still used: every upstream send claims the route's probe and reports its terminal, so an open breaker rejects up front and our 429/5xx feed the shared state.
• The external chat completions API is untouched.
• Same key and billing path as chat: usage is published per upstream request; the first failing question stops the rest (no partial answers), and usage the provider already consumed stays billed. Upstream failures are classified exactly like chat (429/503 → capacity error with capped retry-after; 4xx keep their sanitized status; else 502/504).
• Pro-tier gating and guest/usage-limit checks are shared with chat (authorize_completion_caller).
• Limits mirror TypeSafe's and Clef's: state ≤ 32k tokens, question ≤ 16k tokens, 64 questions, 255 options per choice, 10 score levels, 4 images (4 MiB each, 8 MiB total), 200k upstream prompt tokens per request; 30 s per read, 120 s per request, 16 upstream requests in flight process-wide. Schema rules answer 422 (as TypeSafe does) and size rules 413, each with a fixed x-opensecret-error-code (system_one_*) and message. Note that one 64-question call is 64 upstream requests on the shared Continuum key, throttled to 16 at a time; the cap is a one-line change if that ever matters.
• Options are labelled with single-token numbers: 0–190 contiguously, then the next single-token numbers (192, 195, … 480 for the 255th), measured against Continuum's serving tokenizer (0–190 on 2026-09-25 and 2026-10-06, 0–1199 on 2026-10-07; the published tokenizer checked by review) and re-verified at runtime every 10 minutes with one batched /v1/completions echo (bounded to 30 s, and no lock is held across it: callers that arrive mid-check go by the last verdict); a mismatch disables the endpoint until a later check passes rather than returning plausible wrong answers, and a check that cannot conclude (429, timeout, malformed body) keeps the previous verdict instead of overwriting it. Requests with ≤191 options are unaffected by the sparse tail.
• Extensions beyond TypeSafe's schema: images (data URLs shared by all questions; the first question is sent alone to seat the shared prefix) and temperature.

Also:

• maple-sdk (Rust): POST /v1/systemone added to the inference-endpoint allowlist, with a test. The standalone proxy pins the published crate, so it forwards the new path only after it picks up an SDK release containing this change (local verification used an uncommitted path override).
• proxy: route registered and listed in the banner/README.
• openai.rs: the caller authorization block, the non-2xx response classifier, and the usage/retry-after helpers are shared with the new handler instead of copied.

Verification

• services/opensecret: cargo fmt --check, cargo clippy --locked --all-targets --all-features -D warnings, cargo test --locked --all-features: 746 passed, 0 failed, 31 ignored (14 System One unit tests: schema and caller-order lowering, every validation rule, prompt bytes vs the reference layout, Python-separator formatter, upstream body, read planning, decode with missing ids / no logprobs, calibration and the temperature formula, answer shapes, usage totals, entitlement, upstream error mapping, deadline).
• proxy: cargo test --locked: passing. sdk/rust: allowlist unit test passing.
• End to end in a maple-dev-env workspace (local OpenSecret + Continuum proxy + billing, standalone maple-proxy built against the worktree SDK), Pro fixture account whose Router V2 bucket would route chat to Tinfoil:
  • 4 questions (noul, choice, score, noul): HTTP 200, 4 upstream requests. Calibrated vs raw ("temperature": 1): is_urgent 0.954 vs 0.999, refund_requested 0.065 vs 0.003, department billing 0.985 vs 1.000 (T = 2.48 for 2 options, 2.41 for 3); option_mass 0.98–1.00. The first call after startup includes the token-table probe (4.5 s); the same request again took 0.39 s.
  • Image request (banking screenshot as a data URL): HTTP 200 in 1.0 s, activity finance 0.980, sensitive 0.922; the first question is sent alone to seat the image prefix.
  • 150-option choice: HTTP 200 in 1.4 s, 2 reads merged, 4,608 cached prompt tokens on the second read.
  • Validation: a 1-option choice, the provider id as model, temperature: 0, and a non-data image each return 400 with x-opensecret-error-code system_one_bad_option_count / system_one_unsupported_model / system_one_bad_temperature / system_one_image_not_data_url and a fixed message.
  • Backend log lines name questions by index and kind only. Response objects (answers, probabilities) are JSON objects whose key order is not significant; option numbering in the prompt follows the caller's order.

Review round (benthecarman, 2026-10-06)

All eight "before merge" items are addressed in the second commit (calibration, question-first layout, missing-logprob detection, Continuum lookup + health gating, 4xx mapping, short-circuit + billing policy, limits + deadline + global concurrency, no caller strings in logs), plus the nits except where noted on the threads. Not done here, by choice: handler-level HTTP tests of the billing branches (they need a Postgres-backed AppState; the branches now live in completion_entitlement, which is unit-tested), a send_billed_completion refactor (billing stays in publish_usage_event_internal with two documented call sites), and an eval of the score lowering.

Post-review tweaks (2026-10-07)

Caps raised to match the field (64 questions as Clef; 255 options as Jev, Clef and Perplexity) and schema validation moved to 422 as TypeSafe's API. Contract survey against Jev, Clef, OpenAI's Decisions API (public beta 2026-10-06), Perplexity, Liquid, OpenRouter and others: the request/response schema here is the one every System One-compatible provider uses; OpenAI's /v1/decisions is the lone outlier (arrays, predicate, typed values).

• End to end after the change (same workspace, Pro fixture key): a 200-option choice is answered in 2 reads with the sparse-number tail (HTTP 200, fraud_report 0.61; option_mass 0.52 vs ~0.98 on small option sets, which is the prompt-fit signal that field exists for); 65 questions, a provider id as model, and a 1-option choice return 422 with their codes before any upstream call; an oversized state returns 413; a 2-question request answers as before. The runtime check logged the 255-entry token table as verified against the serving tokenizer. After the table-check fix (bounded probe, verdict kept): the first request after a backend restart probes and logs the table as verified (HTTP 200 in 4.0 s); the next one answers on the fresh verdict in 0.48 s without re-probing.

Follow-ups (not in this PR)

• Publish maple-sdk with the allowlist entry and bump the proxy's pin so the standalone proxy forwards /v1/systemone.
• SDK system_one() helpers (Rust, TypeScript) and a code-docs spec for the wire shape.
• Handler-level integration tests on a DB-backed AppState; a small eval of the score lowering; optional permutations for position bias.

Client surface (second commit)

• Rust SDK (maple-sdk 4.1.0, unpublished): system_one / system_one_with_api_key with typed SystemOneRequest / SystemOneQuestion (noul, choice, score) / SystemOneResponse / SystemOneAnswer models; OrderedEntries keeps question and option order on the wire and in answers; images and temperature extensions and the temperature / option_mass / usage fields are modelled; Error::api_status / Error::api_error_code read the backend's status and system_one_* code. Example examples/system_one.rs; unit tests for order and shapes; tests/system_one_integration.rs (422 with JWT, per-request key and configured key; typed answers when a Continuum route exists, typed 503/403 otherwise).

• TypeScript SDK (@mapleai/sdk 4.2.0, unpublished): systemOne(request, { apiKey? }) with the same types, exported from the package and on useOpenSecret(); rejections throw SystemOneError { status, code }. Unit tests on a mocked transport (order, both credentials, 422 and 403 mapping, pass-through of transport failures); integration case in ai.test.ts.

• Backend: system_one_* rejections now also carry error: {message, code} in the body (like the plan/quota errors), so every client sees the code without reading headers. Answers and choice distributions are now serialized in caller order (answers in question order, probabilities in option order); serde_json::Map had been sorting them alphabetically.

• Standalone proxy and both desktops: proxy, apps/maple-agent and the Research native shell now build the in-tree maple-sdk (path dependency plus a root [patch.crates-io] entry so the embedded proxy's registry requirement resolves to the same source; lockfiles regenerated and verified with cargo metadata --locked). The route is therefore served by the standalone proxy and by both desktops' embedded proxies as built from this repository. Tests: the route forwards method, body and key; 422 rejections keep status, body and x-opensecret-error-* headers.

• CI routing for the in-tree SDK: since the proxy and both desktops now build sdk/rust from the tree, Rust SDK build inputs (Cargo.toml, src/, build.rs, assets/) select their CI again — change_detection.py routes them to the native lanes, agent_change_detection.py follows, and the proxy, Rust-tests and supply-chain workflows list them in their path filters (selector tests + guidance updated; the "accepted gap" from ci: stop running client CI for SDK-only changes #1039 is closed). TypeScript SDK changes still select nothing because the frontends pin a published package.

Policy (Anthony, 2026-10-07): Maple's own consumers use the in-tree SDK sources and do not wait for SDK publication or version bumps; docs/sdk-publishing.md (consumer version policy) and the proxy README say so now. Publishing maple-sdk 4.1.0 / @mapleai/sdk 4.2.0 remains a separate action for external SDK users. The Research frontend's @mapleai/sdk pin is unchanged (it does not need the helper; switching it to file:../../../sdk is the documented one-command change when wanted).

🤖 Generated with Claude Code

[benthecarman]

the core approach checks out: the token table matches the published GLM-5.3-Flash tokenizer (all 191 ids), the prefill renders as intended through GLM's chat template, and the request fields and score/confidence math match Privatemode's reference implementation (edgelesssys/privatemode-decisions).

things i'd want addressed before merge:

1. probabilities aren't calibrated, and raw one-token probabilities are overconfident (L449)
2. state-first prompt layout, which the reference dropped for scoring worse (L284)
3. no check that every requested logprob came back, so a missing option silently reads as 0 (L424)
4. assumes the default proxy is Continuum, and skips health gating (L567, L646)
5. every upstream 4xx becomes a 503 (L673)
6. a failed question doesn't stop the rest, and the user is billed for partial work (L591)
7. limits multiply, and there's no deadline for the whole request (L48, L55)
8. caller-supplied question names go into logs (L215)

comments marked nit: are optional.

[benthecarman]

the score lowering (level_N labels with the legend as the description, plus "ordered from lowest to highest") isn't something the reference benchmarked, so the accuracy and calibration numbers don't cover it. probably fine, but worth a small eval before relying on it

[AnthonyRonning]

Agreed, not benchmarked. Kept as is (level_<n> labels with the level text as the description, plus the ordered-levels sentence); it was exact on the small severity check, but a proper eval against labeled ordinal data is a follow-up. Leaving open as the reminder.

[AnthonyRonning]

SDK and proxy support should be included in this PR before we offer the experimental API.

• Add public typed request/response models for noul, choice, and score, plus dedicated call methods in both the Rust and TypeScript SDKs. Include the existing image/temperature extensions and response calibration/usage fields, preserve caller option order, and use the established Transport V2 path with JWT and API-key authentication. Export the TypeScript API and expose it through the usual SDK context/hook surface.
• Make POST /v1/systemone work through the standalone proxy with its actual resolved SDK dependency. The route is registered, but proxy/Cargo.lock still resolves published maple-sdk 4.0.1, which lacks this endpoint's inference allowlist entry. An uncommitted local path override is useful validation but does not deliver proxy support. Include the appropriate SDK version bumps and dependency/lockfile changes, and document the required publication/rollout order.
• Add examples and integration coverage for both SDK helpers and the proxy: all three answer types, both auth methods, and preservation of backend validation/quota errors and their status/error-code headers.

No Research or Agent classifier-workload integration is needed for this scope. The goal is a usable public SDK/API and proxy surface, rather than requiring callers to hand-build raw requests.

[AnthonyRonning]

Added in 0ce4e9e (second commit on the PR):

• Rust SDK (maple-sdk 4.1.0, unpublished): system_one / system_one_with_api_key, typed SystemOneRequest / SystemOneQuestion::{noul, choice, score} / SystemOneResponse / SystemOneAnswer, OrderedEntries for caller order, the images / temperature extensions and the temperature / option_mass / usage fields, Error::api_status / Error::api_error_code. Example sdk/rust/examples/system_one.rs; tests/system_one_integration.rs covers 422 with JWT, a per-request key and a configured key, and typed answers when a Continuum route exists (typed 503/403 otherwise; the CI backend has no Continuum).
• TypeScript SDK (@mapleai/sdk 4.2.0, unpublished): systemOne(request, { apiKey? }) and SystemOneError { status, code }, exported from the package and on useOpenSecret(). Unit tests on a mocked transport (order, both credentials, 422 and 403 mapping, transport failures left alone); live case added to ai.test.ts.
• Standalone proxy: proxy/Cargo.toml builds the in-tree SDK through [patch.crates-io] and proxy/Cargo.lock now resolves maple-sdk 4.1.0 from ../sdk/rust (cargo metadata --locked confirms; both desktop workspaces still resolve registry 4.0.1 because a non-root patch does not propagate). Tests cover the route and the pass-through of 422 status, body and x-opensecret-error-* headers; the README records the publication order.
• Backend: system_one_* rejections now also carry error: {message, code} in the body, and answers / probabilities serialize in caller order (the integration tests caught serde_json::Map sorting them).

Verified locally against the jevlab stack: both SDK integration suites green with the Pro fixture (422 paths with both auth methods, full three-question answers), and the tree-built proxy answering live plus passing a 422 through unchanged.

Rollout per docs/sdk-publishing.md ("Rolling an SDK fix out to clients"): merge → publish maple-sdk 4.1.0 and @mapleai/sdk 4.2.0 from master (both already bumped in-tree) → consumer PR re-pins proxy (drop the patch, cargo update -p maple-sdk --precise 4.1.0), apps/maple-agent and the Research native shell to 4.1.0, after which their embedded proxies serve the route too.

[AnthonyRonning]

Update: the client surface is now the single second commit ac17ed4 (0ce4e9e folded in), with two changes on top of the summary above:

• In-tree SDK for every Maple consumer. apps/maple-agent and the Research native shell now take maple-sdk from sdk/rust like the proxy does (path dependency plus a root [patch.crates-io] entry so the embedded proxy resolves the same source; lockfiles regenerated). Both desktops' embedded proxies therefore serve POST /v1/systemone as built from this repository, and nothing waits on an SDK publication. docs/sdk-publishing.md (consumer version policy), sdk/README.md and the proxy README now state this; publishing maple-sdk / @mapleai/sdk remains a separate action for external users. maple-agent's just clippy and the Research Rust gate pass against the in-tree SDK (its API delta from the published 4.0.x is only the System One additions).
• Rolled-up types fix. The one CI failure on 0ce4e9e (SDK package (npm): consumer typecheck, TS2552 TransportV2ClientDependencies) came from the @internal systemOneWithDependencies living in the public api module; it now lives in src/lib/systemOne.ts and api.ts re-exports only the public names. scripts/ci/build-sdk-publish.sh npm passes locally ("Package consumers passed: ESM, CommonJS, browser UMD, Bundler and NodeNext").