System One: TypeSafe confidence formulas, strict schema 422s, bounded token counting

[AnthonyRonning]

Why

A review of POST /v1/systemone by an external agent (relayed by Anthony) made three claims. Validated against docs.typesafe.ai and the local stack (notes: ~/workspaces/_notes/systemone-claims-2026-10-11/FINDINGS.md):

1. confidence did not follow the TypeSafe spec. True. We returned entropy peakedness 1 - H/ln n (from the Privatemode reference) for choice and score. The spec defines choice (p_max - 1/n) / (1 - 1/n) and score max(0, 1 - Σ p_i|i - m| / MAD_unif) with m the most likely level. The gap is largest where thresholds live: a 90/10 choice was 0.53, the spec says 0.80.
2. Unknown fields should be 422 naming the field. True as a gap. files instead of images returned a confident answer about nothing and billed it; criterion in a question or yes in noul criteria were dropped the same way; a misspelled required field, an unknown type, a missing questions or malformed JSON all came back as a bare 400 with no code and no field (the middleware's parse failure). The spec says validation failures are 422.
3. Document Maple-only extensions and limits. Partly: image formats, token limits with a byte guide and the confidence definition were missing from the docs page (maple-marketing feat: add system prompt indicator to chat interface #99, still held until deploy).

While checking the state limit: tokens::count_tokens (tiktoken cl100k BPE) is quadratic per whitespace-free run and runs synchronously in the handler. Release build, one word of repeated x: 40k chars 0.6 s, 100k 3.8 s, 200k 16 s (25,000 tokens, so not rejected as too large). The body cap is 50 MB. Reachable by any authenticated caller, nothing billed. The helper serves ~30 call sites on client text, so chat and Responses share the exposure.

Changes

• Confidence follows the spec. choice_confidence and score_confidence replace confidence; SDK doc comments updated (comments only, no SDK behaviour change).
• Strict schema, 422 with the field named. The handler now receives the raw body and parses it itself with deny_unknown_fields at every level. New codes: system_one_unknown_field ("Unknown field files, expected one of model, state, questions, images, temperature."), system_one_missing_field, system_one_unknown_question_type, system_one_malformed_body. Only serde's unknown/missing-field details are forwarded (they contain the caller's own key names, cut at 200 chars); type mismatches get a fixed message because serde's text would echo values. Malformed JSON stays 400. Compatible with TypeSafe clients: their schema has no fields beyond state/model/questions and type/instructions/criteria; our SDKs send only known fields.
• Bounded token counting. count_tokens splits whitespace-free runs longer than 1 KiB (200k chars: 16 s → under 0.1 s; ordinary text counts are unchanged). System One rejects a state over 1 MiB or a question block over 512 KiB with 413 before counting.
• Proxy README: one line on the new 422.

Verification

• cargo test --bin opensecret -- system_one tokens in the service's nix shell: 48 passed (new: confidence_follows_the_typesafe_formulas, shape_errors_name_the_field_and_never_echo_values, byte_caps_reject_before_counting_tokens, long_runs_are_counted_in_bounded_pieces). cargo clippy --tests -- -D warnings clean; the full unit suite: 791 passed, 0 failed, 31 ignored.
• End to end against the local jevlab stack (tree-built backend behind the proxy), all through /v1/systemone:

probe	before	after
files instead of images	200, answered and billed	422 system_one_unknown_field: "Unknown field files, expected one of model, state, questions, images, temperature."
noul criteria yes/no	200, criteria dropped	422 system_one_unknown_field: "Unknown field yes, expected true or false."
instruction typo	400 "Bad Request"	422 system_one_unknown_field naming instruction
"type": "boolean"	400	422 system_one_unknown_question_type
missing questions	400	422 system_one_missing_field
criteria: "SECRET" on a choice	400	422 system_one_malformed_body, value not echoed
malformed JSON	400	400 (unchanged)
300k-character single-word state	hung past a 120 s client timeout (debug build)	413 in 1.9 s
1.1 MB state	–	413 in 0.3 s
TypeSafe choice + score example	entropy confidence	confidence equals the spec formula computed from the returned probabilities (0.9519 vs 0.9520, 0.1312 vs 0.1312)

• Tokenizer bench (release, tiktoken-rs 0.5 cl100k): 40k-char word 0.60 s, 100k 3.8 s, 200k 16.1 s unsplit; 200k in 1 KiB pieces 90 ms; ordinary prose count differs by 0 (pieces are cut only inside long runs).
• Spec check: formulas and the (0, 0.57, 0.43) → 0.35 example from docs.typesafe.ai/confidence; request schema from api.md and primitives/*; both TypeSafe SDKs read TYPESAFE_BASE_URL/TYPESAFE_DEFAULT_MODEL, the Python one ignores unknown response fields (pydantic extra: ignore), the JS one casts JSON.parse output.

Rollout

Backend only; no SDK or proxy code change. Land before the production deploy of #1045 so the public confidence numbers never change under anyone. Docs: maple-marketing #99 updated in the same pass, held until the deploy.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}