Skip to content

Let a discovered candidate be adjusted without restating it - #270

Merged
bsbodden merged 1 commit into
mainfrom
feature/candidate-derivation
Oct 16, 2026
Merged

bsbodden merged 1 commit into
mainfrom
feature/candidate-derivation

Conversation

@bsbodden

Copy link
Copy Markdown
Member

Why

CatalogDiscovery's javadoc states its purpose: it exists "so that using the router does not begin with writing out every model's price, latency and per-task quality by hand — figures a caller has no way to know and would have to invent." It delivers on that only if a caller can then change one thing about what it returned.

ModelCandidate is a record of thirteen components with no derivation. Adding a single capability to a discovered candidate meant rebuilding it component by component and keeping that copy in step by hand — which costs more than declaring the model from scratch. So not discovering was the rational choice, and the invented figures drifted from the artifacts: the deep-dive walkthrough declared qwen3-0.6b with a 32,768-token context window where its GGUF says 40,960, having never asked.

Probed against the released 0.1.60 catalog, discovery already returns for SmolLM2:

smollm2_360m_instruct_q8_0
  local=true capabilities=[chat] contextWindow=8192
  ttft=72ms tok/s=20.70 quality={}

Of the seven builder calls that walkthrough wrote for that model, four were already there, one was half there, and only per-task quality genuinely was not.

What

  • ModelCandidate.toBuilder() — a builder initialised from every component.
  • ModelCandidate.Builder.addCapabilities(String...) — adds rather than replaces, which is the operation a discovered candidate needs: the catalog reports what the model is (chat), an application routes on names only it defines (liveness, grounding).

Both additive and non-breaking.

Capabilities carry over from toBuilder as an explicit choice: on a fresh builder tags() seeds capabilities, and on a derived one that would silently discard what the application just added.

Evidence

  • 7 new tests in CandidateDerivationTest, each checked against a deliberately broken implementation.
  • One fixture was changed after a weak mutant: it held contextWindow at 8192, which is also the builder's default, so dropping that copy could not fail it. It now uses a real 131,072 and two tests catch the mutant.
  • build complianceCheck verifyDependencyPolicy verifyReleaseMetadata: 2,721 tests, 0 failures, 0 errors. :docs:generateSite is excluded — it fails in any git worktree, independent of this change.

CatalogDiscovery's javadoc says it exists "so that using the router does not
begin with writing out every model's price, latency and per-task quality by
hand -- figures a caller has no way to know and would have to invent." It
delivers that only if a caller can then change one thing about what it
returned.

ModelCandidate is a record of thirteen components with no derivation, so
adding a single capability to a discovered candidate meant rebuilding it
component by component and keeping that copy in step by hand. That costs more
than declaring the model from scratch, which makes not discovering the
rational choice -- and the invented figures then drift from the artifacts. The
walkthrough in the deep-dive article declared qwen3-0.6b with a 32,768-token
context window where its GGUF says 40,960, having never asked.

toBuilder() returns a builder initialised from every component.
addCapabilities(String...) adds names to the ones already set, which is the
operation a discovered candidate needs: the catalog reports what the model is
(chat) while an application routes on names only it defines (liveness,
grounding), and replacing would mean restating the catalog's own answer to
keep it.

Capabilities carry over from toBuilder as an explicit choice. On a fresh
builder tags() seeds capabilities; on a derived one that would silently
discard what the application had just added, so the flag is set and a test
pins it.

Seven tests, each checked against a deliberately broken implementation. One
fixture had to be changed: it held contextWindow at 8192, which is also the
builder's default, so dropping that copy could not fail it. It now uses a real
131,072 and two tests catch the mutant.
@bsbodden
bsbodden merged commit 3b6f600 into main Oct 16, 2026
38 checks passed
@bsbodden
bsbodden deleted the feature/candidate-derivation branch October 16, 2026 22:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant