Skip to content

feat(llm): fall back to models from another provider - #169

Open
msmakouz wants to merge 1 commit into
masterfrom
feat/llm-fallback
Open

msmakouz wants to merge 1 commit into
masterfrom
feat/llm-fallback

Conversation

@msmakouz

@msmakouz msmakouz commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

What was changed

Model cards take fallback (model names or class: refs) and fallback_on (error types that switch; default: rate_limit_exceeded, server_error, timeout_error, network_error). generate/structured_output/evaluate try the card's routes by priority, then the fallbacks, under options.deadline_ms; max_tokens is capped to the candidate's output_tokens and reported in metadata.adjusted. Streams switch only before the first chunk reaches the client (stream_started). The answering route is returned in metadata.route / metadata.fallbacks; options.route pins it, and agent:step returns route_pin so hosts can keep it for the rest of a turn.

@msmakouz
msmakouz requested a review from wolfy-j October 8, 2026 19:05
@msmakouz msmakouz self-assigned this Oct 8, 2026
@msmakouz msmakouz changed the title feat(llm): add the ability to use models from another provider as a f… feat(llm): fall back to models from another provider Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant