Distill a repeated Jev classification task into a local model, on the fly — same answers, your hardware.
-
Updated
Oct 1, 2026 - Python
#
model-cascade
Here are 10 public repositories matching this topic...
Distil an expensive LLM API call on a narrow task into a small local model: capture traffic, curate, LoRA fine-tune with MLX on Apple Silicon, evaluate against the teacher, and serve an OpenAI-compatible cascade that escalates low-confidence requests.
text-classification information-extraction calibration lora knowledge-distillation mlx cost-optimization fine-tuning apple-silicon llm llmops qwen llm-evaluation openai-compatible model-cascade
-
Updated
Sep 30, 2026 - Python
Cost-aware routing across agentic pipelines. Empirical benchmarks of accuracy, cost, and dangerous rate.
-
Updated
May 23, 2026 - Python
Studying Minimum Sufficient Inference: when objective execution evidence can stop LLM inference without sacrificing reliability.
inference code-generation ml-systems reliable-ai llm ollama llm-evaluation model-routing llm-cascade model-cascade
-
Updated
Sep 23, 2026 - Python
Adaptive Cascade Tuning via Importance Sampling
-
Updated
Sep 22, 2026 - Python
LLM serving router that prices every request in dollars and milliseconds. Semantic cache plus a confidence-gated cheap-to-GPT-4 cascade: 96.9% lower cost than always calling the strong model, within 1.6 accuracy points, median latency 450ms to sub-ms on repeated traffic. FastAPI, SQLite cost ledger, reproducible benchmark.
python latency embeddings cost-optimization finops mlops inference-optimization fastapi vector-search gpt-4 llm llm-serving semantic-cache model-routing model-cascade
-
Updated
Jul 27, 2026 - Python
Ask the cheap model first. Pay for the strong one only when the cheap answer fails a check, under a budget, with a cost record for every call.
-
Updated
Aug 23, 2026 - TypeScript
NSGA-II search framework for CIFAR-10 big/little dynamic inference cascades under embedded memory constraints.
-
Updated
May 28, 2026 - Python
Route each LLM call to the cheapest model that will get it right. Verifier-gated cascades with an offline threshold optimizer — 94% of the strongest model's accuracy at 34% lower cost.
-
Updated
Sep 23, 2026 - Python
Divergence-aware multi-agent routing that cuts LLM inference cost 10-100x. One signal routes queries, keys a multilingual cache, and detects hallucinations at AUC 0.90. CIKM 2026.
nlp multi-agent cost-optimization inference-optimization financial-sentiment-analysis finbert llm llm-serving semantic-cache llm-inference hallucination-detection llm-routing model-cascade cikm2026
-
Updated
Sep 14, 2026 - Python
Add this topic to your repo
To associate your repository with the model-cascade topic, visit your repo's landing page and select "manage topics."