Deterministic runtime for agent evaluation
-
Updated
Mar 25, 2026 - Python
Deterministic runtime for agent evaluation
KayaDB distributed key-value storage engine
Zero-LLM deterministic jailbreak regression benchmark: runs a repeatable battery of known-pattern attacks against an LLM endpoint and scores refusal/partial/compliance across runs. Catch when a model or prompt update silently got weaker. Single-turn, responsible-use. pip install hermes-jailbench
Deterministic net.Conn/net.Listener fault injection for Go — the testing/synctest philosophy applied to the network layer.
Local-first C# project for deterministic prompt versioning, A/B evaluation, and evidence-based promotion using structured scoring.
Reliability testing for MCP servers: deterministic faults, bounded recovery, and executable regressions.
A TypeScript harness for deterministic frontend testing and development with repeatable scenarios, local fixtures, and browser verification for coding agents.
Deterministic UI testing framework for TS/JS — canvas-first. Virtual time, scene contracts, model-driven discovery.
Record normalized DeepSeek Harness tool results and replay them offline with strict matching and zero tool-body execution.
Safe Rust metaprogramming and adversarial testing: typed code generation with receipts, proc macros, generated trials, mutation, replay, and reproducible evidence.
WordleBench — Deterministic AI Wordle benchmark. Compare 34+ LLMs (GPT-5, Claude 4.5, Gemini, Grok, Llama) head-to-head on accuracy, speed, and cost across 50 standardized words.
Reproduce OpenAI Agents SDK runtime bugs without API keys — real Runner, scripted models, verifiable certificates.
LogOS A closed-loop cognitive Operating surface: written in Rust, deployed via self-verifying 'narrow-waist' Nix OS + Mirage OS Uniquernel to Google Cloud Run/Kubernetes,
Bounded deterministic schedule exploration for cooperative asyncio code
A Raft consensus library built from scratch in Rust, with a deterministic network simulator for testing leader election, log replication, and partition recovery.
Document-driven E2E test harness with AI-assisted compilation and deterministic Hurl execution
Local-first persistent Chromium automation harness for AI agents, built in Rust on raw CDP with semantic snapshots, trusted input, deterministic waits, checkpoints, and background jobs.
Deterministic Rust testing utility for simulation and stochastic workflows
A black-box evaluator that checks agent behavior and evidence independently, then produces reproducible reports.
To associate your repository with the deterministic-testing topic, visit your repo's landing page and select "manage topics."