Verify that an AI agent's on-chain claims are actually true. A claim-level groundedness checker for Solana / memecoin agents.
Grounded multi-domain support-triage agent (HackerRank Orchestrate Hackathon). 8-layer DAG fusing ChromaDB vector + BM25 lexical retrieval, with an adversarial-critic pass that blocks ungrounded answers — 93% groundedness across a 772-document corpus.
Provider-agnostic enterprise RAG and agent evaluation harness for Azure Foundry, vLLM, Ollama, and local demos.
Evaluation patterns, release gates, and anti-hallucination techniques for developer-focused AI workflows.
pip install gauntlet-evals · v0.1.0. Merge-blocking evaluation gates for generative AI features: YAML suites run against any HTTP endpoint or Python callable, fail the build on a miss, and emit both a diffable JSON pack and a reviewer document cross-referenced to California's published GenAI risk framework. Aligned to, never approved by.
v0.2.0. Fail-closed evaluation harness for government-facing chat systems: reproducible, provenance-stamped audit verdicts, byte-identical across Python 3.11 to 3.14, with no third-party dependencies. A silent or unreadable target scores zero rather than passing by absence. Two public projects of my own pin it by exact commit.
In-build reference implementation: an offline-first plant-care assistant and public evaluation harness with cited-corpus answers, calibrated abstention, toxicity guardrails, EN/ES parity, photo plant ID, and local reminders.
Single-agent, evidence-grounded claim verification to catch LLM hallucinations — a pluggable fact-gate for agent-arena and any multi-agent system (CrewAI, AutoGen, LangGraph).
Beta. Reduced-fare policy assistant citing dated corpus passages in English and Spanish; the bilingual-parity gate is currently failing (see EVALS.md). Corpus of eighteen California transit agencies, public 385-case evaluation harness. Deployed demo serves five agencies; published evidence run lags the repository.
A local groundedness judge for RAG: QLoRA-distilled to match a frontier judge 100% at $0/call. Ships only if its own evals beat baseline.
Measure whether a RAG system is actually grounded, and fail the build when it stops being: retrieval metrics, claim-level groundedness, hallucination rate and citation accuracy.
Your RAG, put to proof — self-correcting answers and CI eval gates that block the merge when hallucination rate crosses your threshold
Detect & score LLM hallucinations by groundedness — labeled data, precision/recall/F1, runs offline with no API key. Pluggable LLM-judge backends.
An AI-powered RAG groundedness / hallucination checker - segments an LLM answer into atomic claims and flags those NOT supported by the retrieved context, via an LLM judge. Provider-agnostic, BYO-key, offline --dry-run. Python CLI.
How wrong are LLMs about emission factors — and how often do they name the wrong source for a number? 467 questions, five models, scored against sourced ground truth.
RAG evaluation workbench for retrieval recall, citation coverage, groundedness checks, and failure analysis
Scores whether an AI agent's answer is actually supported by the tool results it had access to. An LLM-as-judge harness for catching ungrounded or fabricated claims in agent outputs.
Deterministic citation and claim-support checks for RAG evaluation datasets.
Groundedness & hallucination checker for LLM/RAG answers: decomposes an answer into atomic claims, verifies each against its sources (LLM judge or offline NLI), and highlights what isn't supported.
TypeScript eval harness for measuring whether Grok answers stay grounded in source evidence
To associate your repository with the groundedness topic, visit your repo's landing page and select "manage topics."