All pull requests

Pull requests

• chore(report): support ReportT

  area-clichore
  #406·poshinchen opened Sep 25, 2026Contributor·

  ·
  5

• fix: blank user query in CorrectnessEvaluator reference mode

  area-evaluatorsbug
  #405·ybdarrenwang opened Sep 22, 2026Collaborator·

  ·
  5

• fix: improve root cause detector parsing returned json

  area-detectorsbug
  #401·ybdarrenwang opened Sep 15, 2026Collaborator·

  ·
  21

• feat: add StructuredOutputSimilarity evaluator (stickler extra)

  area-evaluatorsenhancement
  #400·sromoam opened Sep 14, 2026·

  ·
  26

• feat: report judge context overflow as could-not-evaluate, not score-0

  area-corearea-evaluatorsenhancement
  #399·pdebjyot opened Sep 14, 2026Contributor·

  ·

• feat(output_evaluator): add cache_config support for prompt caching

  area-evaluatorsenhancement
  #397·shssrd opened Sep 11, 2026·

  ·
  8

• feat(tools): add KnowledgeIndex for progressive disclosure over reference knowledge

  area-evaluatorsenhancement
  #396·pdebjyot opened Sep 9, 2026Contributor·

  ·
  1

  1

• fix: expose and serialize user tools on TrajectoryEvaluator

  area-evaluatorsbug
  #394·AmirF194 opened Sep 7, 2026·

  ·
  1

  1

• fix: make EvaluationReport.to_file() reject non-strict JSON

  area-corearea-devxbug
  #393·serenearyal opened Sep 4, 2026·

  ·
  1

  1

• feat(providers): add session discovery + evaluate_sessions batch helper (#143)

  area-tracingenhancement
  #391·lifelmy opened Sep 1, 2026·

  ·
  1

  2

• feat(store): add per-run keying and config-drift guard to EvaluationDataStore (#347)

  area-coreenhancement
  #390·lifelmy opened Sep 1, 2026·

  ·
  3

• feat(experiment): add ExperimentTrendAnalyzer for cross-run regression detection (#186)

  area-coreenhancement
  #389·lifelmy opened Sep 1, 2026·

  ·
  1

  2

• feat(classification): add classify_risk utility for multi-run results (#349)

  area-coreenhancement
  #388·lifelmy opened Aug 31, 2026·

  ·
  4

• feat(evaluators): add FuzzyEquals deterministic evaluator

  area-evaluatorsenhancement
  #387·lifelmy opened Aug 31, 2026·

  ·
  2

• ci: update langfuse requirement from <4,>=2.0.0 to >=2.0.0,<5

  area-communitychoredependenciespython
  #367·dependabot[bot] opened Aug 13, 2026Bot·

  ·
  2

• feat: add ToolEfficiencyEvaluator for session-level tool usage analysis

  area-evaluatorsenhancement
  #362·max-rattray-aws opened Aug 10, 2026Contributor·

  ·
  8

• feat: add evaluator metadata method and types

  area-devxarea-evaluatorsenhancement
  #361·max-rattray-aws opened Aug 10, 2026Contributor·

  ·
  11

• feat: add failure cohort analysis for evaluation reports

  area-corearea-evaluatorsenhancement
  #360·max-rattray-aws opened Aug 10, 2026Contributor·

  ·
  7

• feat: add status field to EvaluationOutput

  area-corearea-evaluatorsenhancement
  #359·max-rattray-aws opened Aug 10, 2026Contributor·

  ·
  9

• feat: add inclusive language evaluator

  area-evaluatorsenhancement
  #358·max-rattray-aws opened Aug 10, 2026Contributor·

  ·
  15

• feat(tools): add TraceIndex for progressive trace disclosure in judge-based evaluators

  area-evaluatorsarea-tracingenhancement
  #343·pdebjyot opened Aug 3, 2026Contributor·

  ·
  23

• feat(experiment): add opt-in model routing for evaluators

  area-corearea-evaluatorsenhancement
  #325·pdebjyot opened Jul 23, 2026Contributor·

  ·
  1

  10

• fix: convert Langfuse AGENT observations regardless of parent

  area-tracingbug
  #314·poshinchen opened Jul 15, 2026Contributor·

  ·
  1

  7

• test(redteam): cover RedTeamCase metadata sync and _build_system_prompt assembly

  area-redteamchore
  #301·zeroshotmind opened Jul 7, 2026·

  ·
  1

• feat(redteam): reframe attacker prompts to survive aligned attacker models

  area-redteamenhancement
  #298·kevmyung opened Jul 2, 2026Contributor·

  ·
  1