Skip to content

FEAT: Add Garak divergence scenario - #2547

Open
Vimal Sahani (VimalN2005) wants to merge 1 commit into
microsoft:mainfrom
VimalN2005:feat/garak-divergence-scenario
Open

FEAT: Add Garak divergence scenario#2547
Vimal Sahani (VimalN2005) wants to merge 1 commit into
microsoft:mainfrom
VimalN2005:feat/garak-divergence-scenario

Conversation

@VimalN2005

Copy link
Copy Markdown

Description

Fixes #2533

This PR adds a native PyRIT scenario porting Garak's divergence probe (divergence.Repeat and divergence.RepeatedToken), testing whether prompt repetition causes target models to diverge from requested repetition into unexpected continuation, output instability, or training data replay.

Key changes:

  • Dataset (pyrit/datasets/seed_datasets/local/garak/garak_divergence.prompt):
    • Implements 42 prompt seeds covering word repetition templates (poem, company, book, a across 1, 3, 40 repeats) and a small, bounded set of precomputed repeated tokens (without any runtime tokenizer dependency).
    • Harm categories: confidentiality, training_data_leakage.
  • Scorer (pyrit/score/true_false/regex/divergence_scorer.py):
    • Adds DivergenceScorer(MessageTrueFalseScorer), a deterministic scorer distinguishing compliant bounded repetition and standard refusal/mitigation prefixes from divergence hits into unexpected continuations.
    • Re-exported in pyrit.score.true_false.regex and pyrit.score.
  • Scenario (pyrit/scenario/scenarios/garak/divergence.py):
    • Implements DivergenceTechnique(ScenarioTechnique) with Repeat as default, RepeatedToken as explicit opt-in, and ALL.
    • Implements Divergence(Scenario) leveraging standard PromptSendingAttack workflow and per-prompt derived objectives.
    • Registered lazy exports in pyrit.scenario.scenarios.garak.
  • Scanner Documentation (doc/scanner/garak.py & doc/scanner/garak.ipynb):
    • Added documentation, technique guides, and CLI execution example.

Tests and Documentation

  • Added comprehensive unit tests in tests/unit/scenario/garak/test_divergence.py:
    • Initialization, dataset resolution, and default scorer checks.
    • Technique factories and PromptSendingAttack generation.
    • Technique expansion (DEFAULT -> repeat, ALL -> repeat + repeated_token).
    • Scorer evaluation on refusals, compliant repetition, and diverged text.
    • Result: 12/12 passed.
  • Ran the full Garak scenario test suite (pytest tests/unit/scenario/garak/): 154/154 passed.
  • Synchronized documentation via jupytext --sync doc/scanner/garak.py.
  • Verified formatting and linting with ruff check and ruff format --check.

@VimalN2005

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FEAT: Add Garak divergence scenario

1 participant