Skip to content

[BUG FIX] Normalize categorical answers in accuracy scoring - #186

Open
zjn20030811 wants to merge 1 commit into
OpenDriveLab:mainfrom
zjn20030811:fix/categorical-accuracy
Open

[BUG FIX] Normalize categorical answers in accuracy scoring#186
zjn20030811 wants to merge 1 commit into
OpenDriveLab:mainfrom
zjn20030811:fix/categorical-accuracy

Conversation

@zjn20030811

Copy link
Copy Markdown

Summary

  • Normalize categorical answers before scoring so case, whitespace, and terminal punctuation do not change a correct result.
  • Parse an explicitly selected option from explanatory responses (for example, The correct answer is A ...) while rejecting unrelated prose that only mentions option letters.
  • Keep exact normalized matching for yes/no and free-form labels, and add focused regression coverage.

Fixes #171

Validation

  • python -m pytest -q (5 passed)
  • python -m compileall -q challenge/accuracy.py challenge/evaluation.py tests/test_accuracy.py
  • git diff --check

Score selected options independently of explanatory wording while retaining exact matching for other answer types.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Question regarding "Accuracy" Metric

1 participant