Skip to content

Fix BuffBench terminal crashes with numeric env values - #1395

Open
yolandaycao wants to merge 1 commit into
CodebuffAI:mainfrom
yolandaycao:fix-buffbench-env-strings
Open

yolandaycao wants to merge 1 commit into
CodebuffAI:mainfrom
yolandaycao:fix-buffbench-env-strings

Conversation

@yolandaycao

Copy link
Copy Markdown

BuffBench configs can contain numeric env values like PORT: 3003. The SDK calls .trim() when scrubbing credentials, so terminal commands crash before starting.

Convert env values to strings before passing them to the SDK. Verified that printenv PORT succeeds with credential scrubbing enabled.

@codebuff-team

Copy link
Copy Markdown
Contributor

Good catch and a minimal, well-scoped fix. The root cause is clearly explained: BuffBench eval JSON configs can produce non-string env values (e.g. PORT: 3003) despite the TS type saying Record<string, string>, and the SDK's credential scrubber calls .trim() on env values, crashing on numbers/booleans. Coercing with String(value) in the runner before handing off to the SDK is a sensible place to guard, since the runner is the boundary where untrusted eval JSON enters typed code.

A couple of things worth double-checking before porting:

  • Object.fromEntries on undefined env would throw, but you've guarded with options.env && first, so that's fine.
  • No test was added. Given this is a one-line coercion with a clear bug report, that's a minor omission, but a small unit test on CodebuffRunner construction with a numeric env value would make this bulletproof and guard against regressions.
  • Consider whether the fix belongs at the type level too (loosening or validating the eval JSON schema) so future callers get a compile-time nudge, but that's a nice-to-have, not a blocker.

Overall: correct, narrowly scoped, addresses a real crash with a believable repro. Worth porting.

@codebuff-team codebuff-team added bot:triaged Classified by the community triage bot pr:port-candidate Worth porting into the private source tree labels Sep 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:triaged Classified by the community triage bot pr:port-candidate Worth porting into the private source tree

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants