fix(usage): report Qwen's real usage instead of the local estimate - #175
Merged
Merged
Conversation
Qwen sends `usage` on every frame under DashScope names (input_tokens / output_tokens, cumulative). /v1/messages read it under the OpenAI names (prompt_tokens / completion_tokens), so the proxy ALWAYS fell back to the tiktoken estimate; /v1/chat/completions read only the last frame's `usage` and the agent runtime never surfaced it. Claude Code's context percentage therefore tracked our estimate, never Qwen's count. - precise-tokenizer.js: normalizeUpstreamUsage (accepts both naming schemes, zero = not reported, all-zero = absent), mergeUpstreamUsage (last frame wins), resolveUsage (per-field fallback to the local estimate), describeUsageSource (upstream only when both fields came from Qwen). - anthropic.js: stream and non-stream /v1/messages report the upstream numbers; cache fields are 0, not null. - chat.js: stream, non-stream and agent paths report the upstream numbers; the agent runtime now returns the accumulated usage of the accepted attempt (several attempts -> last attempt's usage, not the sum). - One `info` line per response: `usage source=upstream|estimated input=<n> output=<m>`, replacing chat.js's twin tiktoken/upstream lines. Tests: unit tests for the normalizer, and passthrough tests for both routes (stream, non-stream, agent, partial usage, all-zero usage, multi-attempt). Not verified here: a live probe against real Qwen (seam 3 of the spec) — pending on qwen-next. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…o reportUsage
Code review of db70f93 found two things the tests did not cover:
- Anthropic stream and non-stream never reset `upstreamUsage` when the
agent gate starts a second attempt, so a later attempt that reported
nothing inherited the first attempt's counters. Spec says the reported
usage is the last attempt's, for both counters. Reset it per attempt
(same place the per-attempt text/think buffers are reset) and cover it
with a two-attempt test for each handler.
- The `resolveUsage` + `logger.info('usage source=...')` pair was copied
at five call sites. Replace with `reportUsage(acc, estimate, tag)`,
which resolves, decides upstream/estimated from whether the estimate
actually ran, logs once and returns the usage. `describeUsageSource`
goes away with it.
Also drop the leftover `Math.max(0, ...)` clamp and `total_tokens`
recompute in chat.js (resolveUsage already guarantees both), and the
numeric-string coercion in `toReportedCount` (no evidence Qwen sends
strings). Test vocabulary follows CONTEXT.md: "attempt", not "retry".
Gate: 1147 tests / 134 suites (was 1146/134), re-blessed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… 134 suites) The two usage commits were cherry-picked onto upstream main; only tests/expected-counts.json clashed. Re-recorded with npm run test:bless under CI's env (API_KEY=ci-test-key DATA_SAVE_MODE=none ACCOUNTS=''). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
/v1/chat/completionsand/v1/messagesnever reported Qwen's real token usage. Every response (stream and non-stream) carried a local tiktoken estimate, even though Qwen sendsusageon the final frame (stream) / in the body (non-stream). Clients that budget onusage(Claude Code's context-window percentage, dashboards) saw numbers that could be off by ~10 % or by tens of thousands of tokens (the attached-file history is invisible to the local estimate).Fix
precise-tokenizer.js):normalizeUpstreamUsageaccepts OpenAI (prompt_tokens/completion_tokens) and DashScope (input_tokens/output_tokens) naming, coerces numeric strings, treats negative / NaN / non-object as absent.mergeUpstreamUsage— last frame wins per field.resolveUsage— per-field fallback to the local estimate only when a field is genuinely absent (an upstream0is a real0)./v1/messages): stream reads Qwen'susagefrom every frame, emits realinput_tokensonmessage_startand realoutput_tokensonmessage_delta; non-stream twin viabody.usage. Across retries, the accepted (last) attempt's usage is what gets reported./v1/chat/completions): stream final chunk and non-stream body carry Qwen's counts; agent runtime reports the accepted attempt's usage.cache_creation_input_tokens,cache_read_input_tokens) are0, notnull: Qwen has no prompt cache.usage source=<upstream|estimated> input=<n> output=<m>.Tests
tests/upstream-usage.test.js(normalizer / merge / resolve unit cases),tests/anthropic-usage-passthrough.test.js,tests/openai-usage-passthrough.test.js(controllers fed synthetic upstream frames: stream, non-stream, partial usage, all-zero, absent → estimated, several attempts → last wins).tests/expected-counts.jsonre-blessed: 1115 → 1146 tests, 134 suites.Live check against real Qwen (qwen3.8-max,
hiprompt):input_tokens=651on both APIs, stream and non-stream, and both containers logusage source=upstream. Not automatable (needs a real account), so not in the suite.Notes
Rebased onto
a947146; no overlap with the SOCKS5 work. Reasoning tokens: Qwen exposes no separate counter, sooutput_tokensis whatever Qwen reports; the local estimate (fallback only) still counts thinking text.Verified live on qwen-next (this branch @ 091fea6, real Qwen,
qwen3.8-max)/v1/messagesnon-stream "hi"/v1/messagesstream "hi"message_delta)/v1/messagesstream + tool/v1/messagesstream + thinking/v1/chat/completionsnon-stream "hi"/v1/chat/completionsstream "hi"/v1/chat/completionsnon-stream + toolfinish_reason=tool_callsEvery response logged
usage source=upstream; the estimate path never fired. Not covered live: "last attempt wins" on agent retries (no retry fired during the probe — unit-tested only).🤖 Generated with Claude Code