test(openrouter): set cached-token fields in mock to stop intermittent MagicMock crash#2776
Merged
Merged
Conversation
…t MagicMock crash test_null_content_recovers_on_retry failed intermittently on the test-api shard with: hindsight_api/metrics.py:591: TypeError: '>' not supported between instances of 'MagicMock' and 'int' (if cached_input_tokens > 0) The mock in _make_chat_response set completion_tokens_details but not the cached-token fields, so the cached-token extraction (openai_compatible_llm.py:948 `response_usage.cached_tokens`, and the prompt_tokens_details path) read an auto-MagicMock and passed it to the metrics recorder. It only surfaced when the metrics path actually ran — which depends on telemetry state that leaks across pytest-xdist workers — so it presented as an intermittent, co-scheduling-dependent failure rather than a deterministic one. Set usage.cached_tokens = 0 and usage.prompt_tokens_details = None so both extraction paths yield int 0. Verified: both tests pass and both paths return int 0 (no MagicMock reaches the `> 0` comparison).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
test_null_content_recovers_on_retryfailed intermittently on thetest-apishard:(
if cached_input_tokens > 0:)_make_chat_responsesetcompletion_tokens_details(added for #2378) but not the cached-token fields. So the cached-token extraction —openai_compatible_llm.py:948(response_usage.cached_tokens) and theusage.prompt_tokens_details.cached_tokenspath — read an auto-MagicMockand passed it to the metrics recorder. It only surfaced when the metrics path actually ran, which depends on telemetry state that leaks across pytest-xdist workers — so it presented as an intermittent, co-scheduling-dependent failure (passed on some runs, failed on others with identical code).Fix: set
usage.cached_tokens = 0andusage.prompt_tokens_details = Noneso both extraction paths yield int0.Verified locally: both tests pass, and both extraction paths return int
0(noMagicMockreaches the> 0comparison).Context: this was the lone remaining red on #2727 after the native-import (#2761) and judge-hardening (#2769) fixes — a pre-existing flake unrelated to that PR's dependency bumps.
🤖 Generated with Claude Code