fix(benchmarks): bound Gemini calls and fail-fast on hung bundle batches

[Davincc77]

Summary

Bundle-based Test B long pilot stalled twice only for bundle_index=4 (b05_drone_mission_ops) on the same head SHA 9448382 with identical params (sessions_per_bundle=150, concurrency=2, retry_max=5, retry_backoff=2, retry_backoff_max=30). Runs 26634227315 and 26637685221 both wedged on the Pilot Test B bundles — execute against Gemini step for >80 min before being cancelled. Bundles 0..3 on the same SHA completed cleanly.

Diagnosis

Two compounding bugs, both independent of the Drone/Mission Ops fixtures themselves:

1. No per-request timeout on Gemini. GeminiProvider.generate did not forward ProviderConfig.timeout_s (60s) to the SDK. google-genai has no client-side deadline in that path, so a stalled TLS read — a known Gemini behaviour around safety-filtered content, and the Drone/Mission Ops bundle deliberately mixes "drone-operator", "mission-control", "security-incident-response", "human veto on a security-relevant step", and "CI weakening resistance" content — can block indefinitely with no error to retry.
2. ThreadPoolExecutor context-manager deadlock. executor_b_bundles ran each batch through with ThreadPoolExecutor(...) as pool: list(pool.map(...)). __exit__ joins all workers, so one hung worker wedges the entire 1800-call run with zero log output — exactly what the GitHub Actions UI showed.

Mapping (bundle_index=4 → b05_drone_mission_ops / Drone/Mission Ops) is correct; no off-by-one. First call is s001 / p01_scoping / no_memory (small prompt) — the stall is not at prompt construction. Maximum prompt size for bundle 4 (~70 KB) is comparable to bundle 0 (~68 KB); content is not pathologically large.

Fix

• Thread ProviderConfig.timeout_s → HttpOptions(timeout=<ms>) so every Gemini call has a 60s per-attempt deadline (configurable via new --request-timeout-s flag, also wired into the workflow execute step).
• Classify timeout exceptions as TransientProviderError so the retry loop can recover.
• Replace the ThreadPoolExecutor in the bundle executor with a queue + daemon-thread pool plus a per-call wall-clock cap derived from timeout_s * (retry_max+1) + retry_backoff_max * retry_max + 5s. A wedged worker no longer blocks shutdown; its job is logged in errors.jsonl as final_error_class=wall_clock_timeout and the batch proceeds.
• Add per-item progress logging to stderr ([bb] start N/M … / [bb] ok … / [bb] error … / [bb] WALL-TIMEOUT …) so GitHub Actions surfaces live progress.
• Add CI-safe regression probe tests/test_bundle4_hang_probe.py:
  • Pins bundle_index=4 → b05_drone_mission_ops.
  • Pins the first call expansion (scoping / no_memory).
  • Synthetic _HangingProvider proves the executor finishes within 30s with all calls classified as wall_clock_timeout.
  • Asserts the Gemini adapter forwards timeout_s as http_options.timeout (ms).

No completed benchmark results are altered. No publish / tag / release / Zenodo / npm / PyPI path touched. Harness still makes no Mem0 compatibility claim.

Test plan

• pytest benchmarks/v4.1/tests/ — 95 passed in 28.13s (90 pre-existing + 4 new + 1 incidental from import wiring)
• pytest benchmarks/v4.1/tests/test_bundle4_hang_probe.py — 4 passed in 8.30s (hang-probe runs in <10s against synthetic hanging provider)
• python -m py_compile on all touched files
• Re-run Benchmark v4.1 — Gemini Test B Bundle Long Pilot (controlled) with bundle_index=4 and verify it now either completes or fails with explicit wall_clock_timeout rows + visible [bb] progress logs (manual dispatch — not gated by this PR's CI)

🤖 Generated with Claude Code