[Bug]: higgs_audio_v3 test_concurrent_pcm_streaming consistently fails HNR threshold (concurrent PCM streaming)

[tzhouam]

Summary

tests/e2e/online_serving/test_higgs_audio_v3.py::TestHiggsAudioV3OnlineHappyPath::test_concurrent_pcm_streaming[higgs_audio_v3_plain_text] fails deterministically:

AssertionError: Audio distortion detected: HNR=-2.73 dB < -2.0 dB. Voice clone decoder may be losing ref_code speaker context on later chunks.

The 3-way concurrent PCM-streaming case measures HNR ~0.7-1.3 dB below the -2.0 dB floor, while the single-request streaming case passes (~0.8 dB). This is a higgs_audio_v3_plain_text request (no ref_audio / voice clone), so the assertion's stock "Voice clone decoder may be losing ref_code speaker context" hint is generic boilerplate, not specific to this case.

Reproducible / deterministic (not flaky)

Same failure on both main and the rebase branch, across multiple runs:

Where	HNR
main @ 31886d6e (3x parallel, identical)	-2.73 dB x3
main @ d75a6433	-3.25 dB
dev/vllm-align (build #2125)	-2.94 dB
dev/vllm-align (build #2158)	-2.94 dB

3 parallel runs on main produced byte-identical -2.73 dB -> deterministic (fixed seed), not a flake. Single-stream passes; only the 3-way concurrent batch dips below threshold. Not introduced by any rebase - main fails it identically.

Note on CI visibility

This test is diff-gated out of main CI builds (the ready pipeline if: is false on main), so main builds appear green without exercising it. It runs (and fails) on nightly / ready-labeled builds.

Threshold context

-2.0 dB is already a Higgs-specific relaxed min_hnr_db override (the global default is 1.0 dB); the concurrent-streaming case still falls below it. Related to the HNR-threshold-too-strict pattern in #4335 / #4390 (VoxCPM2), but those do not cover this Higgs case.

Suggested investigation

Either the concurrent-streaming decode genuinely degrades audio quality (real bug) and needs a fix, or the metric/threshold for the batched-concurrent case needs adjusting (e.g. median across streams instead of worst-of-N). The test is being temporarily skipped pending this investigation.

Environment

• vLLM 0.22 (main) / 0.23 (dev/vllm-align), CUDA 13, torch 2.11
• Reproduces on both H100 (CI) and L20X (local)

[Gaohan123]

@yuekaizhang PTAL. Thanks

[yenuo26]

In addition, there is also an inconsistency issue between the generated audio and the text content during concurrent execution.
you can run
pytest -sv tests/e2e/online_serving/test_higgs_audio_v3.py::TestHiggsAudioV3OnlineHappyPath::test_concurrent_plain_text[higgs_audio_v3_plain_text] --run-level advanced_model

error log

audio content is: It was the night before my birthday.
input text is: It was the night before my birthday.
cosine similarity text1 is: it was the night before my birthday, text2 is: it was the night before my birthday
Cosine similarity: 1.000
audio content is: It was the night before my birthday.
input text is: It was the night before my birthday.
cosine similarity text1 is: it was the night before my birthday, text2 is: it was the night before my birthday
Cosine similarity: 1.000
audio content is: It was the night before my birth was the night which was my birthday
input text is: It was the night before my birthday.
cosine similarity text1 is: it was the night before my birth was the night which was my birthday, text2 is: it was the night before my birthday
Cosine similarity: 0.611

[Gaohan123]

@herotai214 PTAL

[herotai214]

Thank you. I think I can reproduce the issue (with ~-2.74 dB). Futher investigations be done the coming few days for a good quality fix.

I suspect the input "She sells seashells by the seashore." for the concurrent test is the culprit for causing a lower HNR.

You mentioned "the single-request streaming case passes (~0.8 dB)"; Its input in the pytest is "The quick brown fox jumps over the lazy dog.", different from concurrent case.
I tried to run single-request streaming with "She sells seashells by the seashore." -> HNR=-0.65 (failed)

And after hearing the 3 .wav outputs produced from test_concurrent_pcm_streaming, I dont think they are noisy (bug); They have similar, or even better audio quality than the single-request streaming "She sells seashells by the seashore."...