Skip to content

Commit 85c0081

Browse files
committed
fix(test): seed torch._inductor.test_operators to fix test-api shard failures
test-api's reranker-bearing shard (consistently 2/3) has been failing on every run with a misleading "sentence-transformers is required for LocalSTEmbeddings" ImportError. The real cause is torch, not the deps: import torch._inductor.test_operators torch/_inductor/test_operators.py:8: _test_lib_def = torch.library.Library("_inductor_test", "DEF") RuntimeError: Only a single TORCH_LIBRARY can be used to register the namespace _inductor_test ... transformers' lazy loader imports torch._inductor.test_operators while resolving AutoModelForSequenceClassification / GenerationMixin (used by the cross-encoder). Under pytest-xdist that module's body executes twice, and its module-level `_inductor_test` TORCH_LIBRARY re-registration is rejected by torch. transformers wraps the RuntimeError and re-raises it as the sentence-transformers ImportError, so the symptom points at the wrong dep. This is the same class of bug the adjacent `import torch` seed already guards against (torch/overrides.py double-init); extend it to the inductor test-operators module. Seeding it once at conftest import puts it in sys.modules so the later lazy import is a cache hit and the body never re-executes. Verified locally that a re-import after seeding is a no-op (no RuntimeError). Guarded so slim/no-torch envs still collect. Reproduces independent of dependency versions (seen at transformers 5.3.0 and 5.12.1, torch 2.10 and 2.12), which is why it blocked every recent uv.lock PR including Dependabot's.
1 parent c27fafb commit 85c0081

1 file changed

Lines changed: 13 additions & 0 deletions

File tree

hindsight-api-slim/tests/conftest.py

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,19 @@
2424
# per worker process. Guarded so slim/no-torch environments still collect.
2525
try:
2626
import torch # noqa: F401 # eager one-time init; see comment above
27+
28+
# Same class of problem, different torch module. transformers' lazy loader
29+
# imports `torch._inductor.test_operators` while resolving classes such as
30+
# AutoModelForSequenceClassification / GenerationMixin (exercised by the
31+
# cross-encoder / reranker tests). That module registers an `_inductor_test`
32+
# TORCH_LIBRARY namespace at module-body level, and under pytest-xdist its
33+
# body can execute twice, raising "Only a single TORCH_LIBRARY can be used
34+
# to register the namespace _inductor_test". The failure surfaces on
35+
# whichever shard runs the reranker tests, masked by transformers as a
36+
# misleading "sentence-transformers is required for LocalSTEmbeddings"
37+
# ImportError. Seed it once here so the later lazy import is a sys.modules
38+
# cache hit and the body never re-executes.
39+
import torch._inductor.test_operators # noqa: F401 # see comment above
2740
except ImportError:
2841
pass
2942

0 commit comments

Comments
 (0)