Skip to content

Commit 18539c2

Browse files
fix(tests): re-pin LaLonde PSM guard to both observed BLAS anchors
The af28802 tie-break fix achieved bitwise-identical matched sets across all 14 CI jobs (ubuntu/windows/macos x 3.9-3.13: 1967.9396843243246), but the guard was pinned to the local Accelerate/macOS-26 value (1963.43), failing every CI platform. Isolated-venv check with numpy 2.4.6/scipy 1.17.1 reproduces the local value, ruling out a library-version cause: the residual ~$4.5 spread is ULP-level near-tie sensitivity in BLAS-computed propensity distances, outside the exact-tie anchor's scope. Guard now pins both fixed points exactly (+-0.1) so any algorithm change or new third environment still fails loudly. CHANGELOG/MIGRATION corrected; CAMPAIGN.md records the sign-off and the optional epsilon-class follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 843ac07 commit 18539c2

4 files changed

Lines changed: 56 additions & 8 deletions

File tree

.tierd_campaign/CAMPAIGN.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -469,3 +469,24 @@ noisy/forest-dependent — anchored on β₁ + null structure instead.
469469
the LaLonde guard from the old cross-backend PSM band to the pinned
470470
`1963.43` ATT. CHANGELOG/MIGRATION document that only exact-tie designs can
471471
move.
472+
473+
### 2026-06-10 — tie-break fix: CI verdict + maintainer sign-off
474+
- Maintainer signed off on the deferred `sp.match` tie-break batch (af28802).
475+
- **CI evidence (run 27255448391):** all 14 test jobs — ubuntu (OpenBLAS),
476+
windows, macos-15 × Python 3.9–3.13 — produced a *bitwise-identical* LaLonde
477+
PSM ATT of `1967.9396843243246`. The exact-tie anchoring works: identical
478+
matched sets across BLAS builds, vs. the pre-fix $150 spread.
479+
- **Residual (documented, not hidden):** local Accelerate (macOS 26) lands at
480+
`1963.4266356756752` with BOTH numpy 2.3/scipy 1.16 and numpy 2.4/scipy 1.17
481+
(isolated-venv check) — so it is ULP-level *near*-tie sensitivity in the
482+
BLAS-computed propensity distances (distinct rows, distances differing
483+
~1e-13), not a library-version artifact and not the tie-break itself.
484+
- Guard re-pinned from the single local value to **two exact anchors ±0.1**
485+
(`1967.94` CI consensus, `1963.43` Accelerate/macOS-26); a third environment
486+
producing a new fixed point fails loudly for triage. CHANGELOG/MIGRATION
487+
corrected accordingly. Stale `lalonde()` dataset attr 2012.5 → 1963.4
488+
(843ac07; the long-pending documentation correction, now approved).
489+
- **Optional future batch (small):** epsilon-class tie merging (group distances
490+
whose successive sorted gaps are < rtol≈1e-9 before index tie-break) would
491+
remove the residual ~$4.5 cross-BLAS spread entirely. §12-grade numeric
492+
change → needs its own sign-off; not urgent, the two-anchor guard holds.

CHANGELOG.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -193,8 +193,12 @@ All notable changes to StatsPAI will be documented in this file.
193193
units first, with lower-index target units used as the without-replacement
194194
fallback. A shuffled-row regression test guards the policy, and
195195
`tests/test_tierD_lalonde_psm_guard.py` is tightened from the old
196-
cross-backend band to the pinned LaLonde PSM ATT (`1963.43`). Existing results
197-
change only when previous data contained exact equal-distance ties. See
196+
cross-backend band (width ~$300) to two exact anchors (±0.1): GitHub CI is
197+
now bitwise-identical across ubuntu/windows/macos (`1967.94`), while
198+
Accelerate on macOS 26 lands at `1963.43` — a residual ULP-level *near*-tie
199+
sensitivity in the BLAS-computed distances, not in the tie-break itself.
200+
Existing results change only when previous data contained exact
201+
equal-distance ties. See
198202
[MIGRATION.md](MIGRATION.md#matching-nearest-tie-break).
199203

200204
### Known issues

MIGRATION.md

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -56,12 +56,18 @@ units have the same best distance, lower-index treated units are assigned first.
5656
**Who is affected.** Only users whose matching data contain exact
5757
equal-distance ties. Continuous covariates without exact ties are unchanged.
5858
For tied designs, results are now deterministic across row order and backend as
59-
long as the DataFrame index preserves unit identity.
59+
long as the DataFrame index preserves unit identity. One caveat: distances that
60+
are merely *near*-equal (differing at the ~1e-13 ULP level because the BLAS
61+
build computes propensity scores slightly differently) are still resolved by
62+
strict comparison, so a residual backend sensitivity of that magnitude remains
63+
— on LaLonde it amounts to ~$4.5 (vs. ~$150 before the fix), and all GitHub CI
64+
platforms (ubuntu/windows/macos) now agree bitwise.
6065

6166
**Action required.** None for code. If you previously recorded a nearest-match
6267
estimate on tied discrete covariates, re-run it once and treat the new value as
63-
the stable pin. The bundled LaLonde 1:1 NN PSM guard is now pinned at `1963.43`
64-
instead of allowing the old cross-backend tie band.
68+
the stable pin. The bundled LaLonde 1:1 NN PSM guard now pins the two observed
69+
fixed points exactly (`1967.94` on GitHub CI, `1963.43` under Accelerate on
70+
macOS 26) instead of allowing the old ~$300 cross-backend tie band.
6571

6672
---
6773

tests/test_tierD_lalonde_psm_guard.py

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,24 @@ def test_adjusted_ols_matches_matchit(lalonde):
3838
assert float(adj.params["treat"]) == pytest.approx(1548.24, abs=1.0)
3939

4040

41+
# The index-anchored tie-break makes the matched set deterministic given the
42+
# floating-point distance matrix: GitHub CI lands on bitwise-identical
43+
# 1967.9396843243246 across ubuntu (OpenBLAS), windows, and macos-15 runners,
44+
# all Python 3.9-3.13. A residual ULP-level *near*-tie sensitivity remains in
45+
# the BLAS-computed propensity distances themselves (distinct covariate rows
46+
# whose distances differ by ~1e-13 can swap order across BLAS builds): local
47+
# Accelerate on macOS 26 lands on 1963.4266356756752 with both numpy 2.3/scipy
48+
# 1.16 and numpy 2.4/scipy 1.17, ruling out a library-version cause. Pin BOTH
49+
# observed fixed points exactly (±0.1): any algorithm change in sp.match moves
50+
# the value off both anchors and fails loudly, while a third BLAS environment
51+
# producing a new value also fails — surfacing it for triage instead of being
52+
# absorbed by a wide band. Tracked in .tierd_campaign/CAMPAIGN.md.
53+
PSM_ATT_ANCHORS = (1967.94, 1963.43)
54+
55+
4156
def test_psm_att_is_deterministic_and_pinned(lalonde):
42-
# Deterministic across runs; pinned to the current value so any tie-break /
43-
# algorithm change in sp.match is caught (the guard that was missing before).
57+
# Deterministic across runs; pinned to the known fixed points so any
58+
# tie-break / algorithm change in sp.match is caught.
4459
vals = [
4560
float(
4661
sp.match(
@@ -50,7 +65,9 @@ def test_psm_att_is_deterministic_and_pinned(lalonde):
5065
for _ in range(3)
5166
]
5267
assert len(set(round(v, 6) for v in vals)) == 1 # deterministic within a run
53-
assert vals[0] == pytest.approx(1963.43, abs=0.1)
68+
assert any(
69+
vals[0] == pytest.approx(anchor, abs=0.1) for anchor in PSM_ATT_ANCHORS
70+
), f"PSM ATT {vals[0]:.4f} is off both pinned anchors {PSM_ATT_ANCHORS}"
5471

5572

5673
def test_psm_recovers_experimental_benchmark(lalonde):

0 commit comments

Comments
 (0)