|
| 1 | +# StatsPAI v0.9.12 — Probit-scale MTE (Heckman selection frame) |
| 2 | + |
| 3 | +**Author:** Bryce Wang · **Date:** 2026-04-20 · **Status:** design → implementation |
| 4 | + |
| 5 | +## 1. Motivation |
| 6 | + |
| 7 | +v0.9.11 handled multi-IV MTE and the true CHV-2011 PRTE weight. The one remaining thread in the Bayesian MTE frontier is the **Heckman (1979) selection model** with Gaussian/probit-scale errors, which is the conventional parametric frame underlying HV 2005: |
| 8 | + |
| 9 | +``` |
| 10 | +V_i ~ N(0, 1) |
| 11 | +D_i = 1{ logit(p_i) + V_i > 0 } (probit / normal-tail selection) |
| 12 | +MTE(v) = μ_1(X) - μ_0(X) + (ρ_{1V}σ_1 - ρ_{0V}σ_0) · v (linear in V) |
| 13 | +``` |
| 14 | + |
| 15 | +Under the strict bivariate-normal HV assumption, MTE is linear on the V scale. Our existing polynomial-in-U_D model (U_D ∈ [0,1]) is richer but parametrised on the uniform scale; a user who wants the conventional Heckman interpretation needs the V-scale formulation. |
| 16 | + |
| 17 | +v0.9.12 adds a third orthogonal axis to `sp.bayes_mte`: |
| 18 | + |
| 19 | +| axis | options | |
| 20 | +|---|---| |
| 21 | +| `first_stage` | `'plugin'` / `'joint'` | |
| 22 | +| `mte_method` | `'polynomial'` / `'hv_latent'` | |
| 23 | +| **`selection`** (new) | `'uniform'` (default) / `'normal'` | |
| 24 | + |
| 25 | +Together these describe the 8-combo grid of Bayesian MTE specifications. All 8 are expected to fit without error; recovery characteristics differ by DGP. |
| 26 | + |
| 27 | +## 2. Scope |
| 28 | + |
| 29 | +### In scope (v0.9.12) |
| 30 | + |
| 31 | +- **`sp.bayes_mte(..., selection='uniform' | 'normal')`** — new kwarg. `'uniform'` (default) preserves v0.9.11 behaviour. `'normal'` reinterprets the polynomial abscissa as `V_i = Φ^(-1)(U_D_i)` (probit scale, V ∈ ℝ). All combinations with existing `first_stage` / `mte_method` flags supported. |
| 32 | +- **Mathematical identity guarantee**: `'normal'` mode fits the MTE polynomial in V. Under `poly_u=1` + `selection='normal'` + `mte_method='hv_latent'` the model exactly matches the linear Heckman-HV MTE. |
| 33 | +- **Method label** reflects the selection scale: `"Bayesian MTE on V scale (...)"` vs `"Bayesian MTE on U_D scale (...)"`. |
| 34 | + |
| 35 | +### Out of scope (deferred to 0.9.13+) |
| 36 | + |
| 37 | +- **Full bivariate-normal error covariance** `(U_0, U_1, V) ~ N(0, Σ)` with free correlations `ρ_{0V}`, `ρ_{1V}`. Requires explicit mixture modelling and MvNormal over `(Y, D)` that has known convergence pathologies in PyMC. Tracked as a separate release. |
| 38 | +- Selection-on-levels (`μ_0` ≠ `μ_1`) as a free parameter: currently absorbed into `alpha + beta_X·X`. Disentangling requires the full mixture model. |
| 39 | +- Rust Phase 2, VI backends for MTE. |
| 40 | + |
| 41 | +## 3. API |
| 42 | + |
| 43 | +```python |
| 44 | +def bayes_mte( |
| 45 | + data, y, treat, instrument, covariates=None, |
| 46 | + *, |
| 47 | + first_stage: str = 'plugin', |
| 48 | + mte_method: str = 'polynomial', |
| 49 | + selection: str = 'uniform', # NEW |
| 50 | + u_grid: np.ndarray | None = None, |
| 51 | + poly_u: int = 2, |
| 52 | + ... |
| 53 | +) |
| 54 | +``` |
| 55 | + |
| 56 | +### Semantics |
| 57 | + |
| 58 | +Under `selection='uniform'` (v0.9.11 default): |
| 59 | +``` |
| 60 | +Abscissa a ∈ [0, 1] (either p_i in polynomial mode or U_D_i in hv_latent) |
| 61 | +MTE-curve: τ(a) = Σ_k b_k · a^k, a ∈ u_grid ⊂ [0, 1] |
| 62 | +``` |
| 63 | + |
| 64 | +Under `selection='normal'`: |
| 65 | +``` |
| 66 | +Abscissa v ∈ ℝ with v = Φ^{-1}(a) |
| 67 | +MTE-curve: τ(v) = Σ_k b_k · v^k, v ∈ Φ^{-1}(u_grid) ⊂ ℝ |
| 68 | +``` |
| 69 | + |
| 70 | +The returned `mte_curve` DataFrame still has a `u` column (in [0,1]) for user convenience — this is the natural propensity scale. The `v` column is added when `selection='normal'` so users can see the probit coordinate. |
| 71 | + |
| 72 | +ATE / ATT / ATU integrals: on the V scale, we integrate against the Gaussian density `φ(v) dv` rather than uniform `du`. This matches the HV identified-integrand under bivariate-normal. |
| 73 | + |
| 74 | +### Defaults justification |
| 75 | + |
| 76 | +`'uniform'` stays the default because: |
| 77 | +1. No breaking change to v0.9.11 users. |
| 78 | +2. The `[0,1]` propensity scale is the agent-native abstraction most users want. |
| 79 | +3. `'normal'` is a purer "I'm doing Heckman textbook work" mode — users who need it will reach for it. |
| 80 | + |
| 81 | +## 4. File plan |
| 82 | + |
| 83 | +| File | Change | |
| 84 | +|---|---| |
| 85 | +| `src/statspai/bayes/mte.py` | Add `selection` kwarg; transform abscissa via `Φ^{-1}` when `'normal'`; adjust ATE integration to Gaussian measure when `'normal'`; update method label + `model_info`. | |
| 86 | +| `src/statspai/bayes/_base.py` | `mte_curve` DataFrame adds a `v` column when fit was on V scale (empty otherwise). | |
| 87 | +| `tests/test_bayes_mte_selection.py` | NEW — `selection='normal'` recovery + orthogonality. | |
| 88 | +| `pyproject.toml` | `version = "0.9.12"` | |
| 89 | +| `CHANGELOG.md` | 0.9.12 entry | |
| 90 | + |
| 91 | +## 5. Test plan |
| 92 | + |
| 93 | +- `test_selection_uniform_back_compat` — scalar call with default `selection` returns v0.9.11 behaviour. |
| 94 | +- `test_selection_normal_api_surface` — new kwarg flows to `model_info['selection']`; method label mentions V scale. |
| 95 | +- `test_selection_normal_recovers_linear_heckman` — DGP with Gaussian V and MTE linear in V; `poly_u=1 + selection='normal' + hv_latent` recovers the slope. |
| 96 | +- `test_selection_normal_all_orthogonal_combos_run` — 4 combos (plugin/joint × polynomial/hv_latent) × selection='normal'. |
| 97 | +- `test_selection_invalid_value_raises`. |
| 98 | +- `test_selection_normal_mte_curve_has_v_column`. |
| 99 | + |
| 100 | +## 6. Success criteria |
| 101 | + |
| 102 | +1. `selection='normal'` runs on all 4 `(first_stage, mte_method)` combos. |
| 103 | +2. On a DGP with `MTE(V) = 0.5 + 1.5·V` (Gaussian latent V), `poly_u=1 + selection='normal' + hv_latent` recovers `(0.5, 1.5)` within HDI. |
| 104 | +3. Full suite stays green (zero-new-failures rule). |
| 105 | +4. Two rounds of code review, zero ship-blockers. |
| 106 | + |
| 107 | +## 7. Non-goals (explicit) |
| 108 | + |
| 109 | +- Free cross-correlation parameters `(ρ_{0V}, ρ_{1V})` — the bivariate-normal mixture is its own release. |
| 110 | +- Policy weights on V scale — current `policy_weight_*` builders still operate on the `[0,1]` grid; users on `selection='normal'` get `u_grid` transformed via `Φ` so policy integrals remain well-defined on the propensity scale. |
| 111 | +- Changing the sign / direction of `V = Φ^{-1}(p)` (we use the standard convention). |
0 commit comments