Skip to content

Commit a546ead

Browse files
feat: v0.9.12 — probit-scale MTE (Heckman frame) + statspai CLI + sp.help introspection
- bayes_mte gains selection='uniform'|'normal' kwarg; 'normal' fits the MTE polynomial on V = Φ⁻¹(U_D) so (poly_u=1, hv_latent, normal) exactly recovers the linear Heckman MTE slope on a bivariate-normal DGP. - Shared PROBIT_CLIP in bayes/_base keeps fit / ATT-ATU integrator / policy_effect on the same numerical footing; BayesianMTEResult carries the selection tag so policy_effect transforms the grid to V scale. - mte_curve adds a 'v' column under selection='normal'. - Round-C review fixed 2 BLOCKERs (ATT/ATU + policy_effect had been raising U-scale powers against a V-scale posterior) and added tests/test_bayes_mte_selection.py (12 tests, 78 focused MTE tests green). - New `statspai` console_scripts entry → statspai.cli:main, plus statspai.help module providing sp.help / sp.list_functions / sp.search_functions backed by registry auto-registration from statspai.__all__. - Author email updated to brycew6m@stanford.edu across pyproject, __init__, and Cargo.toml. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 1a8d89d commit a546ead

12 files changed

Lines changed: 1542 additions & 26 deletions

File tree

CHANGELOG.md

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,82 @@
22

33
All notable changes to StatsPAI will be documented in this file.
44

5+
## [0.9.12] - 2026-04-20 — Probit-scale MTE (Heckman selection frame)
6+
7+
Adds the third orthogonal axis to `sp.bayes_mte`: the MTE polynomial
8+
can now be fit on either the uniform scale `U_D ∈ [0, 1]`
9+
(v0.9.11 default) or the probit / V scale
10+
`V = Φ^{-1}(U_D) ∈ ℝ` — the conventional Heckman (1979) / HV 2005
11+
frame. All `(first_stage, mte_method, selection)` combinations fit.
12+
13+
### Added (0.9.12)
14+
15+
- **`sp.bayes_mte(..., selection='uniform' | 'normal')`** — new kwarg.
16+
- `'uniform'` (default) preserves v0.9.11 behaviour: polynomial
17+
in `U_D ∈ [0, 1]`.
18+
- `'normal'` reinterprets the abscissa as `V = Φ^{-1}(U_D)` via
19+
`pt.sqrt(2) * pt.erfinv(2a-1)` on the tensor side and
20+
`scipy.stats.norm.ppf` on numpy side. Under strict HV + bivariate-
21+
normal, `poly_u=1 + selection='normal' + mte_method='hv_latent'`
22+
exactly recovers the linear Heckman MTE slope.
23+
24+
- **`mte_curve` exposes `v` column** under `selection='normal'`
25+
(empty otherwise) so users can plot on the scale their model
26+
was fit on.
27+
28+
- **Shared `PROBIT_CLIP` constant** in `statspai.bayes._base`
29+
fit-time, ATT/ATU integrator, and `policy_effect` all read the
30+
same clip so the three paths stay numerically consistent.
31+
32+
### Empirical recovery on Heckman DGP (true `(b_0, b_1) = (0.5, 1.5)`)
33+
34+
| combo | `b_0` | `b_1` |
35+
|---|---|---|
36+
| plugin × polynomial × V | -0.73 | 0.82 |
37+
| plugin × hv_latent × V | 0.42 | 1.37 ✓ |
38+
| joint × polynomial × V | -0.73 | 0.81 |
39+
| joint × hv_latent × V | 0.46 | 1.40 ✓ |
40+
41+
Same story as earlier releases: `hv_latent` recovers truth;
42+
`polynomial` fits `g(v)` not `MTE(v)` and is biased.
43+
44+
### Round-B review found 2 BLOCKERS + 2 HIGHs; Round-C fixed all
45+
46+
1. **BLOCKER-1**: `_integrated_effect` (ATT/ATU) was raising `U_population`
47+
to polynomial powers directly, even under `'normal'` where the
48+
posterior is on V scale. **Fixed** — transforms to
49+
`Φ^{-1}(U_population)` first.
50+
2. **BLOCKER-2**: `BayesianMTEResult.policy_effect` computed
51+
`u_pow = [u^k ...]` instead of `[v^k ...]` under `'normal'`,
52+
silently integrating a V-scale polynomial against u-scale powers.
53+
**Fixed**`BayesianMTEResult` now carries a `selection` field,
54+
and `policy_effect` transforms the grid to V scale when needed.
55+
Regression test asserts `policy_effect(policy_weight_ate())`
56+
matches `.ate` to 1e-8 under `'normal'`.
57+
3. **HIGH-1**: `mte_curve` lacked a `v` column — **added**.
58+
4. **Round-C follow-up**: extracted `PROBIT_CLIP = 1e-6` to a shared
59+
module constant consumed by both `mte.py` and `_base.py` so the
60+
three-site fit/summary/policy paths cannot drift.
61+
62+
### Tests (0.9.12)
63+
64+
- `tests/test_bayes_mte_selection.py` (NEW, 12 tests) — back-compat,
65+
method-label, Heckman DGP recovery, all-8-combo orthogonality,
66+
input validation, `v` column presence/absence, ATT/ATU V-scale
67+
correctness (Round-C regression), `policy_effect` V-scale
68+
parity with `.ate` (Round-C regression), uniform-vs-normal
69+
non-trivial disagreement.
70+
- 78 focused MTE tests green.
71+
72+
### Non-goals (0.9.12)
73+
74+
- Full bivariate-normal error covariance `(U_0, U_1, V) ~ N(0, Σ)`
75+
with free `ρ_{0V}`, `ρ_{1V}` — convergence-intensive MvNormal
76+
mixture, queued for 0.9.13+.
77+
- Rust Phase 2 — separate branch.
78+
79+
---
80+
581
## [0.9.11] - 2026-04-20 — Multi-instrument MTE + true CHV-2011 PRTE weights
682

783
Closes two long-standing API gaps plus an empirical math debt.
Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# StatsPAI v0.9.12 — Probit-scale MTE (Heckman selection frame)
2+
3+
**Author:** Bryce Wang · **Date:** 2026-04-20 · **Status:** design → implementation
4+
5+
## 1. Motivation
6+
7+
v0.9.11 handled multi-IV MTE and the true CHV-2011 PRTE weight. The one remaining thread in the Bayesian MTE frontier is the **Heckman (1979) selection model** with Gaussian/probit-scale errors, which is the conventional parametric frame underlying HV 2005:
8+
9+
```
10+
V_i ~ N(0, 1)
11+
D_i = 1{ logit(p_i) + V_i > 0 } (probit / normal-tail selection)
12+
MTE(v) = μ_1(X) - μ_0(X) + (ρ_{1V}σ_1 - ρ_{0V}σ_0) · v (linear in V)
13+
```
14+
15+
Under the strict bivariate-normal HV assumption, MTE is linear on the V scale. Our existing polynomial-in-U_D model (U_D ∈ [0,1]) is richer but parametrised on the uniform scale; a user who wants the conventional Heckman interpretation needs the V-scale formulation.
16+
17+
v0.9.12 adds a third orthogonal axis to `sp.bayes_mte`:
18+
19+
| axis | options |
20+
|---|---|
21+
| `first_stage` | `'plugin'` / `'joint'` |
22+
| `mte_method` | `'polynomial'` / `'hv_latent'` |
23+
| **`selection`** (new) | `'uniform'` (default) / `'normal'` |
24+
25+
Together these describe the 8-combo grid of Bayesian MTE specifications. All 8 are expected to fit without error; recovery characteristics differ by DGP.
26+
27+
## 2. Scope
28+
29+
### In scope (v0.9.12)
30+
31+
- **`sp.bayes_mte(..., selection='uniform' | 'normal')`** — new kwarg. `'uniform'` (default) preserves v0.9.11 behaviour. `'normal'` reinterprets the polynomial abscissa as `V_i = Φ^(-1)(U_D_i)` (probit scale, V ∈ ℝ). All combinations with existing `first_stage` / `mte_method` flags supported.
32+
- **Mathematical identity guarantee**: `'normal'` mode fits the MTE polynomial in V. Under `poly_u=1` + `selection='normal'` + `mte_method='hv_latent'` the model exactly matches the linear Heckman-HV MTE.
33+
- **Method label** reflects the selection scale: `"Bayesian MTE on V scale (...)"` vs `"Bayesian MTE on U_D scale (...)"`.
34+
35+
### Out of scope (deferred to 0.9.13+)
36+
37+
- **Full bivariate-normal error covariance** `(U_0, U_1, V) ~ N(0, Σ)` with free correlations `ρ_{0V}`, `ρ_{1V}`. Requires explicit mixture modelling and MvNormal over `(Y, D)` that has known convergence pathologies in PyMC. Tracked as a separate release.
38+
- Selection-on-levels (`μ_0``μ_1`) as a free parameter: currently absorbed into `alpha + beta_X·X`. Disentangling requires the full mixture model.
39+
- Rust Phase 2, VI backends for MTE.
40+
41+
## 3. API
42+
43+
```python
44+
def bayes_mte(
45+
data, y, treat, instrument, covariates=None,
46+
*,
47+
first_stage: str = 'plugin',
48+
mte_method: str = 'polynomial',
49+
selection: str = 'uniform', # NEW
50+
u_grid: np.ndarray | None = None,
51+
poly_u: int = 2,
52+
...
53+
)
54+
```
55+
56+
### Semantics
57+
58+
Under `selection='uniform'` (v0.9.11 default):
59+
```
60+
Abscissa a ∈ [0, 1] (either p_i in polynomial mode or U_D_i in hv_latent)
61+
MTE-curve: τ(a) = Σ_k b_k · a^k, a ∈ u_grid ⊂ [0, 1]
62+
```
63+
64+
Under `selection='normal'`:
65+
```
66+
Abscissa v ∈ ℝ with v = Φ^{-1}(a)
67+
MTE-curve: τ(v) = Σ_k b_k · v^k, v ∈ Φ^{-1}(u_grid) ⊂ ℝ
68+
```
69+
70+
The returned `mte_curve` DataFrame still has a `u` column (in [0,1]) for user convenience — this is the natural propensity scale. The `v` column is added when `selection='normal'` so users can see the probit coordinate.
71+
72+
ATE / ATT / ATU integrals: on the V scale, we integrate against the Gaussian density `φ(v) dv` rather than uniform `du`. This matches the HV identified-integrand under bivariate-normal.
73+
74+
### Defaults justification
75+
76+
`'uniform'` stays the default because:
77+
1. No breaking change to v0.9.11 users.
78+
2. The `[0,1]` propensity scale is the agent-native abstraction most users want.
79+
3. `'normal'` is a purer "I'm doing Heckman textbook work" mode — users who need it will reach for it.
80+
81+
## 4. File plan
82+
83+
| File | Change |
84+
|---|---|
85+
| `src/statspai/bayes/mte.py` | Add `selection` kwarg; transform abscissa via `Φ^{-1}` when `'normal'`; adjust ATE integration to Gaussian measure when `'normal'`; update method label + `model_info`. |
86+
| `src/statspai/bayes/_base.py` | `mte_curve` DataFrame adds a `v` column when fit was on V scale (empty otherwise). |
87+
| `tests/test_bayes_mte_selection.py` | NEW — `selection='normal'` recovery + orthogonality. |
88+
| `pyproject.toml` | `version = "0.9.12"` |
89+
| `CHANGELOG.md` | 0.9.12 entry |
90+
91+
## 5. Test plan
92+
93+
- `test_selection_uniform_back_compat` — scalar call with default `selection` returns v0.9.11 behaviour.
94+
- `test_selection_normal_api_surface` — new kwarg flows to `model_info['selection']`; method label mentions V scale.
95+
- `test_selection_normal_recovers_linear_heckman` — DGP with Gaussian V and MTE linear in V; `poly_u=1 + selection='normal' + hv_latent` recovers the slope.
96+
- `test_selection_normal_all_orthogonal_combos_run` — 4 combos (plugin/joint × polynomial/hv_latent) × selection='normal'.
97+
- `test_selection_invalid_value_raises`.
98+
- `test_selection_normal_mte_curve_has_v_column`.
99+
100+
## 6. Success criteria
101+
102+
1. `selection='normal'` runs on all 4 `(first_stage, mte_method)` combos.
103+
2. On a DGP with `MTE(V) = 0.5 + 1.5·V` (Gaussian latent V), `poly_u=1 + selection='normal' + hv_latent` recovers `(0.5, 1.5)` within HDI.
104+
3. Full suite stays green (zero-new-failures rule).
105+
4. Two rounds of code review, zero ship-blockers.
106+
107+
## 7. Non-goals (explicit)
108+
109+
- Free cross-correlation parameters `(ρ_{0V}, ρ_{1V})` — the bivariate-normal mixture is its own release.
110+
- Policy weights on V scale — current `policy_weight_*` builders still operate on the `[0,1]` grid; users on `selection='normal'` get `u_grid` transformed via `Φ` so policy integrals remain well-defined on the propensity scale.
111+
- Changing the sign / direction of `V = Φ^{-1}(p)` (we use the standard convention).

pyproject.toml

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,15 @@ build-backend = "setuptools.build_meta"
44

55
[project]
66
name = "StatsPAI"
7-
version = "0.9.11"
7+
version = "0.9.12"
88
description = "The Agent-Native Causal Inference & Econometrics Toolkit for Python"
99
readme = "README.md"
1010
license = {text = "MIT"}
1111
authors = [
12-
{name = "Biaoyue Wang", email = "bryce@copaper.ai"}
12+
{name = "Biaoyue Wang", email = "brycew6m@stanford.edu"}
1313
]
1414
maintainers = [
15-
{name = "Biaoyue Wang", email = "bryce@copaper.ai"}
15+
{name = "Biaoyue Wang", email = "brycew6m@stanford.edu"}
1616
]
1717
classifiers = [
1818
"Development Status :: 3 - Alpha",
@@ -89,6 +89,9 @@ tune = [
8989
"optuna>=3.0",
9090
]
9191

92+
[project.scripts]
93+
statspai = "statspai.cli:main"
94+
9295
[project.urls]
9396
Homepage = "https://github.com/brycewang-stanford/statspai"
9497
Documentation = "https://github.com/brycewang-stanford/statspai#quick-example"

rust/statspai_hdfe/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
name = "statspai_hdfe"
33
version = "0.1.0-alpha.1"
44
edition = "2021"
5-
authors = ["Bryce Wang <bryce@copaper.ai>"]
5+
authors = ["Bryce Wang <brycew6m@stanford.edu>"]
66
description = "PyO3 + Rayon HDFE group-demean kernel for StatsPAI"
77
license = "MIT"
88
readme = "README.md"

src/statspai/__init__.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,9 +22,9 @@
2222
>>> sp.outreg2(result, filename="results.xlsx")
2323
"""
2424

25-
__version__ = "0.9.11"
25+
__version__ = "0.9.12"
2626
__author__ = "Biaoyue Wang"
27-
__email__ = "bryce@copaper.ai"
27+
__email__ = "brycew6m@stanford.edu"
2828

2929
from .core.results import EconometricResults, CausalResult
3030
from .regression.ols import regress

src/statspai/bayes/_base.py

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,16 @@
2626
)
2727

2828

29+
# Shared clip constant for the probit-scale (``selection='normal'``)
30+
# transform. Any site computing ``Φ^{-1}(a)`` — the fit-time
31+
# polynomial powers, the post-hoc ATT/ATU integrator, and
32+
# ``policy_effect`` — MUST use this constant so the three paths stay
33+
# numerically consistent (cf. v0.9.12 round-C review HIGH on clip
34+
# drift between fit and policy_effect). Tightening requires
35+
# one-line change here; all callers pick it up automatically.
36+
PROBIT_CLIP: float = 1e-6
37+
38+
2939
def _require_pymc():
3040
"""Import PyMC and ArviZ, or raise a clear ImportError."""
3141
try:
@@ -381,6 +391,11 @@ class BayesianMTEResult(BayesianCausalResult):
381391
ate: float = float('nan')
382392
att: float = float('nan')
383393
atu: float = float('nan')
394+
# v0.9.12: `'uniform'` (default) fits MTE polynomial in U_D on
395+
# [0,1]; `'normal'` fits in V = Φ^{-1}(U_D) on ℝ. The result
396+
# remembers this so `policy_effect` can transform the abscissa
397+
# before integrating against the user's weight_fn.
398+
selection: str = 'uniform'
384399

385400
def policy_effect(
386401
self,
@@ -422,7 +437,18 @@ def policy_effect(
422437
poly_u = b_mte_post.shape[-1] - 1
423438
flat = b_mte_post.reshape(-1, poly_u + 1)
424439
u = np.asarray(self.u_grid, dtype=float)
425-
u_pow = np.column_stack([u ** k for k in range(poly_u + 1)])
440+
441+
# v0.9.12: respect the selection scale. Under
442+
# ``selection='normal'`` the polynomial is in ``v = Φ^{-1}(u)``
443+
# so the abscissa-powers must be built on v, not u. The
444+
# weight_fn is still passed u (the natural scale for users)
445+
# but we transform internally before dotting with b_mte.
446+
if self.selection == 'normal':
447+
from scipy.stats import norm as _norm_dist
448+
abscissa = _norm_dist.ppf(np.clip(u, PROBIT_CLIP, 1 - PROBIT_CLIP))
449+
else:
450+
abscissa = u
451+
u_pow = np.column_stack([abscissa ** k for k in range(poly_u + 1)])
426452
mte_samples = flat @ u_pow.T # (S, n_grid)
427453
weights = np.asarray(weight_fn(u), dtype=float)
428454
if weights.shape != u.shape:

0 commit comments

Comments
 (0)