Skip to content

Commit 36b99c4

Browse files
feat(replicate): dual-track classic + modern guides for 4 papers
Promote sp.replicate entries for Card 1995, Abadie 2010 (Prop 99), Lalonde 1986 (DW NSW+PSID-1, MatchIt subset), and Lee 2008 (Senate RD) from single-track stubs to full classic + modern recipes on real public-domain CSVs bundled with the package. - Bundle real CSVs: card_1995, california_prop99, lalonde_matchit (n=614), lee_2008_senate (n=1390) under src/statspai/datasets/data/. - sp.datasets.nsw_lalonde / lee_2008_senate gain a `simulated=False` real- data branch with pinned StatsPAI estimates and published benchmarks exposed via df.attrs; synthetic generators kept reachable as private helpers. - Lalonde classic track: Naive OLS / Adjusted OLS / 1:1 NN PSM with golden numbers vs DW (1999) Tables 3-4. Modern track: DML PLR + entropy balancing. - Lee classic track: conventional jump 7.414 (paper 7.99) at CCT bandwidth. Modern track: CCT bias-corrected robust 7.507, SE 1.741, h 17.754. - Replicate guide formatter splits regression-test drift tolerance (StatsPAI version-to-version) from paper alignment delta and tightens the Lalonde drift tolerance to $5. - Tests: new TestReplicateLalonde and TestReplicateLee2008 dual-track suites; legacy single-track suite trimmed. 28/28 dual-track tests pass; broader sweep across replicate/external_parity/registry/iv stays green. - All 13 BibTeX keys cited across the four entries verified in paper.bib.
1 parent 7741221 commit 36b99c4

7 files changed

Lines changed: 7298 additions & 247 deletions

File tree

src/statspai/datasets/_canonical.py

Lines changed: 107 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -272,7 +272,65 @@ def card_1995(seed: int = 42, simulated: bool = True) -> pd.DataFrame:
272272
# LaLonde (1986) — NSW experimental
273273
# ---------------------------------------------------------------------------
274274

275-
def nsw_lalonde(seed: int = 42) -> pd.DataFrame:
275+
def nsw_lalonde(seed: int = 42, simulated: bool = True) -> pd.DataFrame:
276+
"""LaLonde NSW data — simulated replica or real MatchIt extract.
277+
278+
Parameters
279+
----------
280+
seed : int, default 42
281+
RNG seed for the simulated replica (ignored when ``simulated=False``).
282+
simulated : bool, default True
283+
If True, return a deterministic simulated NSW experimental
284+
subset (185 + 260 = 445 rows) calibrated so naive OLS
285+
recovers the Dehejia-Wahba experimental ATT of about $1,794.
286+
If False, load the real ``MatchIt::lalonde`` extract bundled
287+
in ``statspai/datasets/data/lalonde_matchit.csv`` — the DW NSW
288+
treated cohort (185) plus a 429-unit PSID-1 subset for
289+
observational comparisons (n=614 total, with race factor
290+
already split into ``black`` and ``hispanic`` indicators).
291+
292+
Notes
293+
-----
294+
The bundled real data is ``MatchIt::lalonde`` (n=614), NOT the
295+
larger DW (1999) NSW + PSID-1 sample (n=2,675). On this smaller
296+
subset, naive OLS gives ATT roughly -$635 (less negative than DW
297+
Table 3's headline -$8,498, which uses the full PSID-1). For
298+
the headline naive-bias demonstration, use the simulated
299+
``nsw_dw()`` panel instead.
300+
301+
Simulated replica calibration
302+
-----------------------------
303+
"""
304+
if not simulated:
305+
df = _load_bundled_csv("lalonde_matchit.csv")
306+
df.attrs['paper'] = (
307+
"Dehejia, R. & Wahba, S. (1999). Causal Effects in "
308+
"Nonexperimental Studies: Reevaluating the Evaluation of "
309+
"Training Programs."
310+
)
311+
df.attrs['data_source'] = 'real'
312+
df.attrs['simulated'] = False
313+
df.attrs['source_origin'] = (
314+
"R MatchIt::lalonde (n=614): 185 NSW treated + 429 PSID-1 "
315+
"controls. race factor split into black + hispanic dummies."
316+
)
317+
# StatsPAI-pinned values on this real extract.
318+
df.attrs['statspai_pinned_naive_ols_att'] = -635.0
319+
df.attrs['statspai_pinned_adj_ols_att'] = 1548.2
320+
df.attrs['statspai_pinned_psm_att'] = 2012.5
321+
df.attrs['published_dehejia_wahba_psm'] = 1794
322+
df.attrs['notes'] = (
323+
"Real MatchIt::lalonde extract (n=614). Naive OLS recovers "
324+
"-$635 because PSID-1 is truncated to 429 controls; "
325+
"covariate-adjusted OLS recovers $1,548 and 1:1 NN PSM "
326+
"recovers ~$2,012, both close to the DW (1999) Table 4 "
327+
"experimental benchmark of $1,794."
328+
)
329+
return df
330+
return _nsw_lalonde_simulated(seed)
331+
332+
333+
def _nsw_lalonde_simulated(seed: int = 42) -> pd.DataFrame:
276334
"""Simulated replica of the NSW experimental subset (185 treated + 260
277335
control).
278336
@@ -451,25 +509,65 @@ def _re78(age, educ, black, hisp, married, re74, re75, treat):
451509
# Lee (2008) — US Senate RD
452510
# ---------------------------------------------------------------------------
453511

454-
def lee_2008_senate(seed: int = 42) -> pd.DataFrame:
455-
"""Simulated replica of Lee (2008) US Senate RD.
512+
def lee_2008_senate(seed: int = 42, simulated: bool = True) -> pd.DataFrame:
513+
"""Lee (2008) US Senate RD — simulated replica or real extract.
456514
457-
Running variable: Democratic vote margin in election t (continuous,
458-
centred at 0). Treatment: Democratic win (margin >= 0). Outcome:
459-
Democratic vote share in election t+1.
515+
Parameters
516+
----------
517+
seed : int, default 42
518+
RNG seed for the simulated DGP (ignored when ``simulated=False``).
519+
simulated : bool, default True
520+
If True, return a deterministic simulated panel (n=6558,
521+
``voteshare_next, margin, win``) on a 0-1 vote-share scale,
522+
calibrated to a 0.08 jump at the cutoff.
523+
If False, load the real ``rdrobust::rdrobust_RDsenate`` extract
524+
(n=1390, ``x, y`` where ``y`` is vote share in **percent
525+
points** 0-100 and ``x`` is the lagged Democratic margin).
460526
461-
Published incumbent-advantage estimate: ≈ 0.08 voteshare points
462-
(Lee 2008, Table 2).
527+
Notes
528+
-----
529+
The real-data branch lets you reproduce Lee (2008) Table 1 /
530+
CCT (2014) Table 4 numbers exactly. StatsPAI's
531+
``sp.rdrobust(df, y='y', x='x', c=0, kernel='triangular',
532+
bwselect='cct')`` recovers Conventional ≈ 7.41 and Robust ≈ 7.51
533+
on this dataset (paper headline ≈ 7.99).
463534
464535
Returns
465536
-------
466-
pd.DataFrame with columns: voteshare_next, margin, win.
537+
pd.DataFrame
538+
Simulated columns: ``voteshare_next, margin, win`` (0-1 scale).
539+
Real columns: ``x, y`` (running variable; vote share 0-100).
467540
468541
References
469542
----------
470543
Lee, D. (2008). Randomized experiments from non-random selection in
471544
U.S. House elections. Journal of Econometrics 142, 675-697. [@lee2008randomized]
545+
Calonico, S., Cattaneo, M.D. & Titiunik, R. (2014). Robust
546+
nonparametric confidence intervals for regression-discontinuity
547+
designs. Econometrica 82(6), 2295-2326. [@calonico2014robust]
472548
"""
549+
if not simulated:
550+
df = _load_bundled_csv("lee_2008_senate.csv")
551+
df.attrs['paper'] = (
552+
"Lee, D. (2008). Randomized experiments from non-random "
553+
"selection in U.S. House elections."
554+
)
555+
df.attrs['data_source'] = 'real'
556+
df.attrs['simulated'] = False
557+
df.attrs['source_origin'] = (
558+
"R rdrobust::rdrobust_RDsenate (n=1390): lagged Democratic "
559+
"vote margin (x) and current Democratic vote share (y, "
560+
"percent points 0-100)."
561+
)
562+
df.attrs['statspai_pinned_conv_estimate_cct_bw'] = 7.414
563+
df.attrs['statspai_pinned_robust_estimate_cct_bw'] = 7.507
564+
df.attrs['published_lee2008_table1'] = 7.99
565+
df.attrs['notes'] = (
566+
"Real Lee Senate RD panel (n=1390). Use kernel='triangular' "
567+
"and bwselect='cct' for R-parity with rdrobust."
568+
)
569+
return df
570+
473571
rng = np.random.default_rng(seed)
474572
n = 6558
475573
margin = rng.normal(0, 0.25, n)

0 commit comments

Comments
 (0)