docs(jss): refresh reviewer-facing counts + validation framing

Updates README/README_CN/index/dossier registry counts (1,020 functions,
81 submodules, 51 R-aligned Track A modules, 244 certified+validated
symbols) and reframes the coverage grid as method-family breadth rather
than a per-option validation certificate, pointing readers to
validation_status. CHANGELOG records the rddensity escape hatch, the
causal-forest AIPW correctness fix, the SCM-solver certification, and the
regenerated Track A ledger under [Unreleased].

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: brycewang-stanford

CHANGELOG.md

@@ -4,6 +4,56 @@ All notable changes to StatsPAI will be documented in this file.
 
 ## [Unreleased]
 
+### Added — RD density parity escape hatch
+
+- **`sp.rddensity` now accepts side-specific manual bandwidths and an optional
+  R reference backend.** The native dependency-light implementation remains
+  the default, but `h=(h_left, h_right)` now records matched left/right
+  bandwidths explicitly, and `backend="r"` delegates to
+  `rddensity::rddensity` through `Rscript` when R plus the `rddensity` and
+  `jsonlite` packages are installed. This does not pretend the native
+  bandwidth selector is identical to the R package; it gives users and
+  reviewers a clear path to canonical selector/test-statistic parity when
+  that exact comparison matters. Guarded by `tests/test_rddensity_io.py`;
+  the R backend test is skipped automatically when R is unavailable.
+
+### ⚠️ Correctness fix — causal-forest ATE/ATT now doubly-robust (AIPW)
+
+- **`CausalForest.average_treatment_effect` (and the `target_sample`
+  `"all"`/`"treated"`/`"control"`/`"overlap"` aggregations) now returns
+  the doubly-robust AIPW influence-function mean** — the estimand
+  `grf::average_treatment_effect` reports — instead of a plug-in average
+  of the regularisation-shrunk CATE predictions. The plug-in average
+  overshoots the true effect (≈ 15 % on a clean-overlap DGP) and
+  disagreed with `grf` by an order of magnitude on overlap-pathological
+  samples. The AIPW score reuses the forest's own cross-fitted nuisances
+  `m̂(X)=Ê[Y|X]`, `ê(X)=Ê[T|X]` and the CATE `τ̂(X)`:
+  `Γ_i = τ̂ + (T−ê)/(ê(1−ê))·(Y − m̂ − (T−ê)τ̂)`. **Numbers change** for
+  any code reading `average_treatment_effect(...)['estimate']`; the SE is
+  now the influence-function SE `sd(Γ)/√n`. The plug-in `cf.ate()` /
+  `cf.att()` convenience methods are unchanged and retained, but are no
+  longer the validation estimand. Users who reported a causal-forest ATE
+  from `average_treatment_effect` should re-run.
+  - On a clean-overlap DGP (`e(X)∈[0.30,0.70]`, known ATE = 1) the AIPW
+    ATE recovers the truth within 1.5 SE and agrees with `grf` at
+    `rel = 0.037` (`z = 0.69` combined SE); the R-parity module
+    `13_causal_forest` verdict moves from GAP (500 % band) to PASS
+    (10 % band).
+  - Guards: `tests/reference_parity/test_causal_forest_aipw_recovery.py`
+    (recovery against truth, no R needed) and the tightened
+    `tests/reference_parity/test_grf_parity.py` (combined-SE parity vs a
+    committed `grf` fixture).
+- **Synthetic-control solver certified on identified problems.** Added
+  `tests/reference_parity/test_scm_recovery.py` and the cross-language
+  module `tests/r_parity/52_scm_unique`: on a DGP whose synthetic-control
+  weights are uniquely identified (treated unit exactly a convex
+  combination of donors in the pre-period), `sp.synth(method="classic")`
+  recovers the exact weights and gap (pre-RMSE = 0) and agrees with
+  `Synth::synth` to 0.7 %; the classical/augmented/generalised/MC
+  variants each recover a known factor-model ATT within 3 %. No
+  estimator numerics change — this is added validation isolating the
+  Basque-data SCM gap (module `07`) as genuine weight non-uniqueness.
+
 ### Added — MCP protocol modernization
 
 - **Protocol version negotiation + bump to `2025-06-18`
@@ -53,15 +103,15 @@ All notable changes to StatsPAI will be documented in this file.
   than only frozen.
 - **`sp.validation_report(collect_tests=True)`** — shells out to
   `pytest --collect-only` and returns the authoritative, parametrize-expanded
-  parity test counts (114 reference-parity, 50 external-parity, 12 coverage
-  Monte Carlo on the 1.16.0 tree); a regression test pins those three to the
+  parity test counts (124 reference-parity, 50 external-parity, 12 coverage
+  Monte Carlo on the current source snapshot); a regression test pins those three to the
   JSS manuscript headline so a parity test added/removed without updating the
   paper fails CI. Default `validation_report()` path is unchanged (fast,
   metadata-only).
 - **Strictness-tier breakdown in the Track A parity tables
   (`tests/r_parity/compare.py`)** — each module is classified by its
   registered point-estimate tolerance into machine / iterative / moderate /
-  methodological tiers (6 / 26 / 9 / 9 on the 50 R-rendered modules), shown
+  methodological tiers (6 / 26 / 10 / 9 on the 51 R-joined modules), shown
   in the Markdown ledger and the LaTeX appendix caption so a machine-precision
   match is not flattened together with a deliberately loose
   methodological-difference tolerance.
@@ -82,13 +132,15 @@ All notable changes to StatsPAI will be documented in this file.
 
 ### Changed — Track A R golden values regenerated under the locked environment
 
-- The 50 committed `results/*_R.json` were regenerated under R 4.5.2 with
+- The current R parity ledger covers 51 `results/*_R.json` modules under R 4.5.2 with
   the `renv.lock` package set so each is self-describing. The only material
   numerical movement is `07_scm` (classical SCM), whose headline estimate
   shifted by 2.18e-8 — L-BFGS-B / Apple-Accelerate-BLAS non-associativity,
-  far inside that module's 0.20 SCM non-uniqueness parity tolerance. All
-  cross-language verdicts are unchanged (42 PASS / 10 GAP). This is a
-  reference-fixture refresh, not a `statspai` estimator-output change.
+  far inside that module's 0.20 SCM non-uniqueness parity tolerance. The
+  ledger also adds `52_scm_unique`, an identified synthetic-control DGP that
+  separates solver correctness from the Basque non-uniqueness gap. This is
+  primarily a reference-fixture and evidence refresh, not a broad
+  `statspai` estimator-output change.
 - The 44 committed `results/*_Stata.json` were likewise refreshed to embed
   the engine `provenance` block; their numbers are **unchanged** (every
   module reproduces at exactly 0 under the locked Stata 18 MP environment).

README.md

@@ -14,9 +14,9 @@
 [![status](https://joss.theoj.org/papers/9f1c837b1b1df7adfcdd538c3698e332/status.svg)](https://joss.theoj.org/papers/9f1c837b1b1df7adfcdd538c3698e332)
 [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.19933900.svg)](https://doi.org/10.5281/zenodo.19933900)
 
-StatsPAI is the **first agent-native** Python platform for causal inference and applied econometrics. One `import`, **1,000+ registered functions** across **80 submodules** (live count: `python scripts/registry_stats.py`), covering the complete empirical research workflow — from classical econometrics to cutting-edge ML/AI causal methods to publication-ready tables in Word, Excel, and LaTeX.
+StatsPAI is an **agent-native** Python platform for causal inference and applied econometrics. One `import`, **1,000+ registered functions** across **80+ submodules** (live count: `python scripts/registry_stats.py`), covering the empirical research workflow — from classical econometrics to ML/AI causal methods to publication-ready tables in Word, Excel, and LaTeX.
 
-**Built for AI agents**: every function returns structured result objects with machine-readable schemas (`list_functions()`, `describe_function()`, `function_schema()`) and is numerically validated against R and Stata reference implementations — purpose-built for LLM-driven research workflows while remaining fully ergonomic for human researchers.
+**Built for AI agents**: every registered function has machine-readable discovery metadata (`list_functions()`, `describe_function()`, `function_schema()`), and parity-backed functions expose an explicit `validation_status` so agents and humans can distinguish certified numerical evidence from API-stable breadth.
 
 It brings R's [Causal Inference Task View](https://cran.r-project.org/web/views/CausalInference.html) (fixest, did, rdrobust, gsynth, DoubleML, MatchIt, CausalImpact, ...) and Stata's core econometrics commands into a single, consistent Python API.
 
@@ -115,7 +115,7 @@ Every result object exposes the same interface — `.summary()` / `.tidy()` / `.
 
 ## 📊 Causal Inference Coverage at a Glance
 
-StatsPAI's focus is **causal inference** — and on this axis we aim to be the most complete single package in any language. "Stata" = official + major SSC packages. "R" = CRAN. "sm+lm" = statsmodels + linearmodels.
+StatsPAI's focus is **causal inference**. The grid below summarizes method-family breadth; it is not a validation certificate for every option. "Stata" = official + major SSC packages. "R" = CRAN. "sm+lm" = statsmodels + linearmodels.
 
 | Method family                                                 | Stata | R | sm+lm | DoubleML | **StatsPAI** |
 | ------------------------------------------------------------- | :---: | :---: | :---: | :---: | :---: |
@@ -137,7 +137,7 @@ StatsPAI's focus is **causal inference** — and on this axis we aim to be the m
 
 **Legend**: 🏆 most complete across ecosystems · ✅ full coverage · ⚠️ partial / scattered / single algorithm · ❌ not available.
 
-**StatsPAI at a glance**: 1,018 registered functions in the live agent registry · 80 submodules · ~249k LOC (core) + ~86k LOC (tests). All four numbers are reproducible from the canonical generator (`python scripts/registry_stats.py`); the per-module table in [`docs/stats.md`](docs/stats.md) is regenerated from the same script. For the full coverage matrix (23 method families) and cross-ecosystem line-count comparison, see [`docs/stats.md`](docs/stats.md).
+**StatsPAI at a glance**: 1,020 registered functions in the live agent registry · 81 submodules · 266k LOC (core) + 93k LOC (tests). All four numbers are reproducible from the canonical generator (`python scripts/registry_stats.py`); the per-module table in [`docs/stats.md`](docs/stats.md) is regenerated from the same script. For the full coverage matrix (23 method families) and cross-ecosystem line-count comparison, see [`docs/stats.md`](docs/stats.md).
 
 **Validation tiers matter**: `stability="stable"` means the public API is SemVer-stable; it does not by itself mean R/Stata/paper parity. Use `sp.list_functions(validation_status="certified")` for cross-language or published-reference evidence, and inspect `sp.describe_function(name)["limitations"]` before production use. See [`docs/guides/stability.md`](docs/guides/stability.md).
 
@@ -146,8 +146,9 @@ StatsPAI's focus is **causal inference** — and on this axis we aim to be the m
 StatsPAI 1.16.0 corrects the `sp.qreg` Powell sandwich standard error (was off
 by √n) and rebuilds `sp.xtabond` Arellano–Bond difference GMM to match Stata to
 machine precision — both flagged ⚠️ **Correctness**, so re-run affected
-analyses. The Track A cross-language parity harness grows from 36 to 50
-R-aligned modules (Stata reference for 43 of them), and the JOSS
+analyses. The Track A cross-language parity harness grows from 36 to 51
+R-aligned modules (Stata reference for 43 of them, plus one
+Py-Stata-only `xtabond` migration check), and the JSS
 reviewer-facing docs are refreshed. Full notes in
 [`CHANGELOG.md`](CHANGELOG.md) under `[1.16.0]`.
 
@@ -428,7 +429,7 @@ StatsPAI 1.4.0 is Sprint 2 of the 知识地图 v3 roadmap. Closes the four secon
 | **Particle-filter assimilation** | **`sp.assimilation.particle_filter`** — bootstrap-SIR particle filter with systematic resampling (Gordon-Salmond-Smith 1993; Douc-Cappé 2005). Non-Gaussian priors, heavy-tailed observation noise, nonlinear dynamics via pluggable callbacks. Agrees with exact Kalman to ~0.003 under Gaussian DGPs. **`sp.assimilative_causal(..., backend='particle')`** routes the end-to-end wrapper. |
 | **Documentation (v3 frontier guides)** | `docs/guides/synth_experimental.md` (Abadie-Zhao inverse-SC workflow), `docs/guides/harvest_did.md` (Borusyak-Hull-Jaravel harvesting DID), `docs/guides/assimilative_ci.md` (Nature Comms 2026 streaming CI, Kalman + particle backends). Wired into `mkdocs.yml` nav. |
 | **v1.3 stable foundation (carried forward)** | 11 2025-2026 frontier methods from Sprint 1: `synth_experimental_design`, `rdrobust(..., bootstrap='rbc')`, `evidence_without_injustice`, `target_trial.to_paper(fmt='jama'/'bmj')`, `harvest_did`, `bcf_ordinal`, `bcf_factor_exposure`, `causal_mas`, `shift_share_political`, `causal_kalman`. All v1.0 capstone surfaces (`sp.bridge`, `sp.fairness`, `sp.surrogate`, `sp.epi`, `sp.longitudinal`, `sp.question`, full MR suite, TARGET checklist) remain intact. |
-| **Agent-native platform** | `sp.list_functions()` / `sp.describe_function()` / `sp.function_schema()` expose OpenAI/Anthropic tool-calling schemas for 1,018 registered public functions. 145 curated or explicitly inherited `FunctionSpec` entries carry at least one of assumptions, preconditions, failure modes, limitations, `typical_n_min`, and validation tiers for the flagship surface. `sp.agent.mcp_server` MCP scaffold lets external LLMs call every StatsPAI function via natural-language tool invocation. |
+| **Agent-native platform** | `sp.list_functions()` / `sp.describe_function()` / `sp.function_schema()` expose OpenAI/Anthropic tool-calling schemas for 1,020 registered public functions. 362 curated or explicitly inherited `FunctionSpec` entries carry at least one of assumptions, preconditions, failure modes, limitations, `typical_n_min`, and validation tiers for the flagship surface. `validation_status` distinguishes certified/validated evidence from API-stable breadth. `sp.agent.mcp_server` MCP scaffold lets external LLMs call every StatsPAI function via natural-language tool invocation. |
 | **CI/CD hygiene** | `tabulate` hard-dep from v1.3.0 carried forward. Deflaked `test_forest_ate_recovers_average_tau` by seeding the forest explicitly (`random_state=0`, `n_estimators=300`, larger `n`). 2 699+ tests passing across all OS × Python matrix entries. |
 
 **Previously in v0.9.2 — Decomposition Analysis**: **18 first-class decomposition methods across 13 modules (~6,200 LOC, 54 tests)**, unified under `sp.decompose(method=...)`. Mean (Blinder-Oaxaca/Gelbach/Fairlie/Bauer-Sinning/Yun), distributional (RIF/FFL/DFL/Machado-Mata/Melly/CFM), inequality (Theil/Atkinson/Dagum/Shapley/Lerman-Yitzhaki), demographic (Kitagawa/Das-Gupta), and causal (gap_closing/mediation_decompose/disparity_decompose). Closed-form influence functions for Theil/Atkinson, weighted O(n log n) Dagum Gini, cross-method consistency checks.
@@ -653,7 +654,7 @@ wrappers, no runtime dependencies on upstream DID packages.
 | --- | --- | --- |
 | `rdrobust()` | Sharp/Fuzzy RD with robust bias-corrected inference | Calonico, Cattaneo & Titiunik (2014) |
 | `rdplot()` | RD visualization with binned scatter | — |
-| `rddensity()` | McCrary density manipulation test | McCrary (2008) |
+| `rddensity()` | CJM density manipulation test; optional `backend="r"` bridge to `rddensity::rddensity` | Cattaneo, Jansson & Ma (2020) |
 | `rdmc()` | Multi-cutoff RD | Cattaneo et al. (2024) |
 | `rdms()` | Geographic / multi-score RD | Keele & Titiunik (2015) |
 | `rkd()` | Regression Kink Design | Card et al. (2015) |

README_CN.md

@@ -46,14 +46,15 @@ StatsPAI 聚焦**因果推断**——在这条主线上，我们的目标是成
 
 **图例**：🏆 跨生态最完整 · ✅ 完整覆盖 · ⚠️ 部分 / 分散 / 单算法 · ❌ 无。
 
-**StatsPAI 一句话概览**：live agent registry 中有 1,018 个注册函数 · 80 个子模块 · ~249k 行核心代码 + ~86k 行测试。这四个数字都可以由唯一的生成器 (`python scripts/registry_stats.py`) 现场复算；[`docs/stats.md`](docs/stats.md) 中的按模块拆分表也由同一个脚本回写。完整覆盖矩阵（23 个方法家族）以及跨生态行数对比，详见 [`docs/stats.md`](docs/stats.md)。
+**StatsPAI 一句话概览**：live agent registry 中有 1,020 个注册函数 · 81 个子模块 · 266k 行核心代码 + 93k 行测试。这四个数字都可以由唯一的生成器 (`python scripts/registry_stats.py`) 现场复算；[`docs/stats.md`](docs/stats.md) 中的按模块拆分表也由同一个脚本回写。完整覆盖矩阵（23 个方法家族）以及跨生态行数对比，详见 [`docs/stats.md`](docs/stats.md)。这些覆盖数字描述 API 广度，不等同于每个函数都有 R/Stata 数值验证；生产使用请查看 `validation_status`。
 
 **📦 v1.16.0（2026-05-29）— 正确性修复与跨语言对齐扩展**
 
 StatsPAI 1.16.0 修正了 `sp.qreg` Powell 三明治标准误（此前差一个 √n 因子），
 并重写 `sp.xtabond` Arellano–Bond 差分 GMM 使其与 Stata 机器精度一致——
 两者均标 ⚠️ **正确性**，相关分析请重跑。Track A 跨语言对齐夹具从 36 个
-扩展到 50 个 R 对齐模块（其中 43 个有 Stata 参考），并刷新了 JOSS
+扩展到 51 个 R 对齐模块（其中 43 个有 Stata 参考，另有 1 个
+Py-Stata-only 的 `xtabond` 迁移检查），并刷新了 JSS
 reviewer 相关文档。完整发布说明见 [`CHANGELOG.md`](CHANGELOG.md)
 `[1.16.0]`。
 
@@ -240,7 +241,7 @@ StatsPAI 1.4.0 是知识地图 v3 路线图的 Sprint 2，关闭了 Sprint 1 末
 | **粒子滤波同化** | **`sp.assimilation.particle_filter`** — 带系统重采样的 bootstrap-SIR 粒子滤波（Gordon-Salmond-Smith 1993；Douc-Cappé 2005）。非高斯先验、重尾观测噪声、非线性动力学通过可插拔回调实现。高斯 DGP 下与精确 Kalman 一致到 ~0.003。**`sp.assimilative_causal(..., backend='particle')`** 端到端 wrapper 路由到粒子滤波。 |
 | **文档（v3 前沿指南）** | `docs/guides/synth_experimental.md`（Abadie-Zhao 反向 SC 流程）、`docs/guides/harvest_did.md`（Borusyak-Hull-Jaravel harvest DID）、`docs/guides/assimilative_ci.md`（Nature Comms 2026 流式 CI，Kalman + 粒子后端）。已挂到 `mkdocs.yml` 导航。 |
 | **v1.3 稳定基础（延续）** | Sprint 1 的 11 个 2025-2026 前沿方法：`synth_experimental_design`、`rdrobust(..., bootstrap='rbc')`、`evidence_without_injustice`、`target_trial.to_paper(fmt='jama'/'bmj')`、`harvest_did`、`bcf_ordinal`、`bcf_factor_exposure`、`causal_mas`、`shift_share_political`、`causal_kalman`。所有 v1.0 capstone 面（`sp.bridge`、`sp.fairness`、`sp.surrogate`、`sp.epi`、`sp.longitudinal`、`sp.question`、MR 全家桶、TARGET 清单）保持不变。 |
-| **Agent 平台** | `sp.list_functions()` / `sp.describe_function()` / `sp.function_schema()` 为 1,018 个注册公共函数提供 OpenAI/Anthropic tool-calling schema。145 个已策划或显式继承的 `FunctionSpec` 条目携带假设、前置条件、失败模式、局限、`typical_n_min` 或 validation tier。`sp.agent.mcp_server` MCP 脚手架让外部 LLM 可自然语言调用每个函数。 |
+| **Agent 平台** | `sp.list_functions()` / `sp.describe_function()` / `sp.function_schema()` 为 1,020 个注册公共函数提供 OpenAI/Anthropic tool-calling schema。362 个已策划或显式继承的 `FunctionSpec` 条目携带假设、前置条件、失败模式、局限、`typical_n_min` 或 validation tier。`validation_status` 明确区分 certified / validated / api_stable，避免把 schema 覆盖误读成数值验证。`sp.agent.mcp_server` MCP 脚手架让外部 LLM 可自然语言调用每个函数。 |
 | **CI/CD 卫生** | v1.3.0 的 `tabulate` 硬依赖延续。通过显式播种（`random_state=0`、`n_estimators=300`、扩 `n`）修复 `test_forest_ate_recovers_average_tau` flake。2 699+ 测试在所有 OS × Python matrix 上通过。 |
 
 **v0.6 新功能**：`sp.interactive(fig)` —— 类似 Stata Graph Editor 的 WYSIWYG 图表编辑器，支持 29 种学术主题、实时预览、自动生成可复现代码。
@@ -339,7 +340,7 @@ StatsPAI **不是** R 的 wrapper。我们从原始论文独立重新实现每
 | --- | --- | --- |
 | `rdrobust()` | 尖锐/模糊 RD，稳健偏差校正推断 | Calonico, Cattaneo & Titiunik (2014) |
 | `rdplot()` | RD 可视化（分箱散点图） | — |
-| `rddensity()` | McCrary 密度操纵检验 | McCrary (2008) |
+| `rddensity()` | CJM 密度操纵检验；可选 `backend="r"` 桥接 `rddensity::rddensity` | Cattaneo, Jansson & Ma (2020) |
 
 ### 因果推断 — 匹配与再加权