Skip to content

evidence_ledger.md

Latest commit

 

History

History
92 lines (65 loc) · 4.94 KB

evidence_ledger.md

File metadata and controls

92 lines (65 loc) · 4.94 KB
Raw

Evidence Ledger

Project: Last updated (Beijing): Owner: main orchestrator / evidence critic

Purpose: keep every empirical claim tied to real data, scripts, estimates, exhibits, and manuscript text. A claim is not allowed into the abstract, introduction, results, conclusion, cover letter, or response letter unless its row below points to auditable evidence.

1. Claim Register

Claim ID Claim wording Strength Manuscript location Primary evidence Exhibit Method gate status Robustness status Citation / note
C1 causal / qualified_causal / descriptive / exploratory / no_claim <section/table> 03_analysis/results/ 04_results/ pending / pass / not_pass pending / pass / fail

2. Estimand-to-Claim Map

Purpose: prevent the manuscript from making a broader claim than the design, sample, and diagnostics support.

Claim ID Target estimand Estimation sample estimand External-validity boundary Allowed wording Forbidden wording
C1 ATT / ATE / LATE / local RDD / CATE / descriptive contrast <sample/time/treatment boundary>

Hard rules:

  • If cleaning or attrition changes the target estimand, rewrite the claim before drafting.
  • If the Method Gate allows only descriptive, do not use "effect", "impact", "causes", or "leads to" for that claim.
  • If the evidence is local, state the local population, cutoff, complier group, treated unit, event window, or subgroup.
  • If a robustness result fails, update the allowed wording before changing manuscript prose.

3. Claim Strength Ladder

Use the strongest level allowed by 03_analysis/method_gate.md and references/design-gate-cards.md; never stronger.

  • causal: allowed only when 03_analysis/method_gate.md is PASS for the relevant design and the row has a passed robustness status.
  • qualified_causal: allowed when the method gate passes but diagnostics require an explicit boundary or sensitivity caveat.
  • descriptive: allowed when the estimate is real and reproducible but the identification evidence does not support causal wording.
  • exploratory: allowed only when clearly labeled as hypothesis-generating or appendix evidence.
  • no_claim: required when an artifact is missing, contradicted, or blocked by sample/governance issues.

4. Data and Sample Provenance

Primary sample audit: 02_data/sample_audit.md Primary design risk ledger: 03_analysis/design_risk_ledger.md

Sample step Input Output N / units Exclusion or transform Script and line Notes
Raw import 02_data/raw/ 02_data/ 02_data/<script>:
Analysis file 02_data/ 02_data/clean.parquet 02_data/<script>:

Required checks:

  • Merge keys are unique at the declared unit and frequency.
  • Sample restrictions and attrition are recorded before estimation.
  • Treatment, control, and outcome construction match 02_data/codebook.md.
  • Final estimation sample N, treated/control counts, cluster level, and weights match 03_analysis/results/main_results.json.
  • Restricted data, PII, and archive boundaries match 00_meta/data_governance.md.

5. Variable and Construct Crosswalk

Construct Variable Role Source Construction Timing Codebook path / section Bad-control status
outcome / treatment / control / mechanism pre / post / time-varying 02_data/codebook.md#
 ok / mediator / collider / review

6. Estimation Audit

Result ID Claim ID Design Backend Estimator Script / MCP handle Effect SE / CI N FE / controls Cluster / weights Seed / version
R1 C1 python-statspai / stata / r 03_analysis/<script>: or result_id=

7. Robustness and Threat Matrix

Threat Claim ID Required test Artifact path Pass? Claim consequence Does wording change? Next action
<parallel trends / weak IV / continuity / OVB / selection / p-hacking> C1 03_analysis/robustness/ no causal / qualified_causal / descriptive / exploratory / no_claim yes / no

8. Exhibit and Script Map

Exhibit Claim ID Output formats Generated by Source result Manuscript reference Rebuild checked?
Table 1 C1 tex / docx / xlsx 03_analysis/<script>: R1 main.tex Table no
Figure 1 C1 pdf / png 03_analysis/<script>: R1 main.tex Figure no

9. Open Discrepancies

Issue Affected claim / exhibit Blocking? Owner Resolution
C1 yes / no