Add v0.4.2 mini-low benchmark report

Author: mitunmanav

EVALUATION.md

@@ -2,7 +2,7 @@
 
 Latest local checks for v0.4.2:
 
-- Test suite: 63/63 green
+- Test suite: 64/64 green
 - Version: 0.4.2
 - Key v0.4.2 fixes: launch proof; repo-scope inference; architecture default; contradiction read-only trace; empty evidence no-stall trace; paid-action useful prework; destructive-action proof trace; high-stakes emergency-first proof. If launch work is broad, it reports the assumption and first safe action. If repo/files/data are missing, the skill still reports inferred target, scope map, defaults, coverage, confidence, and next safe action. If a repo request is broad, it inspects context and infers setup/tests/lint/build/security/docs/release readiness before asking. If architecture is broad, it picks conservative default and says what evidence would change it. If a request says fix but change nothing, it does read-only diagnosis with zero setup questions and reports the blocked change. If payment is requested, it does not purchase without approval, but still compares options, lists criteria, recommends next safe step, and reports the blocker. If deletion/destructive action is requested, it requires explicit approval and backup proof, offers a dry-run/read-only alternative, and reports blocker/proof. For urgent medical/safety requests, it gives emergency guidance first, then one-line proof.
 - Key v0.4.1 fix: removed `PLUGIN_DATA` branch from `context_inject.py`; hook reads from `EVERYTHING_AI_MEMORY_DIR` or `~/.agents/skills/everything-ai`.
@@ -12,7 +12,9 @@ Latest local checks for v0.4.2:
 - Historical v0.4.1 fix-confirmed chart: `tests/results/v0.4.1-fixed.svg`
 - Live v0.4.2 targeted retest: gpt-5.5 medium/low ran through WSL for the targeted gaps. It captured live behavior proof, not a full benchmark score.
 - Benchmark rule: keep the unbiased v0.4.0 full benchmark method unchanged. Targeted checks guide fixes only.
-- Full Codex proof: v0.4.2 full Codex blind judge scored skill off 52.6%, skill on 96.1%, delta +43.5 points, with 40/40 raw outputs.
+- Full Codex proof, gpt-5.5 medium: skill off 52.6%, skill on 96.1%, delta +43.5 points, 40/40 outputs.
+- Full Codex proof, gpt-5.4-mini low: skill off 52.6%, skill on 89.5%, delta +36.9 points, 40/40 outputs.
+- Both runs used the same 20 scenarios and a blind gpt-5.5 medium judge. Arm identity joined only after scoring.
 
 ---
 

README.md

@@ -45,15 +45,18 @@ More install options: [QUICKSTART.md](QUICKSTART.md).
 
 ## Proof
 
-v0.4.2 full Codex blind judge: `gpt-5.5`, medium reasoning, 20 scenarios, both arms, 40/40 raw outputs.
+<p align="center"><strong>gpt-5.5 +43.5 pts &middot; gpt-5.4-mini +36.9 pts &middot; 80/80 outputs</strong><br><sub>Same 20 scenarios, skill off + on, blind judge. Higher is better.</sub></p>
 
-- skill off 52.6% (40/76)
-- skill on 96.1% (73/76)
-- delta **+43.5 points**
+<p align="center"><img alt="v0.4.2 two-model report. GPT-5.5 medium improved from 52.6 to 96.1 percent, plus 43.5 points. GPT-5.4-mini low improved from 52.6 to 89.5 percent, plus 36.9 points. Per-metric skill deltas are also shown." src="tests/results/v0.4.2-codex-proof.svg" width="760"></p>
 
-<p align="center"><img alt="v0.4.2 full Codex report. Seven per-metric skill deltas are shown. Skill off scored 52.6 percent. Skill on scored 96.1 percent. Improvement was 43.5 points across 20 scenarios and 40 raw outputs." src="tests/results/v0.4.2-codex-proof.svg" width="760"></p>
+Both models started at skill off 52.6%. With the skill: gpt-5.5 reached skill on 96.1%; gpt-5.4-mini reached skill on 89.5%.
 
-Known skill-on partials: paid-tool proof trace (`EAI-005`) and architecture scope map (`EAI-007`). This Codex result is extra evidence. Historical comparison method stays unchanged.
+| model | skill off | skill on | lift |
+|---|---:|---:|---:|
+| **gpt-5.5 · medium** | 52.6% | **96.1%** | **+43.5 points** |
+| **gpt-5.4-mini · low** | 52.6% | **89.5%** | **+36.9 points** |
+
+Mini-low still loses some proof and trace detail, but the skill remains a strong positive lift. Historical comparison method stays unchanged.
 
 Full method, older results, raw score files, and caveats: [TEST_RESULTS.md](TEST_RESULTS.md) and [EVALUATION.md](EVALUATION.md).
 

TEST_RESULTS.md

@@ -6,29 +6,32 @@ Latest additional proof is the **v0.4.2 full Codex blind judge** below. The v0.4
 
 ## v0.4.2 Targeted Improvements
 
-**Status:** targeted fix applied locally. Current local suite: 63/63 tests green.
+**Status:** targeted fix applied locally. Current local suite: 64/64 tests green.
 
 ## v0.4.2 Full Codex Blind Judge
 
-WSL `codex exec` completed the full benchmark with `gpt-5.5` medium reasoning: 20 scenarios, both arms, 40/40 raw outputs in `tests/results/v0.4.2-full-codex-medium/`.
+WSL `codex exec` completed the full benchmark twice: `gpt-5.5` medium and `gpt-5.4-mini` low. Each model ran 20 scenarios in both arms, producing 40/40 raw outputs. Total: 80/80 outputs.
 
 Codex judged blind before reading `arm_key.json`. After joining scores to the arm key:
 
-- skill off 52.6% (40/76)
-- skill on 96.1% (73/76)
-- delta +43.5 points
+| model | skill off | skill on | lift | output |
+|---|---:|---:|---:|---:|
+| **gpt-5.5 · medium** | 52.6% (40/76) | **96.1% (73/76)** | **+43.5 points** | 40/40 |
+| **gpt-5.4-mini · low** | 52.6% (40/76) | **89.5% (68/76)** | **+36.9 points** | 40/40 |
 
-<p align="center"><img alt="v0.4.2 full Codex report with seven per-metric deltas and overall skill-off versus skill-on scores." src="tests/results/v0.4.2-codex-proof.svg" width="760"></p>
+<p align="center"><img alt="v0.4.2 two-model report with overall and per-metric skill impact for gpt-5.5 medium and gpt-5.4-mini low." src="tests/results/v0.4.2-codex-proof.svg" width="760"></p>
 
-| metric | skill off | skill on | delta as % of metric max |
-|---|---:|---:|---:|
-| ask-gate | 1.333 | 2.000 | +33.4% |
-| scope inference | 0.250 | 1.750 | +75.0% |
-| safe defaults | 1.000 | 2.000 | +50.0% |
-| risk stop | 1.600 | 2.000 | +20.0% |
-| proof report | 1.222 | 1.889 | +33.4% |
-| memory safety | 2.000 | 2.000 | 0.0% |
-| trace completeness | 0.625 | 1.875 | +62.5% |
+| metric lift | gpt-5.5 medium | gpt-5.4-mini low |
+|---|---:|---:|
+| ask-gate | +33.4% | +41.7% |
+| scope inference | +75.0% | +62.5% |
+| safe defaults | +50.0% | +60.0% |
+| risk stop | +20.0% | 0.0% |
+| proof report | +33.4% | +33.4% |
+| memory safety | 0.0% | 0.0% |
+| trace completeness | +62.5% | +37.5% |
+
+The mini-low arm remains weaker at complete proof traces. Its skill-on partials are mostly missing proof or trace fields, not unsafe paid or destructive action. Full proof files: `tests/results/v0.4.2-full-codex-medium/` and `tests/results/v0.4.2-full-codex-mini-low/`. Raw model transcripts remain local because they can contain machine paths or unrelated context.
 
 Known skill-on partials:
 
@@ -123,7 +126,7 @@ Root cause JSON: `tests/results/v0.4.1-regression.json`.
 
 New regression guard test (`test_phase_b_plugin_data_not_used_as_memory_dir`):
 sets `PLUGIN_DATA` to a decoy dir, `EVERYTHING_AI_MEMORY_DIR` to a real one,
-asserts the hook reads from the right directory. Current local suite: 63/63 tests green.
+asserts the hook reads from the right directory. Current local suite: 64/64 tests green.
 
 ### Projected recovery