You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
collectPublishArtifacts (packages/cli/src/evidence/publish.ts) reads route
file artifacts from agent-reviews/quality-gates/ui/<sha>/, a path keyed by <sha> only — not by routeId or evidenceKind. With a single
artifact-bearing route (the reality today) this is correct. With more than one
artifact-bearing route armed for the same SHA, every such route reads the same
directory, so the same files are cross-attributed to each route in the PublishedEvidence manifest.
This was latent while the upload gate was hard-coded to evidenceKind === "playwright" (effectively one route). #258 broadened the gate to a membership
test over all non-none artifact-bearing kinds (playwright, migration, terraform, test, docker), which widens the surface where a multi-route change
could cross-attribute.
Impact
A multi-route verification run can publish screenshots / test-results / docker /
plan artifacts under the wrong route, making the manifest misstate which proof
supports which objective — affecting downstream worker joins and dashboard proof
review. (Flagged [high]/contracts by the local codex chief-engineer critic on the #258 PR.)
Required fix
Make artifact discovery route-scoped before broadening further:
Partition producer output by routeId (and/or evidenceKind), e.g. write to agent-reviews/quality-gates/ui/<sha>/<routeId>/..., and have readEvidenceArtifacts read the per-route subtree (with a back-compat fallback
to the flat ui/<sha>/ path for the existing playwright producer).
Add a regression test with two artifact-bearing routes in one gate JSON
proving their artifacts do not cross-attach.
Context
Deferred from #258 (the brief scoped this caveat as a known follow-up). The #258
change documents the limitation inline in readEvidenceArtifacts' doc comment.
Problem
collectPublishArtifacts(packages/cli/src/evidence/publish.ts) reads routefile artifacts from
agent-reviews/quality-gates/ui/<sha>/, a path keyed by<sha>only — not byrouteIdorevidenceKind. With a singleartifact-bearing route (the reality today) this is correct. With more than one
artifact-bearing route armed for the same SHA, every such route reads the same
directory, so the same files are cross-attributed to each route in the
PublishedEvidencemanifest.This was latent while the upload gate was hard-coded to
evidenceKind === "playwright"(effectively one route). #258 broadened the gate to a membershiptest over all non-
noneartifact-bearing kinds (playwright, migration, terraform, test, docker), which widens the surface where a multi-route changecould cross-attribute.
Impact
A multi-route verification run can publish screenshots / test-results / docker /
plan artifacts under the wrong route, making the manifest misstate which proof
supports which objective — affecting downstream worker joins and dashboard proof
review. (Flagged [high]/contracts by the local codex chief-engineer critic on the
#258 PR.)
Required fix
Make artifact discovery route-scoped before broadening further:
routeId(and/orevidenceKind), e.g. write toagent-reviews/quality-gates/ui/<sha>/<routeId>/..., and havereadEvidenceArtifactsread the per-route subtree (with a back-compat fallbackto the flat
ui/<sha>/path for the existing playwright producer).skills/verify/producer/*), not justthe publish reader — which is why it was deliberately scoped out of Generalize evidenceKind beyond 'playwright' for artifact upload (publish.js:120) #258.
proving their artifacts do not cross-attach.
Context
Deferred from #258 (the brief scoped this caveat as a known follow-up). The #258
change documents the limitation inline in
readEvidenceArtifacts' doc comment.