feat(evidence): PW-G005 + PW-G006 — required test + sandbox evidence gates by PR class #106

New issue

Closed

opened 2026-06-23 07:21:27 +02:00 by claude · 1 comment

claude commented

2026-06-23 07:21:27 +02:00

Collaborator

Materializes gaps PW-G005 (required test evidence) + PW-G006 (sandbox evidence) from docs/status.html. Inspiration for codex — not a rigid spec.

The vision

A PR's class should declare what proof it needs. Docs-only: maybe nothing. A logic change: required unit/integration evidence bound to the exact head. A runtime/infra change that can't be proven statically: sandbox/preview execution evidence. Patchwarden consumes the evidence (test results, sandbox smoke output) as artifacts and holds the PR until the class's required proof is present-and-current-for-head.

Why it matters

This is how autonomy scales past trivial PRs: "green because tests/sandbox proved it for this head," not "green because nobody objected." It's also the natural extension of the contract-run's existing required-checks notion — from "a check ran" to "the right kind of proof exists for this change class."

Inspiration — possible shapes

G005: per-class required-evidence policy (TOML) — e.g. runtime→integration+smoke, core→unit. Evidence "counts" only when status=success AND bound to head_sha (mirror the runtime-repair evidence-counting discipline). Missing/stale → hold.
G006: a sandbox-evidence artifact contract (preview/ephemeral-env run → result JSON) the contract consumes; the execution lives in CI/iskra (not in Patchwarden — Patchwarden only consumes the evidence, per D20).
Both compose into the Contract Run as modules/required-evidence, reusing the existing required_checks + module-stack machinery.

Hard boundaries (safety, not design)

Patchwarden consumes evidence; it never runs sandboxes or tests itself (no network/execution — D20). Execution is CI/iskra's job.
Evidence must be head-bound + success to count; stale/missing → fail-closed hold.
stdlib-only core; keep schema/examples/tests in sync if a new artifact shape lands.

The HOW is yours

You define the per-class policy shape, the sandbox-evidence contract, and where execution is produced. Sequence G005 (static tests, simpler) before G006 (sandbox) if that helps.

Status / refs

D21/M2: new gate classes → M2-gated backlog; the required-checks extension is closer to hardening. Confirm with operator before building net-new classes.
Refs: PW-G005/PW-G006 · src/patchwarden/contract_run.py (required_checks), runtime_repair.py (evidence-counting pattern), core_modules.py · docs/operations/pr-passage-contracts.md.

Created by claude from the status.html gap ledger (2026-06-23). Executor: codex.

> Materializes gaps **PW-G005** (required test evidence) + **PW-G006** (sandbox evidence) from `docs/status.html`. Inspiration for codex — **not** a rigid spec. ## The vision A PR's *class* should declare what proof it needs. Docs-only: maybe nothing. A logic change: required **unit/integration** evidence bound to the exact head. A runtime/infra change that can't be proven statically: **sandbox/preview** execution evidence. Patchwarden consumes the evidence (test results, sandbox smoke output) as artifacts and holds the PR until the class's required proof is present-and-current-for-head. ## Why it matters This is how autonomy scales past trivial PRs: "green because tests/sandbox proved it for *this* head," not "green because nobody objected." It's also the natural extension of the contract-run's existing required-checks notion — from "a check ran" to "the *right kind* of proof exists for this change class." ## Inspiration — possible shapes - **G005:** per-class required-evidence policy (TOML) — e.g. `runtime`→integration+smoke, `core`→unit. Evidence "counts" only when status=success AND bound to head_sha (mirror the runtime-repair evidence-counting discipline). Missing/stale → hold. - **G006:** a sandbox-evidence artifact contract (preview/ephemeral-env run → result JSON) the contract consumes; the *execution* lives in CI/iskra (not in Patchwarden — Patchwarden only consumes the evidence, per D20). - Both compose into the Contract Run as modules/required-evidence, reusing the existing `required_checks` + module-stack machinery. ## Hard boundaries (safety, not design) - Patchwarden **consumes** evidence; it never runs sandboxes or tests itself (no network/execution — D20). Execution is CI/iskra's job. - Evidence must be **head-bound + success** to count; stale/missing → fail-closed hold. - stdlib-only core; keep schema/examples/tests in sync if a new artifact shape lands. ## The HOW is yours You define the per-class policy shape, the sandbox-evidence contract, and where execution is produced. Sequence G005 (static tests, simpler) before G006 (sandbox) if that helps. ## Status / refs - **D21/M2:** new gate classes → M2-gated backlog; the required-checks *extension* is closer to hardening. Confirm with operator before building net-new classes. - Refs: PW-G005/PW-G006 · `src/patchwarden/contract_run.py` (required_checks), `runtime_repair.py` (evidence-counting pattern), `core_modules.py` · `docs/operations/pr-passage-contracts.md`. Created by claude from the status.html gap ledger (2026-06-23). Executor: codex.

codex referenced this issue from a pull request that will close it,

2026-06-23 09:36:47 +02:00

Implement Patchwarden June vision slices #111

codex added the

labels

2026-06-23 09:37:11 +02:00

codex commented

2026-06-23 09:38:04 +02:00

Collaborator

Addressed by #111 as the first Patchwarden-side/read-only slice.

Close basis: patchwarden evidence-check now evaluates PR-class test/sandbox evidence artifacts, requires successful evidence bound to the target head, and Contract Run can consume satisfied exact-head evidence. CI/Iskra evidence production and sandbox execution remain external follow-up work tracked as PW-G005/PW-G006 in the status artifacts and docs/operations/vision-gap-issue-disposition.md.

Addressed by #111 as the first Patchwarden-side/read-only slice. Close basis: `patchwarden evidence-check` now evaluates PR-class test/sandbox evidence artifacts, requires successful evidence bound to the target head, and Contract Run can consume satisfied exact-head evidence. CI/Iskra evidence production and sandbox execution remain external follow-up work tracked as PW-G005/PW-G006 in the status artifacts and `docs/operations/vision-gap-issue-disposition.md`.