test(v0): jsonschema validation for CLI artifact outputs (closes #62) #71

Merged

pdurlej merged 1 commit from gemini/jsonschema-artifact-validation into main

2026-06-08 18:15:05 +02:00

claude commented

2026-06-08 18:06:49 +02:00

Collaborator

Authored by gemini (Gemini 3.1 Pro via Antigravity), implemented autonomously under claude's supervision. claude reviewed the diff, verified tests independently, fixed the commit identity (worktree shared-config footgun), and is opening this PR. This is the first cousin contribution under the gemini identity. 🎉

What

Implements #62 — wires jsonschema.validate() against real CLI artifact outputs so the spec/schemas/ contracts are actually enforced, not just documentation.

Changes (4 files, +193, tests/packaging only — zero `src/` logic touched)

pyproject.toml: jsonschema>=4.0 added to test + dev extras only.
tests/test_artifact_schema_contract.py (new): validates evaluate_pull_request, evaluate_issue, run_review (empty + findings + soft-fail), and registry outputs against their schemas; each schema gets a happy-path validate + a deliberately-invalid case proving the validator runs.
tests/test_boring_pr_lifecycle.py: jsonschema.validate() after each emit in all 5 scenarios.
tests/test_cli_cloud_review_mode.py (new): _cloud_review_mode_from_lane() — allowed→cloud, required-with-redaction→redacted, else→local.

Verified (claude, independently)

PYTHONPATH=src python3 -m unittest discover tests → 168/168 OK (160 baseline + 8 new).
grep -rn "import jsonschema" src/patchwarden/ → zero hits (stdlib-only preserved; jsonschema is test-only).
No schema files modified; no schema_version bumped.

⚠️ Findings surfaced (the whole point of #62 — schemas drifted from code)

The new validation immediately caught two real contract drifts. Per #62's rule, gemini did NOT loosen the schemas; it documented the gaps:

review-artifact.schema.json rejects the soft-fail runtime_metadata fields (error_kind, error_detail, fell_back, model_used) that PR #54 added to review_run output. The schema predates #54 and is now stricter than the code. The soft-fail test asserts the current (broken) rejection so the suite is honest about the drift.
finding-resolution.schema.json does not exist — resolve_findings() emits patchwarden.finding_resolution.v1 but there is no schema file for it. The fourth schema slot was covered with the registry schema instead.

→ Tracked for fix in a follow-up issue (see linked). Once fixed, the soft-fail assertRaises flips to a positive validate().

Cousin-discipline notes

unittest, not pytest (pytest stays opt-in). ✅
Commit authored + committed by gemini (fixed after a worktree shared-.git/config identity clobber during parallel cousin work — lesson logged).

Closes #62.

> **Authored by gemini** (Gemini 3.1 Pro via Antigravity), implemented autonomously under claude's supervision. claude reviewed the diff, verified tests independently, fixed the commit identity (worktree shared-config footgun), and is opening this PR. This is the first cousin contribution under the `gemini` identity. 🎉 ## What Implements #62 — wires `jsonschema.validate()` against real CLI artifact outputs so the `spec/schemas/` contracts are actually enforced, not just documentation. ## Changes (4 files, +193, tests/packaging only — zero `src/` logic touched) - `pyproject.toml`: `jsonschema>=4.0` added to **test + dev extras only**. - `tests/test_artifact_schema_contract.py` (new): validates `evaluate_pull_request`, `evaluate_issue`, `run_review` (empty + findings + soft-fail), and registry outputs against their schemas; each schema gets a happy-path validate + a deliberately-invalid case proving the validator runs. - `tests/test_boring_pr_lifecycle.py`: `jsonschema.validate()` after each emit in all 5 scenarios. - `tests/test_cli_cloud_review_mode.py` (new): `_cloud_review_mode_from_lane()` — allowed→cloud, required-with-redaction→redacted, else→local. ## Verified (claude, independently) - `PYTHONPATH=src python3 -m unittest discover tests` → **168/168 OK** (160 baseline + 8 new). - `grep -rn "import jsonschema" src/patchwarden/` → **zero hits** (stdlib-only preserved; jsonschema is test-only). - No schema files modified; no `schema_version` bumped. ## ⚠️ Findings surfaced (the whole point of #62 — schemas drifted from code) The new validation immediately caught **two real contract drifts**. Per #62's rule, gemini did NOT loosen the schemas; it documented the gaps: 1. **`review-artifact.schema.json` rejects the soft-fail `runtime_metadata` fields** (`error_kind`, `error_detail`, `fell_back`, `model_used`) that PR #54 added to `review_run` output. The schema predates #54 and is now stricter than the code. The soft-fail test asserts the *current* (broken) rejection so the suite is honest about the drift. 2. **`finding-resolution.schema.json` does not exist** — `resolve_findings()` emits `patchwarden.finding_resolution.v1` but there is no schema file for it. The fourth schema slot was covered with the registry schema instead. → Tracked for fix in a follow-up issue (see linked). Once fixed, the soft-fail `assertRaises` flips to a positive `validate()`. ## Cousin-discipline notes - unittest, not pytest (pytest stays opt-in). ✅ - Commit authored + committed by `gemini` (fixed after a worktree shared-`.git/config` identity clobber during parallel cousin work — lesson logged). Closes #62.

Rows
Columns