fix(tests): smoke.sh runtime drift rewrite #52

Merged

pdurlej merged 1 commit from codex/orders/smoke-sh-rewrite into main

2026-05-05 01:32:00 +02:00

codex commented

2026-05-04 02:06:19 +02:00

Collaborator

Canary status: missing — manual canary 3+3 required before merge

Canary Context Pack

Product story

tests/smoke.sh --all should be a trustworthy daily drift detector. PR #40 v1 found useful module drift, but canary showed the implementation could also produce false failures across many modules. This rewrite fixes the specific PR #40 findings and keeps runtime smoke independent from schema validation dependencies.

What changed

Rewrote tests/smoke.sh as runtime smoke only: manifest presence, runbook-derived container name, live container status, registry digest drift, HTTP health, optional module hook.
Added tests/validate-schema.sh for Python/jsonschema/PyYAML validation outside the runtime smoke path.
JSON output now uses jq, not printf-built JSON.
Digest comparison reads the live image RepoDigest via docker image inspect, not container .Image.

Choices made

Picked HIGH 3 option (b): split schema validation from runtime smoke.
Picked runbook Container: as primary container-name source; no home-platform- prefix fallback in the critical path.
Kept smoke-extra support as an existing extension hook, but no new remediation/cron framework.

Verification

bash -n tests/smoke.sh — PASS
bash -n tests/validate-schema.sh — PASS
tests/validate-schema.sh n8n-worker — PASS
tests/smoke.sh --json n8n-worker | jq — PASS
Selected live targets: n8n-worker OK, agaria-postgres OK, agent-plane-shadow-control OK; uptime-kuma and karakeep show real live drift/health findings.
tests/smoke.sh --all inventory: 80 modules checked, 63 OK, 17 FAIL.

n8n-worker drift status

Confirmed PR #40 v1 false-positive. Live RepoDigest matches modules/n8n-worker/module.yaml: sha256:72e2242d5d3b89f2501c9ffe4208b98bcd7345f3fa8c40a57d502d6c1a5315d3.

Known constraints

shellcheck is not installed locally, so static shellcheck was not run.
--all still surfaces real module drift/open-loop failures; this PR does not update module manifests.

Canary status: missing — manual canary 3+3 required before merge ## Canary Context Pack ### Product story `tests/smoke.sh --all` should be a trustworthy daily drift detector. PR #40 v1 found useful module drift, but canary showed the implementation could also produce false failures across many modules. This rewrite fixes the specific PR #40 findings and keeps runtime smoke independent from schema validation dependencies. ### What changed - Rewrote `tests/smoke.sh` as runtime smoke only: manifest presence, runbook-derived container name, live container status, registry digest drift, HTTP health, optional module hook. - Added `tests/validate-schema.sh` for Python/jsonschema/PyYAML validation outside the runtime smoke path. - JSON output now uses `jq`, not printf-built JSON. - Digest comparison reads the live image RepoDigest via `docker image inspect`, not container `.Image`. ### Choices made - Picked HIGH 3 option (b): split schema validation from runtime smoke. - Picked runbook `Container:` as primary container-name source; no `home-platform-` prefix fallback in the critical path. - Kept smoke-extra support as an existing extension hook, but no new remediation/cron framework. ### Verification - `bash -n tests/smoke.sh` — PASS - `bash -n tests/validate-schema.sh` — PASS - `tests/validate-schema.sh n8n-worker` — PASS - `tests/smoke.sh --json n8n-worker | jq` — PASS - Selected live targets: `n8n-worker` OK, `agaria-postgres` OK, `agent-plane-shadow-control` OK; `uptime-kuma` and `karakeep` show real live drift/health findings. - `tests/smoke.sh --all` inventory: 80 modules checked, 63 OK, 17 FAIL. ### n8n-worker drift status Confirmed PR #40 v1 false-positive. Live RepoDigest matches `modules/n8n-worker/module.yaml`: `sha256:72e2242d5d3b89f2501c9ffe4208b98bcd7345f3fa8c40a57d502d6c1a5315d3`. ### Known constraints - `shellcheck` is not installed locally, so static shellcheck was not run. - `--all` still surfaces real module drift/open-loop failures; this PR does not update module manifests.

codex added 1 commit

2026-05-04 02:06:19 +02:00

fix(tests): rewrite smoke runtime drift checks

canary-required / collect-diff (pull_request) Failing after 2s

Details

canary-required / canary (pull_request) Has been skipped

Details

6f20160a49

Per prompts/tooling-smoke-sh-rewrite.md.\n\n- remove Python package dependency from runtime smoke path\n- resolve containers from runbook instead of hardcoded home-platform prefix\n- compare registry RepoDigest safely via docker image inspect\n- emit JSON through jq and move schema validation to tests/validate-schema.sh\n\nVerification: bash -n, schema validator smoke, n8n-worker live smoke, selected live module checks, --all drift inventory. n8n-worker drift from PR #40 v1 is confirmed false-positive.