Add dogfood evidence aggregator #52

Merged

pdurlej merged 1 commit from codex/dogfood-evidence-aggregator-29 into main

2026-05-16 11:22:53 +02:00

codex commented

2026-05-16 08:36:38 +02:00

Collaborator

Canary Context Pack

Product story

The dogfood window should not depend on chat memory or manual scrolling through CI runs. The operator needs one repeatable, low-friction way to aggregate fallow-py evidence before Phase B/C resumes.

What changed

Added scripts/dogfood/aggregate_evidence.py, a stdlib-only aggregator for Forgejo Actions run metadata and locally available pyfallow-report.json artifacts.
Added tests for agent-fix-plan reports, plain JSON reports, Forgejo run normalization, and CLI output.
Documented cron-friendly usage in docs/dogfood.md and updated docs/dogfood-evidence-status.md.
Removed now-completed #49/#50 operator actions from the dogfood status page.

Why it changed

This closes the infrastructure part of #29 without introducing a daemon, database, hosted service, or non-stdlib dependency.

Files touched

scripts/dogfood/aggregate_evidence.py
tests/test_dogfood_aggregator.py
docs/dogfood.md
docs/dogfood-evidence-status.md

Runtime evidence

python3.13 -m pytest -q tests/test_dogfood_aggregator.py
python3.13 scripts/dogfood/aggregate_evidence.py --repo pdurlej/fallow-py --runs-limit 3 --output /tmp/fallow-dogfood.md --json-output /tmp/fallow-dogfood.json
python3.13 -m compileall -q src tests mcp/src mcp/tests scripts/dogfood
python3.13 -m pytest -q
python3.13 -m pytest -q mcp/tests
PYTHONPATH=src python3.13 -m fallow_py analyze --root . --fail-on warning --min-confidence medium
PYTHONPATH=src:mcp/src python3.13 -m fallow_py analyze --root mcp --fail-on warning --min-confidence medium
git diff --check

Known constraints

This Forgejo instance exposes Actions run metadata through API, but the public Swagger surface does not expose an artifact-download endpoint. The aggregator therefore consumes locally available report artifacts; a trusted rs2000 cron can download/extract those artifacts by whatever mechanism is available operationally.

Explicit out-of-scope

New analyzer rules.
Phase B/C execution.
A hosted dashboard or database.
Secret/token management.

Requested decision

Approve and merge if the aggregator is appropriately small, testable, and useful for #29.

Merge blockers

Failing tests, non-stdlib runtime dependency, or misleading claims that artifacts are automatically downloaded by this script.

Closes #29.

## Canary Context Pack ### Product story The dogfood window should not depend on chat memory or manual scrolling through CI runs. The operator needs one repeatable, low-friction way to aggregate fallow-py evidence before Phase B/C resumes. ### What changed - Added `scripts/dogfood/aggregate_evidence.py`, a stdlib-only aggregator for Forgejo Actions run metadata and locally available `pyfallow-report.json` artifacts. - Added tests for agent-fix-plan reports, plain JSON reports, Forgejo run normalization, and CLI output. - Documented cron-friendly usage in `docs/dogfood.md` and updated `docs/dogfood-evidence-status.md`. - Removed now-completed #49/#50 operator actions from the dogfood status page. ### Why it changed This closes the infrastructure part of #29 without introducing a daemon, database, hosted service, or non-stdlib dependency. ### Files touched - `scripts/dogfood/aggregate_evidence.py` - `tests/test_dogfood_aggregator.py` - `docs/dogfood.md` - `docs/dogfood-evidence-status.md` ### Runtime evidence - `python3.13 -m pytest -q tests/test_dogfood_aggregator.py` - `python3.13 scripts/dogfood/aggregate_evidence.py --repo pdurlej/fallow-py --runs-limit 3 --output /tmp/fallow-dogfood.md --json-output /tmp/fallow-dogfood.json` - `python3.13 -m compileall -q src tests mcp/src mcp/tests scripts/dogfood` - `python3.13 -m pytest -q` - `python3.13 -m pytest -q mcp/tests` - `PYTHONPATH=src python3.13 -m fallow_py analyze --root . --fail-on warning --min-confidence medium` - `PYTHONPATH=src:mcp/src python3.13 -m fallow_py analyze --root mcp --fail-on warning --min-confidence medium` - `git diff --check` ### Known constraints This Forgejo instance exposes Actions run metadata through API, but the public Swagger surface does not expose an artifact-download endpoint. The aggregator therefore consumes locally available report artifacts; a trusted rs2000 cron can download/extract those artifacts by whatever mechanism is available operationally. ### Explicit out-of-scope - New analyzer rules. - Phase B/C execution. - A hosted dashboard or database. - Secret/token management. ### Requested decision Approve and merge if the aggregator is appropriately small, testable, and useful for #29. ### Merge blockers Failing tests, non-stdlib runtime dependency, or misleading claims that artifacts are automatically downloaded by this script. Closes #29.

codex added 1 commit

2026-05-16 08:36:38 +02:00

Add dogfood evidence aggregator

CI / Python 3.11 (push) Successful in 55s

Details

CI / Python 3.12 (push) Successful in 57s

Details

CI / Python 3.13 (push) Successful in 56s

Details

CI / Python 3.11 (pull_request) Successful in 54s

Details

CI / Python 3.12 (pull_request) Successful in 59s

Details

CI / Python 3.13 (pull_request) Successful in 57s

Details

6839c3a959

Implements Forgejo #29 with a stdlib-only evidence aggregator that combines Forgejo Actions run metadata and local fallow-py report artifacts into Markdown/JSON summaries. Documents cron usage and removes completed #49/#50 operator actions from the status page.

Verified:

- python3.13 -m pytest -q tests/test_dogfood_aggregator.py

- python3.13 scripts/dogfood/aggregate_evidence.py --repo pdurlej/fallow-py --runs-limit 3 --output /tmp/fallow-dogfood.md --json-output /tmp/fallow-dogfood.json

- python3.13 -m compileall -q src tests mcp/src mcp/tests scripts/dogfood

- python3.13 -m pytest -q

- python3.13 -m pytest -q mcp/tests

- PYTHONPATH=src python3.13 -m fallow_py analyze --root . --fail-on warning --min-confidence medium

- PYTHONPATH=src:mcp/src python3.13 -m fallow_py analyze --root mcp --fail-on warning --min-confidence medium

- git diff --check

codex commented

2026-05-16 08:47:24 +02:00

Author

Collaborator

Codex follow-up: local verification and Forgejo CI are green, but merge_pull_request as codex returns 405 User not allowed to merge PR. Operator merge is required.

Terminal action: approve-with-permission-gap. The change is ready from my side; I cannot press the merge button with the current branch/repo permissions.

Codex follow-up: local verification and Forgejo CI are green, but `merge_pull_request` as `codex` returns `405 User not allowed to merge PR`. Operator merge is required. Terminal action: approve-with-permission-gap. The change is ready from my side; I cannot press the merge button with the current branch/repo permissions.

pdurlej scheduled this pull request to auto merge when all checks succeed

2026-05-16 09:04:13 +02:00