pdurlej/platform
1
1
Fork 0
Code Issues 44 Pull requests 1 Projects Releases Packages Wiki Activity Actions

chore(rs2000): add journald cap and disk threshold alerts #798

New issue
Closed
opened 2026-06-17 21:27:26 +02:00 by codex · 0 comments
codex commented 2026-06-17 21:27:26 +02:00
Collaborator
Copy link

Summary

RS2000 now has Docker log rotation and BuildKit GC, but journald and host-level alert thresholds are still policy-only. The closeout observed about 4G in journald and a healthy root filesystem at 58%, but there is no explicit early warning before 75%/85%.

Scope

  • Add an explicit journald size/age policy for RS2000.
  • Add or document root filesystem threshold checks: warn at 75%, urgent/operator attention at 85%.
  • Keep this read-mostly and policy-driven; do not surprise-delete logs.
  • Integrate with the existing disk report timer or an adjacent host-ops timer.

Spec sources (whitelist)

  • runbooks/rs2000-disk-hygiene.md
  • state/reports/rs2000-disk-hygiene-2026-06-17.md
  • state/reports/rs2000-runtime-health-closeout-2026-06-17.md

Do NOT read

  • Application logs beyond aggregate disk usage.
  • Secret-bearing runtime env files.
  • Full repo unless the existing disk report implementation points to a specific file.

Extracted context

Closeout baseline:

/dev/vda3 503G 276G 207G 58% /
/var/log/journal around 4G observed during investigation

Acceptance criteria

  • Journald cap is documented and implemented through a reversible systemd/journald config change.
  • Disk threshold behavior is documented and tested in dry-run/mock form before live rollout.
  • Existing platform-disk-report.timer either includes thresholds or points to the new check.
  • Verification includes journalctl --disk-usage, df -h /, and timer status.

Hard stops

  • Do not run destructive log vacuum as an unreviewed cleanup shortcut.
  • Do not delete application logs or Docker volumes.
  • Do not page/notify external systems until the notification path is explicitly reviewed.

Source

Created from RS2000 runtime health closeout on 2026-06-17.

## Summary RS2000 now has Docker log rotation and BuildKit GC, but journald and host-level alert thresholds are still policy-only. The closeout observed about 4G in journald and a healthy root filesystem at 58%, but there is no explicit early warning before 75%/85%. ## Scope - Add an explicit journald size/age policy for RS2000. - Add or document root filesystem threshold checks: warn at 75%, urgent/operator attention at 85%. - Keep this read-mostly and policy-driven; do not surprise-delete logs. - Integrate with the existing disk report timer or an adjacent host-ops timer. ## Spec sources (whitelist) - `runbooks/rs2000-disk-hygiene.md` - `state/reports/rs2000-disk-hygiene-2026-06-17.md` - `state/reports/rs2000-runtime-health-closeout-2026-06-17.md` ## Do NOT read - Application logs beyond aggregate disk usage. - Secret-bearing runtime env files. - Full repo unless the existing disk report implementation points to a specific file. ## Extracted context Closeout baseline: ```text /dev/vda3 503G 276G 207G 58% / /var/log/journal around 4G observed during investigation ``` ## Acceptance criteria - Journald cap is documented and implemented through a reversible systemd/journald config change. - Disk threshold behavior is documented and tested in dry-run/mock form before live rollout. - Existing `platform-disk-report.timer` either includes thresholds or points to the new check. - Verification includes `journalctl --disk-usage`, `df -h /`, and timer status. ## Hard stops - Do not run destructive log vacuum as an unreviewed cleanup shortcut. - Do not delete application logs or Docker volumes. - Do not page/notify external systems until the notification path is explicitly reviewed. ## Source Created from RS2000 runtime health closeout on 2026-06-17.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
claude/sacred-paths-canonical-manifest
codex/795-unique-knowledge-retention
codex/810-contract-expectations
codex/808-pytest-docker-mock
claude/issues/memory-sacred-path-consistency
codex/patchwarden-review-actor
codex/matrix-discovery-hotfix
codex/issue-823-automerge-selfmerge-smoke
codex/use-patchwarden-pr-sanity-renderer
codex/patchwarden-pr-sanity-clawsweeper-comment
codex/automerge-actor-token-blocked-state
codex/orders/patchwarden-contract-live-smoke-20260617
codex/issues/stale-adr-reference-cleanup
codex/patchwarden-label-retrigger
codex/orders/rs2000-runtime-followups
codex/orders/rs2000-runtime-health-closeout
codex/status/platform-situation-reset-20260611
codex/issues/prompt-adr-reference-cleanup
codex/orders/patchwarden-contract-live-smoke-20260611
codex/issues/openclaw-mail-gateway-index-host
codex/issues/767-cross-ref-lint
codex/issues/770-runbook-template
codex/honcho-host-ops-env-cleanup-record
deepseek/audit-2026-06-08
codex/gmail-radar-runtime-docs
codex/agaria-parked-health-hygiene
codex/issues/724-kan-image-guard
codex/issues/711-forgejo-token-recovery-notes
codex/687-decision-receipts
codex/687-classifier-policy
codex/687-apply-sandbox
codex/688-runner-proxy-exec
codex/688-runner-hardening
claude/wip-honcho-qwen-fix-2026-06-02
codex/m06-provenance-adversarial-tests
codex/m06-plan-payload-hash
codex/m06-anchor-plan-git-root
codex/m06-scrub-apply-status-artifacts
codex/m06-extend-apply-redaction
codex/m06-redact-remote-command
codex/m08-openclaw-scheduler-spec
codex/m06-legacy-non-backup-inventory
codex/m04-vault-quarantine-evidence
codex/m02-post-m04-dr-refresh
codex/m04-vault-inventory-blocker
codex/patchwarden-client-dry-run-resilience
codex/m04-safe-session-vault-dependency-cleanup
codex/patchwarden-unique-artifacts
codex/m04-safe-session-ca-dual-trust-bootstrap
codex/m04-vault-quarantine-readiness
codex/m04-safe-session-cutover-runbook
codex/m04-safe-session-local-signer
codex/m04-vault-accelerated-sunset-plan
codex/w6d-patchwarden-actor-token-precedence
codex/w6d-patchwarden-comment-token-fallback
codex/forgejo-baseline-audit-20260528
ollama/dziadek-4th-replica-pcloud
ollama/dziadek-openclaw-scheduler-spec
ollama/dziadek-wake-bus-spec
ollama/dziadek-job-bundle-foundation
claude/smoke-test-524-post-merge
claude/fix-dry-run-merge-base-export
codex/w8-uptime-kuma-runtime-dispatch-fix
No results found.
Labels
Clear labels   
W6d-automerge-calibration

   
agent/claude-code
Vistula actor: Claude Code.

  
agent/codex
Vistula actor: Codex.

  
agent/hermes
Vistula actor: Hermes Upstream.

  
agent/iskra
Vistula actor: Iskra.

  
agent/ollama
Vistula actor: Ollama or compatible model.

  
agent/patchwarden
Vistula actor: Patchwarden.

  
automerge-candidate
Candidate for narrow W6d autonomous merge readiness pilot

  
class/security-sensitive
Security-sensitive class of service: smallest coherent PRs and full canary

  
cutover-gate
blocks production declare / final RS2000 replacement

  
dependency/blocked
Vistula dependency state: blocked.

  
dependency/blocks-others
Vistula dependency state: blocks other work.

  
dependency/cross-repo
Vistula dependency scope: cross-repository.

  
dependency/needs-confirmation
Vistula dependency state: needs confirmation.

  
domain:agents
Agent identity, delegation, subagents, routing, or workflows.

  
domain:ci
CI, checks, Actions, runners, or release automation.

  
domain:docs
Documentation, packets, practices, or source-of-truth text.

  
domain:forgejo
Forgejo issues, PRs, labels, repo settings, or identity.

  
domain:infra
Infrastructure, storage, backup, DNS, network, or host substrate.

  
domain:memory
Memory, recall, write gates, salience, or Honcho-adjacent behavior.

  
domain:runtime
Runtime services, deploys, daemons, timers, or live behavior.

  
domain:signal
Signal UX, delivery, context, or outbound behavior.

  
domain:ux
Product UX, operator experience, or conversation ergonomics.

  
flow/architecture
Vistula lifecycle: architecture phase in progress.

  
flow/blocked
Vistula lifecycle: blocked on dependency or decision.

  
flow/deployed
Vistula lifecycle: deployed to runtime where applicable.

  
flow/done
Vistula lifecycle: work merged or otherwise done.

  
flow/implementation
Vistula lifecycle: implementation phase in progress.

  
flow/intake
Vistula lifecycle: incoming signal not refined yet.

  
flow/maintained
Vistula lifecycle: maintained/steady state.

  
flow/observed
Vistula lifecycle: observed after delivery.

  
flow/ready
Vistula lifecycle: ready for architecture or implementation.

  
flow/refining
Vistula lifecycle: Hermes/spec refinement in progress.

  
flow/retired
Vistula lifecycle: retired or intentionally closed.

  
flow/review
Vistula lifecycle: review phase in progress.

  
iterating
Orchestrator/Codex actively amending in response to review — operator can ignore until resolved

  
judge/codex-candidate
Candidate for Codex implementation.

  
judge/hermes-candidate
Candidate for Hermes discovery/research.

  
judge/low-confidence
Judgment confidence is low.

  
judge/needs-refinement
Needs clearer evidence or scope.

  
judge/operator-needed
Needs Piotr decision or input.

  
judge/p0
Urgent, operator attention or unblock now.

  
judge/p1
High value, schedule soon.

  
judge/p2
Useful, normal queue.

  
judge/p3
Low priority, keep but do not chase.

  
judge/park
Parked from current attention.

  
judge/patchwarden-candidate
Needs Patchwarden merge-safety attention.

  
judge/stale-priority
Existing priority judgment appears stale.

  
kind/adr
Vistula work kind: architecture decision record.

  
kind/bug
Vistula work kind: bug fix.

  
kind/chore
Vistula work kind: chore.

  
kind/feature
Vistula work kind: feature.

  
kind/infra
Vistula work kind: infrastructure.

  
kind/ops
Vistula work kind: operations.

  
kind/refactor
Vistula work kind: refactor.

  
kind/research
Vistula work kind: research/discovery.

  
large-impact
Per charter §3 review-by-impact: large impact (≥300 LOC OR new module OR identity/secret change OR architectural pivot) — Oracle pre-review recommended

  
merge/auto
Vistula merge mode: automated merge.

  
merge/manual
Vistula merge mode: manual merge.

  
merge/manual-dependency-conflict
Manual merge reason: dependency conflict.

  
merge/manual-failing-tests
Manual merge reason: failing tests.

  
merge/manual-merge-conflict
Manual merge reason: merge conflict.

  
merge/manual-missing-review
Manual merge reason: missing review.

  
merge/manual-operator-preference
Manual merge reason: operator preference.

  
merge/manual-red-zone
Manual merge reason: red-zone risk.

  
merge/manual-security-sensitive
Manual merge reason: security-sensitive work.

  
merge/manual-unclear-scope
Manual merge reason: unclear scope.

  
merge/manual-unknown
Manual merge reason: unknown.

  
meta
meta-decomposition issue; first agent decomposes into atomic children

  
mode:operator-only
Apply step requires explicit operator approval.

  
mode:patchwarden-iskra-approved
Work eligible for Patchwarden/Iskra approval inside repo policy.

  
mode:safe-auto
Low-risk work eligible for lightweight autonomous handling.

  
needs-operator-decision
PR is blocked on operator yes/no decision — fresh-Claude/Codex won't make this call alone

  
needs-triage
decomposing agent has open questions / unknowns

  
not-ready
reviewed but blocked on a precondition

  
observed/erroring
Vistula observation: erroring.

  
observed/needs-followup
Vistula observation: needs follow-up.

  
observed/pending
Vistula observation: pending.

  
observed/retire-candidate
Vistula observation: retire candidate.

  
observed/unused
Vistula observation: unused.

  
observed/used
Vistula observation: used.

  
operator-emotional
operator-emotional importance (Iskra memory etc.)

  
owner-attention
owner must decide; blocks default path

  
phase/02
Phase 02 cataloging

  
phase/03
Phase 03 control plane (platformctl)

  
priority:p0
Immediate safety, uptime, data, or operator-blocking issue.

  
priority:p1
Important active work; should be pulled soon.

  
priority:p2
Useful normal-priority work.

  
priority:p3
Opportunistic cleanup, research, or backlog work.

  
proposed
auto-generated by meta-decomposition; awaiting human review

  
ready-for-agent
reviewed and pickable; agents may claim per cookbook

  
ready-for-operator
PR is ready for operator review/merge — orchestrator finished its part

  
recovery
disaster recovery / restore semantics relevant

  
review:claude-reviewed
Reviewed by Claude-family agent.

  
review:codex-reviewed
Reviewed by Codex-family agent.

  
review:dziadek-reviewed
Reviewed by Dziadek pinch-point reviewer.

  
review:needs-human
Needs human review before merge or apply.

  
risk/exposure
incorrectly public/private/tailnet/auth/routed

  
risk/process
workflow/review/orchestration may produce bad outcomes

  
risk/product
work may have lost owner/user value

  
risk/runtime
platform may not run, recover, validate, behave reproducibly

  
safety:external-write
May write to external systems, users, rooms, repos, or services.

  
safety:no-prod-mutation
Must not mutate production or live external state.

  
safety:prod-impact
May affect production/runtime behavior.

  
safety:secret-touch
Touches secrets, credentials, auth, or secret-adjacent config.

  
size/large
Vistula size: large.

  
size/medium
Vistula size: medium.

  
size/small
Vistula size: small.

  
size/tiny
Vistula size: tiny.

  
size/unknown
Vistula size: not classified yet.

  
source/adr
Vistula source: ADR.

  
source/agent-generated
Vistula source: agent generated.

  
source/manual
Vistula source: manual entry.

  
source/operator-chat
Vistula source: operator chat.

  
source/voice-note
Vistula source: voice note.

  
status:blocked
Blocked by missing dependency, evidence, access, or operator decision.

  
status:codex-ready
Ready for a Codex implementation pass.

  
status:merged:pending-evidence
Merged but waiting for live or artifact evidence before closure.

  
status:needs-evidence
Needs proof, logs, tests, or operator-visible evidence.

  
status:operator-needed
Requires explicit operator input before safe progress.

  
status:parked
Intentionally deferred; not active campaign work.

  
tier/full
Full review tier per ADR-0007

  
tier/lite
Lite review tier per ADR-0007

  
tier/stacked
Stacked PR (base != main) escape hatch per ADR-0017. Requires consolidator-PR plan in PR body.

  
tier:0-platform-substrate
Data, secrets, backup, storage, or critical platform substrate.

  
tier:1-iskra-value-layer
Iskra/OpenClaw behavior, memory, runtime, or value-layer work.

  
tier:2-tools-products-modules
Tools, products, modules, experiments, and lower-blast-radius work.

  
type:bug
Incorrect behavior or regression.

  
type:chore
Maintenance, cleanup, metadata, or operational task.

  
type:docs
Documentation-only work.

  
type:feat
New capability or product behavior.

  
type:policy
Governance, protocol, boundary, or decision-contract change.

  
type:research
Investigation, spike, or evidence-gathering task.

No labels
W6d-automerge-calibration
agent/claude-code
agent/codex
agent/hermes
agent/iskra
agent/ollama
agent/patchwarden
automerge-candidate
class/security-sensitive
cutover-gate
dependency/blocked
dependency/blocks-others
dependency/cross-repo
dependency/needs-confirmation
domain:agents
domain:ci
domain:docs
domain:forgejo
domain:infra
domain:memory
domain:runtime
domain:signal
domain:ux
flow/architecture
flow/blocked
flow/deployed
flow/done
flow/implementation
flow/intake
flow/maintained
flow/observed
flow/ready
flow/refining
flow/retired
flow/review
iterating
judge/codex-candidate
judge/hermes-candidate
judge/low-confidence
judge/needs-refinement
judge/operator-needed
judge/p0
judge/p1
judge/p2
judge/p3
judge/park
judge/patchwarden-candidate
judge/stale-priority
kind/adr
kind/bug
kind/chore
kind/feature
kind/infra
kind/ops
kind/refactor
kind/research
large-impact
merge/auto
merge/manual
merge/manual-dependency-conflict
merge/manual-failing-tests
merge/manual-merge-conflict
merge/manual-missing-review
merge/manual-operator-preference
merge/manual-red-zone
merge/manual-security-sensitive
merge/manual-unclear-scope
merge/manual-unknown
meta
mode:operator-only
mode:patchwarden-iskra-approved
mode:safe-auto
needs-operator-decision
needs-triage
not-ready
observed/erroring
observed/needs-followup
observed/pending
observed/retire-candidate
observed/unused
observed/used
operator-emotional
owner-attention
phase/02
phase/03
priority:p0
priority:p1
priority:p2
priority:p3
proposed
ready-for-agent
ready-for-operator
recovery
review:claude-reviewed
review:codex-reviewed
review:dziadek-reviewed
review:needs-human
risk/exposure
risk/process
risk/product
risk/runtime
safety:external-write
safety:no-prod-mutation
safety:prod-impact
safety:secret-touch
size/large
size/medium
size/small
size/tiny
size/unknown
source/adr
source/agent-generated
source/manual
source/operator-chat
source/voice-note
status:blocked
status:codex-ready
status:merged:pending-evidence
status:needs-evidence
status:operator-needed
status:parked
tier/full
tier/lite
tier/stacked
tier:0-platform-substrate
tier:1-iskra-value-layer
tier:2-tools-products-modules
type:bug
type:chore
type:docs
type:feat
type:policy
type:research
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
pdurlej/platform#798
No description provided.