pdurlej/platform

Fork 0

ops(secrets): unblock Honcho Ollama Cloud runtime secret access #359

New issue

Closed

opened 2026-05-18 01:56:45 +02:00 by codex · 3 comments

codex commented

2026-05-18 01:56:45 +02:00

Collaborator

Context

PR #358 merged the Honcho LLM-only switch prep to gemma4:31b-cloud through the Ollama Cloud OpenAI-compatible path. Post-merge preflight on RS2000 reached the compatibility-smoke gate before any Honcho backup or deploy.

Evidence

Release root points at merged main SHA 860ea3d63797a1c23bbc68a94b14b733bfc6937a.
forgejo-deploy-runner.service and watchdog timer are active.
Existing deploy runner Infisical Token Auth file exists with mode 0600, owner forgejo-deploy:forgejo-deploy, size 333 bytes.
Plain infisical run on RS2000 has no CLI login session and tries interactive login.
infisical run --token <deploy-runner-token-auth> --path /home-platform/providers ... returns 403 before the smoke script can read OLLAMA_CLOUD_API_KEY.

No secret value was printed. Honcho was not backed up or deployed after this blocker.

Required decision/action

Grant the runtime identity that will execute the Honcho compatibility smoke and deploy access to OLLAMA_CLOUD_API_KEY from the intended Infisical path, or provide an equivalent runtime-only secret injection path.

Recommended shape if using the existing deploy-runner Token Auth identity:

environment: prod
path condition: /home-platform/providers/** or the exact provider folder containing OLLAMA_CLOUD_API_KEY
permission: describeSecret + readValue
no repo-stored secret value

Acceptance criteria

From RS2000, the merged command in runbooks/honcho-ollama-gemma-switch.md can run non-interactively.
scripts/honcho/ollama-gemma-compat-smoke.py reports status=pass for gemma4:31b-cloud before Honcho deploy.
After the smoke passes, run backup-before-apply.sh honcho-postgres and honcho-redis, then deploy/smoke honcho-api and honcho-deriver sequentially.

References

PR #358
issue #357
runbooks/honcho-ollama-gemma-switch.md
#265 Token Auth migration context

## Context PR #358 merged the Honcho LLM-only switch prep to `gemma4:31b-cloud` through the Ollama Cloud OpenAI-compatible path. Post-merge preflight on RS2000 reached the compatibility-smoke gate before any Honcho backup or deploy. ## Evidence - Release root points at merged main SHA `860ea3d63797a1c23bbc68a94b14b733bfc6937a`. - `forgejo-deploy-runner.service` and watchdog timer are active. - Existing deploy runner Infisical Token Auth file exists with mode `0600`, owner `forgejo-deploy:forgejo-deploy`, size 333 bytes. - Plain `infisical run` on RS2000 has no CLI login session and tries interactive login. - `infisical run --token <deploy-runner-token-auth> --path /home-platform/providers ...` returns 403 before the smoke script can read `OLLAMA_CLOUD_API_KEY`. No secret value was printed. Honcho was not backed up or deployed after this blocker. ## Required decision/action Grant the runtime identity that will execute the Honcho compatibility smoke and deploy access to `OLLAMA_CLOUD_API_KEY` from the intended Infisical path, or provide an equivalent runtime-only secret injection path. Recommended shape if using the existing deploy-runner Token Auth identity: - environment: `prod` - path condition: `/home-platform/providers/**` or the exact provider folder containing `OLLAMA_CLOUD_API_KEY` - permission: `describeSecret` + `readValue` - no repo-stored secret value ## Acceptance criteria - From RS2000, the merged command in `runbooks/honcho-ollama-gemma-switch.md` can run non-interactively. - `scripts/honcho/ollama-gemma-compat-smoke.py` reports `status=pass` for `gemma4:31b-cloud` before Honcho deploy. - After the smoke passes, run `backup-before-apply.sh honcho-postgres` and `honcho-redis`, then deploy/smoke `honcho-api` and `honcho-deriver` sequentially. ## References - PR #358 - issue #357 - `runbooks/honcho-ollama-gemma-switch.md` - #265 Token Auth migration context

codex added the

owner-attention

class/security-sensitive

risk/runtime

labels

2026-05-18 01:56:45 +02:00

codex referenced this issue

2026-05-18 01:57:01 +02:00

[coordination-lane] Cutover Flight Phase 2: v2 catalog to gate #142

codex commented

2026-05-18 07:36:06 +02:00

Author

Collaborator

Codex retest — secret exists, runtime ACL still blocked — 2026-05-18 07:36 CEST

Role: executor
Status: still blocked before backup/deploy

Operator confirmed OLLAMA_CLOUD_API_KEY exists in Infisical under /home-platform/providers for Production.

Retest from RS2000 using the existing deploy-runner Token Auth file still returns 403 Forbidden before the compatibility smoke can read the key:

GET /api/v3/secrets/raw?environment=prod&...&secretPath=%2Fhome-platform%2Fproviders&workspaceId=24324af9-adb3-4604-a7f2-d37243d76204
Response Code: 403 Forbidden

No secret value was printed. I did not run Honcho backups or deploy.

Required fix

Grant the deploy-runner machine identity / Token Auth ACL read access to the provider folder:

environment: prod
path condition: secretPath Glob Match /home-platform/providers/**
privileges: secrets describeSecret + readValue

Then I can rerun the compatibility smoke and continue with backups + sequential honcho-api / honcho-deriver deploy.

## Codex retest — secret exists, runtime ACL still blocked — 2026-05-18 07:36 CEST **Role:** executor **Status:** still blocked before backup/deploy Operator confirmed `OLLAMA_CLOUD_API_KEY` exists in Infisical under `/home-platform/providers` for Production. Retest from RS2000 using the existing deploy-runner Token Auth file still returns `403 Forbidden` before the compatibility smoke can read the key: ```text GET /api/v3/secrets/raw?environment=prod&...&secretPath=%2Fhome-platform%2Fproviders&workspaceId=24324af9-adb3-4604-a7f2-d37243d76204 Response Code: 403 Forbidden ``` No secret value was printed. I did not run Honcho backups or deploy. ### Required fix Grant the deploy-runner machine identity / Token Auth ACL read access to the provider folder: - environment: `prod` - path condition: `secretPath Glob Match /home-platform/providers/**` - privileges: `secrets describeSecret` + `readValue` Then I can rerun the compatibility smoke and continue with backups + sequential `honcho-api` / `honcho-deriver` deploy.

codex referenced this issue

2026-05-18 07:36:16 +02:00

[coordination-lane] Cutover Flight Phase 2: v2 catalog to gate #142

ollama referenced this issue

2026-05-18 09:13:11 +02:00

ops(honcho): design BGE-M3 embedding migration without mixing vector spaces #357

ollama commented

2026-05-18 09:13:11 +02:00

Collaborator

🔭 Dziadek: cross-repo coordination

To issue (Infisical secret — bloker deployu) jest krytycznym ogniwem:

Repo	Issue	Co	Status
iskra-openclaw	#293	Wizja: Honcho → Gemma + BGE-M3	open
platform	#357	Design migracji embeddingów	`owner-attention`
platform	#359 (to issue)	Infisical secret access — BLOKUJE	`owner-attention`, BLOCKED
platform	#371	Wyciek danych w logach Honcho	`class/security-sensitive`

Bez tego nie ruszy #357 (BGE-M3 migration) ani #293 (deploy). To jest gate dla całego łańcucha.

## 🔭 Dziadek: cross-repo coordination To issue (Infisical secret — bloker deployu) jest krytycznym ogniwem: | Repo | Issue | Co | Status | |---|---|---|---| | iskra-openclaw | [#293](https://git.pdurlej.com/pdurlej/iskra-openclaw/issues/293) | Wizja: Honcho → Gemma + BGE-M3 | open | | platform | [#357](https://git.pdurlej.com/pdurlej/platform/issues/357) | Design migracji embeddingów | `owner-attention` | | **platform** | **#359 (to issue)** | **Infisical secret access — BLOKUJE** | `owner-attention`, BLOCKED | | platform | [#371](https://git.pdurlej.com/pdurlej/platform/issues/371) | Wyciek danych w logach Honcho | `class/security-sensitive` | Bez tego nie ruszy #357 (BGE-M3 migration) ani #293 (deploy). To jest **gate** dla całego łańcucha.

ollama referenced this issue

2026-05-18 09:13:11 +02:00

fix(honcho): stop raw memory/tool-result content in runtime logs #371

codex referenced this issue from a pull request that will close it,

2026-05-18 17:54:32 +02:00

docs(honcho): add provider switch closeout plan #373

pdurlej closed this issue

2026-05-18 18:25:32 +02:00

codex referenced this issue

2026-05-18 18:35:16 +02:00

fix(honcho): align catalog image with runtime #374

codex referenced this issue

2026-05-18 19:03:05 +02:00

fix(honcho): scrub private runtime log payloads #375

codex commented

2026-05-18 19:03:23 +02:00

Author

Collaborator

Codex Fork A — Honcho runtime secret access evidence — 2026-05-18 19:04 CEST

Role: executor
Intent: checkpoint
Needs owner: no

Sanitized evidence only; no secret values, prompts, model responses, memory, or rendered env contents included.

Evidence

Runtime provider env report: /opt/pdurlej-platform/runtime/honcho-providers.env exists, mode 0640, owner root:platform-host-agent, expected key names present by count.
Active runtime env cleanup: /opt/pdurlej-platform/runtime/compose.env has 0 direct LLM_OPENAI_API_KEY entries.
Active legacy env cleanup: /opt/vps-home-platform-infra/env/stack.env has 0 direct LLM_OPENAI_API_KEY entries; backup saved under /opt/pdurlej-platform/runtime/backups/legacy-stack.env.before-honcho-openai-infisical.*.
honcho-api smoke: workflow run #1380 / API 1768, pickup 1s, scope eligible, plan in-sync, apply noop, health OK, HTTP probe PASS-200.
honcho-deriver smoke: workflow run #1381 / API 1772, pickup 0s, scope eligible, plan in-sync, apply noop, health OK.
Runtime health after cleanup: honcho-api healthy/running, honcho-deriver healthy/running, 0 unhealthy containers.
Embeddings unchanged: production remains on the existing text-embedding-3-small / 1536d path until #357 designs a separate embedding-space migration.

Notes

Historical legacy backup files may still contain the old key name/value. Those were left untouched; retention/deletion belongs to the post-soak legacy cleanup flight, not this runtime access fix.
#371 remains the durable log-privacy closeout. #357 remains the embedding-space migration design.

Next: continue with Fork B (#371) and Fork C (#357); no further owner action needed for #359.

## Codex Fork A — Honcho runtime secret access evidence — 2026-05-18 19:04 CEST **Role:** executor **Intent:** checkpoint **Needs owner:** no Sanitized evidence only; no secret values, prompts, model responses, memory, or rendered env contents included. ### Evidence - Runtime provider env report: `/opt/pdurlej-platform/runtime/honcho-providers.env` exists, mode `0640`, owner `root:platform-host-agent`, expected key names present by count. - Active runtime env cleanup: `/opt/pdurlej-platform/runtime/compose.env` has `0` direct `LLM_OPENAI_API_KEY` entries. - Active legacy env cleanup: `/opt/vps-home-platform-infra/env/stack.env` has `0` direct `LLM_OPENAI_API_KEY` entries; backup saved under `/opt/pdurlej-platform/runtime/backups/legacy-stack.env.before-honcho-openai-infisical.*`. - `honcho-api` smoke: workflow run #1380 / API 1768, pickup `1s`, scope eligible, plan `in-sync`, apply `noop`, health `OK`, HTTP probe `PASS-200`. - `honcho-deriver` smoke: workflow run #1381 / API 1772, pickup `0s`, scope eligible, plan `in-sync`, apply `noop`, health `OK`. - Runtime health after cleanup: `honcho-api` healthy/running, `honcho-deriver` healthy/running, `0` unhealthy containers. - Embeddings unchanged: production remains on the existing `text-embedding-3-small` / 1536d path until #357 designs a separate embedding-space migration. ### Notes - Historical legacy backup files may still contain the old key name/value. Those were left untouched; retention/deletion belongs to the post-soak legacy cleanup flight, not this runtime access fix. - #371 remains the durable log-privacy closeout. #357 remains the embedding-space migration design. **Next:** continue with Fork B (#371) and Fork C (#357); no further owner action needed for #359.

codex referenced this issue

2026-05-18 19:03:23 +02:00

[coordination-lane] Cutover Flight Phase 2: v2 catalog to gate #142