Phase 07 spike: voice clone TTS feasibility for Hermes audio (Polish male, operator's voice) #176

New issue

Closed

opened 2026-05-11 07:43:59 +02:00 by claude · 0 comments

claude commented

2026-05-11 07:43:59 +02:00

Collaborator

Context

Per operator voice-note 2026-05-11 07:26 + PR #174 comment: Hermes voice deliverable spec is Polish male, with operator-preferred exploration of voice cloning of operator's own voice as separate spike. ADR-0012 §G + GPT-5.5 Pro §7 cover the deliverable; this issue covers the feasibility spike for voice cloning.

Goal

Determine whether voice clone of operator's voice is feasible on RS2000 or VPS1000 with reasonable inference speed (target: 4-6 min audio rendered in <60s wall-clock), quality (target: operator-recognizable as own voice, ~B+ verisimilitude), and operator-bandwidth-acceptable setup cost (target: ≤1 audio-sample collection session of ≤30 min).

Scope

Survey TTS-with-voice-cloning options that run locally / self-hosted (NO API-only services for operator's voice — privacy reason):
- Coqui-TTS (XTTS-v2) — multilingual including Polish, voice cloning from ≤10s sample
- Tortoise-TTS — slow but high quality; check Polish quality
- F5-TTS / E2-TTS — newer, faster
- Bark — multilingual; voice presets
- StyleTTS-2 — quality-focused; check Polish support
Benchmark top 2-3 on RS2000 (16GB RAM; per audit) + VPS1000 (TBD spec):
- Inference speed (4 min audio → wall-clock)
- Quality (operator A/B test with 3 sample texts)
- Model size + RAM footprint
- Polish language quality (Slavic phonemes can be hard for English-first models)
Operator sample collection protocol:
- How much audio is needed (e.g. 10 min reading vs 30 min varied texts)
- Source: existing voice notes operator has? Or dedicated recording session?
- Privacy: samples never leave operator-owned infrastructure
Output: feasibility report under state/reports/hermes-voice-clone-spike-2026-MM-DD.md with:
- Recommended TTS engine + hosting plan
- Operator's sample collection requirements
- Quality verdict (ship now / iterate / abandon)
- Cost: M1 vs RS2000 vs VPS1000 inference economics

Out of scope

Voice cloning of anyone other than operator (privacy)
Cloud-API services (ElevenLabs, etc.) — operator's voice should stay on operator-owned infrastructure
Production-quality (this is feasibility spike; ship-ready render is Phase 07 ticket if spike succeeds)

Phase

Phase 07 §G (Hermes pipeline parallel-track). Implementation order: spike → if PASS → Phase 07 implementation ticket adds clone option to Hermes voice synthesis pipeline.

Acceptance criteria

Feasibility report exists in state/reports/
Includes A/B audio samples (generic male TTS + operator-clone-attempt)
Includes inference economics on RS2000 + VPS1000
Operator listens to samples + makes ship/iterate/abandon decision

Refs

PR #174 (Hermes templates) + my comment on it
ADR-0012 §G (Phase 07 scope-lock — Hermes parallel-track)
GPT-5.5 Pro oracle review 2026-05-11 §7

Coordination

Lane: Phase 07 spike (advisor/exploration)
Owner: TBD (likely claude-orchestrator dispatches Ollama Cloud + local benchmarks)
Operator: provides voice samples; final ship/abandon decision
Codex: not involved until Phase 07 implementation ticket post-spike

Role: advisor (claude)

## Context Per operator voice-note 2026-05-11 07:26 + PR #174 comment: Hermes voice deliverable spec is **Polish male**, with operator-preferred exploration of voice cloning of operator's own voice as separate spike. ADR-0012 §G + GPT-5.5 Pro §7 cover the deliverable; this issue covers the **feasibility spike** for voice cloning. ## Goal Determine whether voice clone of operator's voice is feasible on RS2000 or VPS1000 with reasonable inference speed (target: 4-6 min audio rendered in <60s wall-clock), quality (target: operator-recognizable as own voice, ~B+ verisimilitude), and operator-bandwidth-acceptable setup cost (target: ≤1 audio-sample collection session of ≤30 min). ## Scope 1. **Survey** TTS-with-voice-cloning options that run locally / self-hosted (NO API-only services for operator's voice — privacy reason): - Coqui-TTS (XTTS-v2) — multilingual including Polish, voice cloning from ≤10s sample - Tortoise-TTS — slow but high quality; check Polish quality - F5-TTS / E2-TTS — newer, faster - Bark — multilingual; voice presets - StyleTTS-2 — quality-focused; check Polish support 2. **Benchmark** top 2-3 on RS2000 (16GB RAM; per audit) + VPS1000 (TBD spec): - Inference speed (4 min audio → wall-clock) - Quality (operator A/B test with 3 sample texts) - Model size + RAM footprint - Polish language quality (Slavic phonemes can be hard for English-first models) 3. **Operator sample collection protocol**: - How much audio is needed (e.g. 10 min reading vs 30 min varied texts) - Source: existing voice notes operator has? Or dedicated recording session? - Privacy: samples never leave operator-owned infrastructure 4. **Output**: feasibility report under `state/reports/hermes-voice-clone-spike-2026-MM-DD.md` with: - Recommended TTS engine + hosting plan - Operator's sample collection requirements - Quality verdict (ship now / iterate / abandon) - Cost: M1 vs RS2000 vs VPS1000 inference economics ## Out of scope - Voice cloning of anyone other than operator (privacy) - Cloud-API services (ElevenLabs, etc.) — operator's voice should stay on operator-owned infrastructure - Production-quality (this is feasibility spike; ship-ready render is Phase 07 ticket if spike succeeds) ## Phase Phase 07 §G (Hermes pipeline parallel-track). Implementation order: spike → if PASS → Phase 07 implementation ticket adds clone option to Hermes voice synthesis pipeline. ## Acceptance criteria - [ ] Feasibility report exists in `state/reports/` - [ ] Includes A/B audio samples (generic male TTS + operator-clone-attempt) - [ ] Includes inference economics on RS2000 + VPS1000 - [ ] Operator listens to samples + makes ship/iterate/abandon decision ## Refs - PR #174 (Hermes templates) + my comment on it - ADR-0012 §G (Phase 07 scope-lock — Hermes parallel-track) - GPT-5.5 Pro oracle review 2026-05-11 §7 ## Coordination - Lane: Phase 07 spike (advisor/exploration) - Owner: TBD (likely claude-orchestrator dispatches Ollama Cloud + local benchmarks) - Operator: provides voice samples; final ship/abandon decision - Codex: not involved until Phase 07 implementation ticket post-spike **Role:** advisor (claude)