Critical title
Focused source 01:02–01:12
Focused source 01:02–01:12
Scrub the full 02:00 opening
Song title / named entity
Voxtype“caught and eye Joe”
VibeVoice“Cotton-Eyed Joe”
Voxtype VibeVoice
Local ROCm qualification · 2026-09-26
ASR + diarization comparison
A bounded, local comparison of transcription and speaker-segmentation implementations for the retained Zoom recording.
Yes — trimmed firstThe 76:48 terminal silent tail was removed before test clips were extracted.
07:24.192444.192 seconds total across the four tested clips; not a full-recording test.
10 GPU / 12 CPU pathsMatched measurements include transcript, word-timestamp, embedding, and segmentation lanes; unsupported backends are labeled explicitly.
What the test actually used
Trim status: Candidate clips were cut from
2026-09-26 10-35-24.trimmed.mp4, not the 02:21:12 original. The trim boundary is 01:04:24.178, removing 4,608.462 s of terminal silence. The “tail” sample is the last retained 84.192 s; it does not include the removed terminal tail.| Clip | Source-relative interval | Duration | Purpose |
|---|---|---|---|
| Opening | 00:00–02:00 | 120.000 s | Early multi-party planning, names, titles |
| Middle | 30:00–32:00 | 120.000 s | Rapid speaker changes |
| Sign-off | 52:00–54:00 | 120.000 s | Prayer, acknowledgements, short turns |
| Retained-end sample | 63:00–64:24.192 | 84.192 s | Non-speech handling before the cut |
Simple results
GPU and CPU times below are total processing times for all 07:24.192 of audio. Startup is included once when a model remained loaded for the four-clip batch; Voxtype and BitNet reload models per clip. GPU allocations are shown only where measured.
VibeVoice 1.5B correction: the existing result is only for independent 120 s / 84.192 s clips. It is not evidence for a 60-minute recording. Its local
streaming_generate wrapper first loads the entire clip, creates every encoded feature chunk, and retains transformer cache state across all chunks. A 60-minute run is therefore an unvalidated, likely memory-growth failure mode; do not use this implementation for the full recording until it is replaced with a bounded-state streaming driver and independently qualified.Voxtype Parakeet TDT
56.825 s GPU
31.870 s CPU / AVX2
VibeVoice 1.5B
36.860 s GPU
No CPU result: unsafe 60-minute path
whisper.cpp Large-v3-turbo
11.027 s GPU
251.119 s CPU
VibeVoice BitNet
116.876 s CPU
CPU-only official runtime
| Implementation | Output scope | GPU total | CPU total | Backend evidence |
|---|---|---|---|---|
| Voxtype Parakeet TDT ASR | Transcript | 56.825 s | 31.870 s | MIGraphX GPU retest; installed ONNX AVX2 binary for CPU |
| VibeVoice ASR Streaming 1.5B integrated ASR + labels | Transcript plus automatic speaker indices | 36.860 s | Not run — invalid full-input state model | HIP, gfx1201, and 4.96 GiB PyTorch peak recorded on bounded clips only |
| whisper.cpp Large-v3-turbo ASR | Transcript | 11.027 s | 251.119 s | Upstream HIP build; ROCm0, RX 9070 XT gfx1201; CPU run used -ng |
| Qwen3-ASR 1.7B ASR | Transcript | 46.899 s | 350.263 s | PyTorch ROCm 7.2, gfx1201, 4.57 GiB peak allocation |
| Qwen3-ASR + ForcedAligner 0.6B ASR + word timestamps | Transcript with forced word times | 75.125 s | 459.267 s | Same ROCm stack; 4.57 GiB peak allocation. Both runs cover the matched pack. |
| faster-whisper Large-v3-turbo ASR | Transcript | 33.034 s | 150.357 s | CTranslate2 4.8.2 built with WITH_HIP for gfx1201; requires CT2_CUDA_ALLOCATOR=cub_caching |
| WhisperX Large-v3-turbo ASR + word alignment | Transcript plus word timestamps | 33.412 s | 106.740 s | CTranslate2 HIP ASR plus PyTorch ROCm alignment; GPU requires CT2_CUDA_ALLOCATOR=cub_caching |
| Parakeet Redux ASR | Transcript | Unsupported | 11.790 s | Photon/Kestrel CPU runtime; installed GPU runtime requires NVIDIA CUDA |
| Parakeet Ultra ASR | Transcript | Unsupported | 37.381 s | Photon/Kestrel CPU runtime; installed GPU runtime requires NVIDIA CUDA |
| VibeVoice BitNet / VibeASR.cpp ASR | Transcript | Unsupported | 116.876 s | Official C++ runtime is CPU-only; 8 threads, greedy decode |
| Nemotron3 Diarization segmentation | Speaker turns only | 6.672 s | 9.187 s | HIP, gfx1201, 281.6 MiB PyTorch peak; CPU float32 |
| pyannote Community-1 segmentation | Speaker turns only | 28.129 s | 193.725 s | HIP, gfx1201, 1.66 GiB PyTorch peak; CPU with PyAV decode |
| SpeechBrain ECAPA speaker embeddings | 3 s embedding windows; no clustering claim | 11.434 s | 18.584 s | Official PyTorch and torchaudio ROCm 7.2 wheels; 456.6 MiB PyTorch peak |
All displayed CPU times use the matched pack; Voxtype and BitNet reload models per clip. VibeVoice 1.5B is intentionally excluded from CPU measurement because the same full-input cache design makes a 60-minute run unsafe. GPU processing means model inference ran on the GPU. The Voxtype allocation is system-level rocm-smi sampling; other allocation values are PyTorch allocator measurements, so they are not identical accounting methods. WhisperX now uses its full alignment path on both backends; both WhisperX and faster-whisper require CT2_CUDA_ALLOCATOR=cub_caching on this gfx1201 system.
Segmentation output difference
Not a DER measurement: speaker identifiers are arbitrary. Per clip, labels were mapped by maximum temporal overlap; boundaries match when within 0.5 s. This compares the two outputs, not correctness against an audited human reference.
| Clip | Nemotron turns / labels / speech | pyannote turns / labels / speech | Agreement while both active | Boundary match |
|---|---|---|---|---|
| Opening | 55 / 6 / 85.920 s | 39 / 5 / 93.960 s | 97.37% | 69 of 109 Nemotron boundaries |
| Middle | 44 / 5 / 76.310 s | 45 / 5 / 78.452 s | 95.40% | 73 of 88 Nemotron boundaries |
| Sign-off | 39 / 6 / 44.560 s | 32 / 2 / 46.423 s | 88.58% | See comparison artifact |
| Retained end | 0 / 0 / 0 s | 0 / 0 / 0 s | — | — |
The opening and sign-off need audible review: pyannote marks more speech in the opening, while Nemotron emits more turns and more distinct labels in the sign-off.
Recommended modular path: transcription + segmentation
This preserves the light ASR baseline while replacing the weak speaker-clustering stage with a dedicated diarizer. It is the right shape for a fair quality trial; the final reconciliation step has not yet been run with Nemotron3 or pyannote on the full recording.
Step 1
Transcribe
Voxtype Parakeet TDT produces text in bounded ASR chunks.
56.825 s test total198.8 MiB→
Step 2
Segment speakers
Nemotron3 or pyannote produces speaker turns/boundaries from the same audio.
281.6 MiB or 1.66 GiB→
Step 3
Reconcile
Assign each ASR chunk to the overlapping dominant speaker turn; preserve uncertainty and overlap rather than inventing identity.
speaker-attributed transcriptAlternative integrated path: VibeVoice 1.5B directly emits text with automatic speaker indices. Its measured clip result is fast but costs 4.96 GiB of allocated GPU memory. It is not presently qualified for the 60-minute recording because the tested wrapper retains full-input/chunk state; its words and labels still require human review.
Current delivered baseline: Voxtype ASR + local spectral KMeans produced the existing draft. That KMeans stage is a weaker fallback, not the proposed dedicated segmentation step above. The table separates the tested diarizers because neither alone transcribes speech.
Coverage against the research shortlist
| Candidate / lane | Status | What happened |
|---|---|---|
| Voxtype Parakeet TDT baseline | Tested / retested | ASR baseline; ROCm/MIGraphX |
| Nemotron3 Diarization | Tested | Dedicated diarizer; ROCm |
| VibeVoice ASR Streaming 1.5B | Bounded clips only | Integrated ASR + automatic speaker labels; ROCm on ≤120 s clips. Not qualified for the 60-minute recording. |
| pyannote Community-1 | Tested | Authorized local diarizer; ROCm model inference with PyAV decoding |
| whisper.cpp HIP | Tested | Large-v3-turbo: 11.027 s ROCm versus 251.119 s CPU on the matched pack |
| VibeVoice BitNet / VibeASR.cpp | Tested | CPU-only: 116.876 s on the matched pack |
| VibeVoice ASR Streaming 7B; long-form VibeVoice ASR | 7B blocked; long-form untested | 7B GPU exhausted the 15.92 GiB device during model load; CPU did not complete the matched pack in one hour. The official 7B release contains no quantized checkpoint. Long-form VibeVoice was not downloaded because its suitability needs a bounded-state driver, not the known full-input streaming wrapper. |
| faster-whisper | ROCm repaired | Large-v3-turbo: 33.034 s ROCm / 150.357 s CPU. A locally built CTranslate2 4.8.2 WITH_HIP backend works only with CT2_CUDA_ALLOCATOR=cub_caching; its default allocator faults on gfx1201. |
| WhisperX | ROCm repaired | Full ASR plus word alignment: 33.412 s ROCm / 106.740 s CPU. It combines CTranslate2 HIP with PyTorch ROCm and the same CUB caching allocator workaround; every aligned JSON output was written. |
| Parakeet Redux; Parakeet Ultra | Tested | Photon CPU: 11.790 s Redux, 37.381 s Ultra. GPU runtime requires NVIDIA CUDA; Redux falsely emitted tail speech, Ultra emitted empty tail text. |
| Qwen3 ASR + ForcedAligner | Tested | Qwen3-ASR 1.7B: 46.899 s ROCm / 350.263 s CPU. With ForcedAligner 0.6B, transcript plus word timestamps: 75.125 s ROCm / 459.267 s CPU on the matched pack. |
| batched ECAPA baseline | ROCm repaired | Official matching PyTorch/torchaudio ROCm 7.2 wheels eliminate the CUDA-only dependency mistake. ECAPA emitted 149 three-second speaker embeddings in 11.434 s ROCm / 18.584 s CPU; no clustering-quality claim. |
| Sortformer4spk-v2.1; Insanely Fast Whisper | Blocked / unsupported | Sortformer access is gated; no terms were accepted. Insanely Fast Whisper supports neither this ROCm path nor CPU. |
Accuracy audit queue
No WER/DER claim: Listen to the target, enter the heard reference, and score each transcript: 2 exact/acceptable, 1 materially incomplete or wrong but understandable, 0 wrong. Named terms and prayer wording are critical. The score is browser-local and does not alter artifacts.
Opening · ~00:38–00:48
Ambiguous dance phrase
Voxtype: “the church clap”VibeVoice: “the church class”
Voxtype VibeVoice
Opening · ~00:12–00:20
Conditional phrase omitted or retained
Voxtype: “Um trombone, I could play…”VibeVoice: “if I bring my trombone, I could play…”
Voxtype VibeVoice
Middle · ~00:50–01:05
Question/comment list
Voxtype: “questions, queries, attitudes, frustrations”VibeVoice: “questions, crazed attitudes, frustrations, where?”
Voxtype VibeVoice
Critical prayer wording
Sign-off · ~00:25–01:05
Sign-off · ~00:25–01:05
Prayer introduction and closing
Voxtype: “I’m going to pray for a second.”
“We pray this name of Son, Jesus Christ.”VibeVoice: “I’m gonna pray first, okay?”
“We pray the same for the sins of Jesus Christ.”
“We pray this name of Son, Jesus Christ.”VibeVoice: “I’m gonna pray first, okay?”
“We pray the same for the sins of Jesus Christ.”
Voxtype VibeVoice
Retained end · full 84.192 s
Non-speech behavior
Voxtype: empty textVibeVoice: 29 × [Silence]
Voxtype VibeVoice