v0.17.0: window merge duplicates words on long audio (single-window decode of the same audio is correct)

[maboa]

Since v0.17.0, transcribing long audio duplicates occasional words. The same audio decoded as a single window is correct, so it is the cross-window merge rather than the acoustic model.

Noticed while upgrading an app from 0.15.6 to 0.17.0.

Reproduction

A ~660 s recording of a conference talk (single speaker, natural hesitant delivery), 16 kHz mono WAV, through the CLI at each tag:

fluidaudiocli transcribe talk.wav --word-timestamps --output-json out.json

	doubled-word pairs in 1,6xx words
v0.15.6	8
v0.17.0	9

Most are genuine — the speaker really does say "taking taking", "the the", "I I". The interesting ones are the two v0.17.0 adds:

v0.15.6   …they gave them no budget and uh no guidance on like what the regulation…
v0.17.0   …they gave them no budget and uh no no guidance on like what the regulation…
                                            ^^^^^

v0.15.6   …It also can use JSYC which is another experiment presentation library.…
v0.17.0   …It also can use JSYC, which is another experiment presentation presentation library.…
                                                             ^^^^^^^^^^^^^^^^^^^^^^^^^

v0.17.0 also loses one I I that v0.15.6 emitted, so the net count barely moves — but the inserted tokens are new.

The part that localises it

Cutting a 14.5 s clip of the same audio around each word — short enough to decode in one window, so no merging happens — gives the correct text at both versions:

clip around "no"            0.15.6: …gave them no budget and…
                            0.17.0: …gave them no budget and…
clip around "presentation"  0.15.6: …another experiment presentation library. Then…
                            0.17.0: …another experiment presentation library. Then…

Same model, same audio, correct output. The duplication appears only when that audio is part of a longer file that gets windowed.

Where I would look

This looks like the window-entry rework in AsrManager+Transcription.swift for #855 / #897 — every window after the first now re-decodes from frame 0 on a fresh decoder state with source-level emission suppression, leaving dedup "only the jitter margin" (redecodeEmissionJitterFrames = 5). The leaked tokens are short function words and word-initial repeats, which is what I would expect if a re-emitted token lands outside that margin or is not matched by reconcileFinalWindowSeam's head-stripping.

One caveat on the evidence: the reported startTime of both inserted tokens (179.28 s and 386.56 s) does not fall inside a nominal overlap region for 15 s windows at 13 s stride, so I could not confirm alignment from timestamps alone. The clip test above is the stronger signal.

Why the benchmarks may not show it

Both cases sit in hesitant, filler-heavy delivery ("uh no guidance", a restarted phrase). Clean read speech does not seem to trigger it — a 64 s single-speaker studio narration produced zero doubles at either version, with only punctuation differences. Utterance-level corpora like LibriSpeech would also never exercise the window merge at all.

Happy to share the sample audio if useful, and to test a patch.

Not a regression report against the fix itself

To be clear, 0.17.0 is otherwise a clear improvement in our testing — noticeably better sentence punctuation, ~28% faster, and the macOS 14 BNNS diarizer warning in 0.15.7 matters to us. The old mid-window entry dropping 10 s of speech is plainly worse than a duplicated "no". This is a rough edge in the new approach, not an argument for the old one.

[maboa]

Following up with the rest of the comparison on the same public recording, because the duplications are not the only change — and on balance the word accuracy is worse on this material.

Same file, same CLI invocation, v0.15.6 vs v0.17.0. Ignoring pure punctuation and casing differences there are 38 substantive word differences. Classifying them:

Regressions in v0.17.0

0.15.6   …We just received our 501c3, so we are sprinting ahead.
0.17.0   …We just received our 501, so we are sprinting ahead.

0.15.6   …It's a bit of an oversell in terms of the capabilities…
0.17.0   …It's a bit of an overcell in terms of the capabilities…

0.15.6   …create computational notebooks, store it very uh much in a low…
0.17.0   …create computational notebooks, store it v uh much in a low…

0.15.6   …you can serve this to hundreds of thousands of students.
0.17.0   …you can serve this to hundreds of thousands of s students.

0.15.6   …So that is what born the nonprofit.
0.17.0   …So that is what born the non profit.

0.15.6   …ME Python, it's a really cool library. You should check it out.
0.17.0   …ME Python is a really cool library. Should check it out.

0.15.6   …We have a research team called Brain AI. The BrainAI team is…
0.17.0   …We have a research team called BrainAI. The BrainNAI team is…

plus the two duplications in the original report. Nine in total.

Improvements in v0.17.0

0.15.6   …the Gotham Data Clinic and we are new nonprofit based in New York City.
0.17.0   …the Gotham Data Clinic and we are a new nonprofit based in New York City.

0.15.6   …it's the Python stat compile to WebAssembly.
0.17.0   …it's the Python stat compiled to WebAssembly.

0.15.6   Other any questions for Tion?
0.17.0   Are there any questions for Tion?

Four in total. The last is a genuinely good catch that v0.15.6 mangled.

So roughly nine regressions against four improvements on this file.

The pattern that ties most of them together is word-initial truncation and fragmentation: very → v, students → s students, 501c3 → 501, You should → Should, oversell → overcell. Together with the duplications in the original report, that reads as the same boundary-handling mechanism losing or repeating material at word onsets rather than as an acoustic-model change — which is consistent with the 14.5 s single-window clips decoding correctly at both versions.

If that is right, one fix may address all of it.

Why this matters for the use case: long-form interview and oral-history transcription, where the transcript is the archived artifact. Better sentence punctuation — which v0.17.0 clearly does deliver — is cosmetic and easily corrected by a human. A silently dropped c3 or a truncated word is not: nobody re-checks a finished transcript against the audio looking for them. On separate private material we also saw a proper noun lost and one sentence's meaning inverted, which I cannot share but which fits the same shape.

We have held our 0.15.6 → 0.17.0 upgrade on this basis. Happy to test a patch against both recordings.

[maboa]

Additional data point: it does not reproduce on a different encoding of the same talk.

I have two encodings of this recording, and they behave differently at v0.17.0:

	what it is	duration
A	an editor render of the talk, with burned-in captions	659.7 s
B	a YouTube download of the same talk	597.7 s

The duplications in the original report were measured on A. On B:

                  v0.15.6                      v0.17.0
no guidance       …and um uh no guidance…      …and um uh no guidance…      ← both correct
presentation      …experiment presentation…    …experiment presentation…    ← both correct
oversell          …a bit of an oversell…       …a bit of an oversell…       ← both correct

Doubled-word counts: 8 vs 9 on A, and 5 vs 6 on B in ~1,650 words.

v0.15.6 produced consistent output across both encodings. v0.17.0 did not. That seems the more precise statement of the problem than "v0.17.0 transcribes worse" — the window merge appears more sensitive to whatever differs between these two files than v0.15.6's was. I have not isolated which property is responsible; A is 62 s longer, carries burned-in captions, and has been through an extra lossy round-trip.

Worth saying plainly that both files are ordinary input. Neither is a master, and users transcribe re-encoded material constantly — exports, downloads, conference renders. A regression that only shows on re-encoded audio is still a regression, and arguably a more commonly hit one.

One accuracy difference does reproduce on B:

v0.15.6   …based in New York City. We just received our 501c3, so we are sprinting ahead.
v0.17.0   …based in New York City. We just received our 501,   so we are sprinting ahead.

The single-window observation from the original report also stands: on A, a 14.5 s clip decoded correctly at both versions while the full file duplicated — which points at the merge rather than the acoustic model.

Happy to share either file.

[maboa]

Stronger evidence, on a different and much longer recording. A 36-minute two-speaker interview, 2,173 s, carrying no encoder tag (unlike the two conference-talk encodings above, this appears to be a recording platform's own output rather than a re-render).

Transcribed through the CLI at each tag. I cannot share this audio or quote it — it is a private interview — so these are described rather than excerpted. Each was verified by exact-string search in both outputs:

	v0.15.6	v0.17.0
a two-word proper noun (a surname following a forename)	present	surname dropped
a clause of the form "I wanted to …"	present	absent
the same clause as "I don't want to …"	absent	present
a quantifier phrase "wasn't a lot of …"	present	absent
the same clause as "wasn't a …"	absent	present
doubled words in the full transcript	42	52

Two of those invert meaning. A negation is inserted where the speaker did not say one, and a quantifier phrase is dropped in a way that reverses the sense of the sentence. In both cases v0.15.6 got it right on the same audio.

Doubled words rise 42 → 52, about 24%, on material where v0.15.6 already reflects the speakers' genuine repetitions.

What ties this to the original report

Every failure in this issue is now the same shape — short tokens gained or lost:

• duplicated: no no, presentation presentation, and 10 more here
• truncated: very → v, students → s students
• dropped: 501c3 → 501, a surname, a quantifier phrase
• inserted: a negation

That is consistent with the window-merge rework rather than an acoustic-model change, and consistent with the single-window clip decoding correctly at both versions.

Why this matters more than the earlier examples

This is long-form interview material for an oral-history archive, where the transcript is the artifact that gets kept. A duplicated the is visible and harmless. A silently inserted "don't", or a dropped surname, is not — nobody re-checks a finished transcript against 36 minutes of audio looking for a negation that was never spoken.

We have held our 0.15.6 → 0.17.0 upgrade on this basis and will retest against a patch whenever you have one.