[davidzhao]

most STT plugins shared a very similar strategy (websocket, push audio, process server-side events) but duplicated a lot of code on connection lifecycle and error handling.

this created a lot of inconsistencies. some plugins completely ignored error frames, causing them to not handle fallback correctly when remotes stop responding. others had inconsistent timeouts.

since we now have a common monitor in #8123, this change improves the handling of timeouts & errors so plugins no longer have to worry about them.

also added a comprehensive test suite that describes each provider's protocol, so our plugins can run unit tests against that protocol against the full suite.

[iamdgarcia]

One datum for this monitor, from the soniox/simplismart half-open repro in #8031: the dangerous case never produces an ERROR frame until the pong timeout fires. A peer that vanishes without a FIN or RST accepts the upgrade and then answers nothing, and the plugin's keepalive writes keep succeeding into the half-open TCP connection (ten writes over one cycle in our capture, one every 5 s), because a write into that socket only fails once the send buffer fills. First detection was 51.5 s (ServerTimeoutError: No PONG received after 15.0 seconds), the reconnect landed 8 ms later into the same silent server, the cycle repeated every 46.0 s, and no exception ever reached the consumer.

So the thing worth centralizing is not only the handling but the reporting: the plugins that did notice did it via the pong timeout and then told nobody, so stt.FallbackAdapter never heard about it and the caller stayed unheard for the rest of the call. If WSMonitor owns reconnects as well as errors, the rule that covers this shape is the one from the #8031 thread: a reconnect that produces no traffic is a failure. It needs no tunable number and it fails safe both ways; the measured shape (51.5 s first detection, 8 ms reconnect, 46.0 s cycle, 10 keepalive writes succeeding) is there to test against.

[davidzhao]

One datum for this monitor, from the soniox/simplismart half-open repro in #8031: the dangerous case never produces an ERROR frame until the pong timeout fires. A peer that vanishes without a FIN or RST accepts the upgrade and then answers nothing, and the plugin's keepalive writes keep succeeding into the half-open TCP connection (ten writes over one cycle in our capture, one every 5 s), because a write into that socket only fails once the send buffer fills. First detection was 51.5 s (ServerTimeoutError: No PONG received after 15.0 seconds), the reconnect landed 8 ms later into the same silent server, the cycle repeated every 46.0 s, and no exception ever reached the consumer.

@iamdgarcia the latest should be addressing this in a consistent way across all plugins. could you try this branch and let me know how it works for you?

[iamdgarcia]

Ran the #8031 half-open repro against 23dee5d and it resolves the reporting gap.

Setup (same shape as the #8031 repro): local aiohttp WS server that accepts the upgrade and then answers nothing, no pongs; soniox plugin from this branch, max_retry=0, conn timeout 10s.

Before (1.8.8, numbers from the #8031 thread): first detection 51.5 s (ServerTimeoutError, No PONG after 15 s), reconnect 8 ms later into the same dead server, cycle repeating every 46.0 s, ten keepalive writes "succeeding" on the half-open socket, and no exception ever reaching the consumer over 115 s of draining.

After (23dee5d): a single connection, no reconnect cycle, and APIConnectionError("WebSocket connection lost" <- ServerTimeoutError: No WebSocket messages received for 6.0 seconds) raised to the consumer at 7.1 s wall clock (6.0 s idle timeout plus ~1 s of setup). The keepalive trap is bypassed by construction: the monitor keys on received activity, not on write success, so ten "successful" writes into the dead socket can no longer pass for liveness.

That is the property we argued for in #8031: detection that reports. Happy to run the same dead-socket probe against other migrated plugins (deepgram or assemblyai) if that is useful as an end-to-end complement to the harness contract tests.

[devin-ai-integration]

Devin Review found 1 new potential issue.

[devin-ai-integration]

🔴 Failed utterances disappear without an error

When OpenAI rejects the only transcription item, pending_items drops it without raising an error. The stream then finishes successfully with no transcript, so fallback cannot handle the failed utterance.

Learn more

The receiver tracks committed audio items in pending_items and waits for them before completing a drained stream. A failed item now leaves that set just like a completed item, but emits no speech event or error. If it was the only item, the stream reports success despite losing all user speech. If other items remain, they can complete normally, so treating every failed item as an immediate fatal error would lose valid later transcripts.

Example: A caller sends one utterance, ends input, and OpenAI sends conversation.item.input_audio_transcription.failed for its item followed by input_audio_buffer.cleared. pending_items becomes empty and the stream closes without a final transcript or failure; the caller cannot invoke fallback.

Recommended fix: Track failed item errors separately. Allow remaining committed items to finish, then propagate an API error if any item failed so the stream cannot report full success after dropping speech. Account for failures that occur before input is closed as well.

---

Was this helpful? React with 👍 or 👎 to provide feedback.