Repository navigation
Custom Live STT floods logs with 'Unsupported message type: String' when Omi controls reach audio-only socket #24191
Description
Problem
With Android Omi configured for Cloud Provider → Custom (Live), the Custom STT logs repeatedly show:
[WARN] 13:00:55 [customLive] Unsupported message type: String
[WARN] 13:00:55 [customLive] Unsupported message type: String
[WARN] 13:00:55 [customLive] Unsupported message type: String
Transcription otherwise works in the reported setup (a custom PCM16 WebSocket bridge). The warnings flood the diagnostic log. This warning alone does not establish that audio was lost.
App build/version was not recorded. The code analysis below uses upstream commit df73eba414e0db23c12839cb74b9ab59971ef84c; it is not a new device reproduction or a captured dump of the offending String payload.
Cause identified in the code
This exact warning is emitted on the outgoing send path, not while decoding transcript responses:
- PureStreamingSttSocket.send accepts only
Uint8List/List<int>audio. Any String emits the warning and returns. - CompositeTranscriptionSocket.send unconditionally calls
primarySocket.send(message)before deciding whether to forward to the secondary Omi socket. Thus Omi-specific JSON control messages also reach the custom STT audio sender. - sendEvidenceFrame emits a JSON String of type
capture_evidence_frameimmediately before each binary frame when source identity is present. This is a concrete high-frequency producer capable of explaining the warning flood. sendText / _sendClientState provide other String/control producers.
The individual payloads in the original log were not captured, so their exact message types remain unconfirmed. However, the routing mismatch above is visible in the current implementation.
Changing the custom server's transcript responses from text WebSocket frames to binary frames is not the appropriate fix for this warning: the warning occurs in send(), and the generic receive handler expects String JSON.
Reproduction
Device path:
- Configure a working Custom Live STT WebSocket endpoint.
- Start wearable transcription and inspect the Custom STT logs.
- With source evidence metadata enabled/present, observe the repeated warning as frames are submitted.
Deterministic code-level trigger:
- Connect a composite with a real
PureStreamingSttSocketas primary. - Call
composite.send(jsonEncode({'type': 'capture_evidence_frame', ...})), followed by its binary audio payload. - The String is passed to the primary, whose audio-only sender logs the warning; the subsequent audio is handled separately.
Expected behavior / suggested fix
Separate audio, Omi backend controls, and provider-specific controls at the routing boundary:
- Send binary audio to the custom STT primary; forward it to Omi only according to the existing raw-audio consent policy.
- Route Omi-only controls such as client state and speaker metadata exclusively to the Omi secondary, respecting its degraded/disconnected state.
- Keep
capture_evidence_framemetadata ordered with its matching binary frame on the Omi connection. When raw forwarding is disabled, do not send an unpaired audio evidence claim to Omi. - Preserve provider-specific keepalive/close messages through the provider's explicit protocol methods; do not blindly forward arbitrary Strings to third-party endpoints or encode control JSON as PCM.
- Prefer an explicit control-message API or socket capability over merely suppressing this warning.
Regression coverage
The existing composite test currently expects a speaker_assigned String to reach both sockets. Update that expectation for Omi-only controls.
Please cover:
- Omi controls do not reach the custom audio socket.
- Audio still reaches the primary with raw forwarding on and off.
- Secondary evidence metadata and audio remain paired and ordered.
- Raw forwarding off does not leak audio or emit unpaired evidence claims.
- Secondary outages preserve the working primary.
- Provider keepalive/close handling and normal transcript reception still work.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsSomeday
Activity