Skip to content

Custom Live STT floods logs with 'Unsupported message type: String' when Omi controls reach audio-only socket #24191

Open

Description

@yhinai

Problem

With Android Omi configured for Cloud Provider → Custom (Live), the Custom STT logs repeatedly show:

[WARN] 13:00:55 [customLive] Unsupported message type: String
[WARN] 13:00:55 [customLive] Unsupported message type: String
[WARN] 13:00:55 [customLive] Unsupported message type: String

Transcription otherwise works in the reported setup (a custom PCM16 WebSocket bridge). The warnings flood the diagnostic log. This warning alone does not establish that audio was lost.

App build/version was not recorded. The code analysis below uses upstream commit df73eba414e0db23c12839cb74b9ab59971ef84c; it is not a new device reproduction or a captured dump of the offending String payload.

Cause identified in the code

This exact warning is emitted on the outgoing send path, not while decoding transcript responses:

  1. PureStreamingSttSocket.send accepts only Uint8List / List<int> audio. Any String emits the warning and returns.
  2. CompositeTranscriptionSocket.send unconditionally calls primarySocket.send(message) before deciding whether to forward to the secondary Omi socket. Thus Omi-specific JSON control messages also reach the custom STT audio sender.
  3. sendEvidenceFrame emits a JSON String of type capture_evidence_frame immediately before each binary frame when source identity is present. This is a concrete high-frequency producer capable of explaining the warning flood. sendText / _sendClientState provide other String/control producers.

The individual payloads in the original log were not captured, so their exact message types remain unconfirmed. However, the routing mismatch above is visible in the current implementation.

Changing the custom server's transcript responses from text WebSocket frames to binary frames is not the appropriate fix for this warning: the warning occurs in send(), and the generic receive handler expects String JSON.

Reproduction

Device path:

  1. Configure a working Custom Live STT WebSocket endpoint.
  2. Start wearable transcription and inspect the Custom STT logs.
  3. With source evidence metadata enabled/present, observe the repeated warning as frames are submitted.

Deterministic code-level trigger:

  • Connect a composite with a real PureStreamingSttSocket as primary.
  • Call composite.send(jsonEncode({'type': 'capture_evidence_frame', ...})), followed by its binary audio payload.
  • The String is passed to the primary, whose audio-only sender logs the warning; the subsequent audio is handled separately.

Expected behavior / suggested fix

Separate audio, Omi backend controls, and provider-specific controls at the routing boundary:

  • Send binary audio to the custom STT primary; forward it to Omi only according to the existing raw-audio consent policy.
  • Route Omi-only controls such as client state and speaker metadata exclusively to the Omi secondary, respecting its degraded/disconnected state.
  • Keep capture_evidence_frame metadata ordered with its matching binary frame on the Omi connection. When raw forwarding is disabled, do not send an unpaired audio evidence claim to Omi.
  • Preserve provider-specific keepalive/close messages through the provider's explicit protocol methods; do not blindly forward arbitrary Strings to third-party endpoints or encode control JSON as PCM.
  • Prefer an explicit control-message API or socket capability over merely suppressing this warning.

Regression coverage

The existing composite test currently expects a speaker_assigned String to reach both sockets. Update that expectation for Omi-only controls.

Please cover:

  • Omi controls do not reach the custom audio socket.
  • Audio still reaches the primary with raw forwarding on and off.
  • Secondary evidence metadata and audio remain paired and ordered.
  • Raw forwarding off does not leak audio or emit unpaired evidence claims.
  • Secondary outages preserve the working primary.
  • Provider keepalive/close handling and normal transcript reception still work.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions