Message Protocol

All control messages are JSON. Audio data is sent as binary WebSocket frames.

Client → Server

Action	Message	Description
Start session	{"action": "start"}	Start the session. Wait for {"state": "listening", ...} before sending audio.
Stop session	{"action": "stop"}	Signal end of audio. The server drains remaining audio and sends {"state": "stopped"} before closing.
Stream audio	<binary>	PCM WAVE mono 16 kHz frames. See Audio Formats.

Sessions cannot be restarted. Disconnect and reconnect to start a new one.

Server → Client

State changes

Message	Description
{"state": "listening", "session_id": "<id>"}	Session is ready. session_id is the unique ID for this session. You can now send audio.
{"state": "stopped"}	Session has ended. The connection will close.
{"state": "shutting_down", "at": <unix_timestamp>}	Server will shut down at the given time (up to 3600 s in the future). Finish your session before then.

Transcription results

Results arrive continuously as audio is processed.

Partial results

Some sessions emit interim results that may change:

{"partial": "text so far"}

These are not final and will be replaced. Clients must also support sessions that only send final results.

Final results

{
  "result": [
    ["word", time_start_ms, time_end_ms, confidence],
    ["word", time_start_ms, time_end_ms, confidence]
  ],
  "text": "full sentence as a string"
}

Field	Type	Description
text	string	Full sentence combining all words in this result.
result	array	Per-word detail. Each entry: [word, start_ms, end_ms, confidence].
result[n][0]	string	Recognized word.
result[n][1]	integer	Word start time in milliseconds (relative to total audio sent).
result[n][2]	integer	Word end time in milliseconds.
result[n][3]	float	Confidence score 0–1.
speaker	string | null	Speaker label when diarization is active. Optional field.
sconf	float | null	Sentence-level confidence. Optional field.
channel	integer | null	Audio channel index. Optional field.

note

Timestamps are based on the total audio processed, not the wall-clock session duration. This means you can send audio at up to 2× real-time speed and timestamps will still be correct relative to the source audio.

Example

{
  "result": [
    ["Je", 12046, 12286, 1.0],
    ["hoort", 12286, 12526, 1.0],
    ["natuurlijk", 12526, 12796, 1.0],
    ["zeker", 12796, 13096, 1.0],
    ["verhalen", 13666, 14055, 0.9997]
  ],
  "text": "Je hoort natuurlijk zeker verhalen"
}

Errors

Message	Description
{"error": "Session not started"}	Audio was sent before {"action": "start"} was acknowledged.
{"error": "backend already listening"}	{"action": "start"} was sent while a session is already active.
{"error": "restarting of sessions is not supported"}	Cannot restart after {"action": "stop"}. Reconnect instead.
{"error": "unable to start backend"}	The transcription service could not start. Contact support if this persists.
{"error": "engine_not_responding"}	The transcription service stopped responding. Contact support if this persists.
{"error": "unauthenticated"}	Binary audio was received before the token was authenticated.
{"error": "license_missing_entitlement"}	On-prem license does not include the realtime entitlement.

WebSocket Close Codes

Code	Reason	Description
4400	missing_language	?language= parameter was not provided.
4400	invalid_language	?language= was provided but is not available for your organization.
4402	no_subscription_found	No active realtime subscription found for this token's organization.
4403	invalid_s2t_token	Token is invalid, deleted, or not a realtime-type token.
4403	unauthenticated	No token provided and no {"action": "start"} with a key field.
4444	server_shutdown	Server is performing an emergency shutdown. Reconnect immediately.
4451	license_missing_entitlement	On-prem license does not include realtime.