Desktop Speech Recognition WebSocket

Stream audio for real-time recognition while preserving the Desktop session, transcript, recording, diarization, and summary lifecycle.

Endpoint: /api/v1/desktop-services/speech-recognition

WebSocket subprotocol: koldan.desktop.v1

Requires Authentication - Scopes: desktop:app:use

Concurrency Limited - The installation can limit simultaneous Desktop recognition sessions.

Connect

Open a secure WebSocket connection and offer koldan.desktop.v1 in the Sec-WebSocket-Protocol header. The server rejects an upgrade that does not offer this subprotocol with 400 Bad Request.

const socket = new WebSocket(
  "wss://koldan.dixilang.com/api/v1/desktop-services/speech-recognition",
  "koldan.desktop.v1",
);

The access token is sent in the first WebSocket message, not in the connection URL.

Message sequence

Messages must follow this order:

1. Open the connection with the koldan.desktop.v1 subprotocol.
2. Send one authenticate text message.
3. Send one binary SpeechRecognitionRequest containing header.
4. Send zero or more binary SpeechRecognitionRequest messages containing packet.
5. Receive binary SpeechRecognitionResponse messages while recognition is active.
6. Send one complete text message after the final audio packet.
7. Continue reading responses until the server sends completed, then closes normally.

The server does not send a separate authentication acknowledgement. Ordered WebSocket delivery allows the client to send the header immediately after authenticate. The first SpeechRecognitionResponse.info confirms that recognition started and supplies the Desktop session identifier.

By default, the client must send authenticate within 30 seconds of opening the connection and the recognition header within 30 seconds of successful authentication. Either deadline closes the connection with an error message and WebSocket close code 1008. Installations can configure both deadlines.

Client text messages

authenticate

The first client message must authenticate the connection.

Authentication

{
  "type": "authenticate",
  "authentication": {
    "token": "eyJhbGciOiJSUzI1NiIs...",
    "tenantId": "c5a7d459-f313-4aa0-a978-182e7e3d2a3b"
  }
}

complete

Signal that no more recognition requests will be sent. This half-closes client input; the connection remains open while the server finishes processing buffered audio.

{
  "type": "complete"
}

Send complete only after authentication, the header, and all audio packets. Do not send another client message afterward.

Client binary messages

Every client binary message is one serialized Api.SpeechRecognitionRequest protobuf message. Do not add a length prefix, envelope, JSON encoding, or base64 encoding.

The first binary message must contain header. Every later binary message must contain packet.

SpeechRecognitionRequest

Exactly one step variant can be set. A request with no variant is invalid for this protocol.

SpeechRecognitionHeader

Protobuf field 4 is reserved and must not be used.

OnlineSpeakerDiarizationSettings

SpeechRecognitionHeaderSummariesRequest

SpeechRecognitionPacket

Use short audio packets suitable for live streaming. A packet is wrapped in SpeechRecognitionPacket, then in SpeechRecognitionRequest; the WebSocket binary message contains the serialized outer request.

Server binary messages

Every server binary message is one serialized Api.SpeechRecognitionResponse protobuf message. Decode the complete WebSocket message directly, without removing a length prefix.

SpeechRecognitionResponse

Exactly one response oneof variant is set in each server binary message.

SpeechSessionInfo

SpeechSessionResult

Hypothesis

Word

Server text messages

completed

Sent after complete when all buffered audio and final results have been processed. The server then closes the WebSocket with close code 1000.

{
  "type": "completed",
  "requestId": "ws-2df8573e"
}

error

Sent when authentication, message ordering, validation, capacity, or recognition fails. After this message, the server closes the connection.

{
  "type": "error",
  "code": "FAILED_PRECONDITION",
  "message": "The first recognition message must contain a header",
  "requestId": "ws-2df8573e"
}

Common status codes:

Client, authentication, permission, and ordering errors close with WebSocket code 1008. Capacity and temporary-availability errors close with 1013; other server failures close with 1011. A client or network disconnect before completed cancels the Desktop recognition session; no terminal text message can be delivered over an already closed connection.

During a planned service shutdown, active recognition connections are allowed to finish until the configured drain deadline. New connections receive 503 Service Unavailable, or an upgraded connection caught during shutdown receives UNAVAILABLE and closes with code 1013. Clients can reconnect to another available instance.

Message limits

An installation can configure different limits. Keep authentication messages small, split audio across packet messages, and allow room for recognition headers containing summary metadata. A message rejected by the WebSocket transport for exceeding a limit may close without a protocol error message.

Browser serialization outline

Use generated protobuf bindings for Api.SpeechRecognitionRequest and Api.SpeechRecognitionResponse. The method names depend on the selected protobuf generator; the wire steps remain the same:

socket.addEventListener("open", () => {
  socket.send(JSON.stringify({
    type: "authenticate",
    authentication: { token: accessToken },
  }));

  socket.send(SpeechRecognitionRequest.create({
    header: {
      presetName: "general-he",
      enableEndpointDetection: true,
    },
  }).toBinary());
});

socket.addEventListener("message", async (event) => {
  if (event.data instanceof Blob) {
    const response = SpeechRecognitionResponse.fromBinary(
      new Uint8Array(await event.data.arrayBuffer()),
    );
    handleRecognitionResponse(response);
    return;
  }

  const control = JSON.parse(event.data);
  if (control.type === "completed") {
    console.log("Recognition completed", control.requestId);
  } else if (control.type === "error") {
    console.error(control.code, control.message, control.requestId);
  }
});

Voice Studio dictation always sends enable_itn=true and enable_word_mappings=true; these options have no Studio controls. Meeting mode sends both as false. API defaults remain false when omitted.

Set socket.binaryType = "arraybuffer" if the client should receive binary messages as ArrayBuffer instead of Blob.