[rakeshv247]

This PR introduces the new Melia-1 model for RT. It also adds a new LanguageInfo message from the transcriber, which is sent to the client whenever a new language is detected in the audio. This message includes the word delimiter and writing direction for the detected language.

[giorgosHadji]

Demonstrate the LanguageInfo message emitted by the melia-1 multilingual model.

LanguageInfo is sent the first time a new language code is detected in the
audio (once per language, immediately before that language appears in a
transcript message). It reports the word delimiter and writing direction to
use when rendering that language, which matters for right-to-left scripts
(e.g. Hebrew) and delimiter-free scripts (e.g. Mandarin, Japanese).

This streams several single-language clips back-to-back in one session, with
"language": "multi" and "model": "melia-1", so the server has to switch
languages mid-session and LanguageInfo fires once per language encountered.

Requires:
SPEECHMATICS_API_KEY set in the environment.
SPEECHMATICS_RT_URL=wss://preview.rt.speechmatics.com/v2 (melia-1 preview endpoint)

import asyncio
import wave
from dataclasses import dataclass
from pathlib import Path

from speechmatics.rt import (
    AsyncClient,
    AudioEncoding,
    AudioFormat,
    Model,
    ServerMessageType,
    TranscriptionConfig,
    TranscriptResult,
)

DEFAULT_WORD_DELIMITER = " "
RIGHT_TO_LEFT_EMBED = "\u202b"  # RLE
POP_DIRECTIONAL_FORMATTING = "\u202c"  # PDF


@dataclass
class LanguageInfo:
    word_delimiter: str
    writing_direction: str


LANGUAGE_CLIPS = [
    "tests/voice/assets/languages/de_de_000675.wav",
    "tests/voice/assets/languages/fr_fr_000378.wav",
    "tests/voice/assets/languages/es_419_000896.wav",
    "tests/voice/assets/languages/he_il_000432.wav",
    "tests/voice/assets/languages/cmn_hans_cn_000328.wav",
    "tests/voice/assets/languages/ja_jp_000595.wav",
    "tests/voice/assets/languages/th_th_000208.wav",
]

SAMPLE_RATE = 16000
SILENCE_GAP_SECONDS = 0.5


def load_clip_frames(path: str) -> bytes:
    """Read raw PCM16 frames from a mono 16kHz wav file."""
    with wave.open(path, "rb") as wf:
        assert wf.getframerate() == SAMPLE_RATE
        assert wf.getsampwidth() == 2
        assert wf.getnchannels() == 1
        return wf.readframes(wf.getnframes())


def render_transcript(result: TranscriptResult, languages: dict[str, LanguageInfo]) -> str:
    """Rebuild the transcript text using the word_delimiter/writing_direction
    reported by LanguageInfo for each word, instead of the server's own
    pre-joined `metadata.transcript` (which assumes a single language).
    """
    text = ""
    direction = "left-to-right"
    prev_attaches_to_next = False

    for item in result.results:
        if not item.alternatives:
            continue
        alt = item.alternatives[0]
        info = languages.get(alt.language) if alt.language else None
        delimiter = info.word_delimiter if info else DEFAULT_WORD_DELIMITER
        if info:
            direction = info.writing_direction

        if not text or item.attaches_to == "previous" or prev_attaches_to_next:
            text += alt.content
        else:
            text += delimiter + alt.content
        prev_attaches_to_next = item.attaches_to == "next"

    if direction == "right-to-left":
        text = RIGHT_TO_LEFT_EMBED + text + POP_DIRECTIONAL_FORMATTING
    return text


async def main() -> None:
    audio_format = AudioFormat(
        encoding=AudioEncoding.PCM_S16LE,
        sample_rate=SAMPLE_RATE,
        chunk_size=4096,
    )
    transcription_config = TranscriptionConfig(
        language="multi",
        model=Model.MELIA_1,
        enable_partials=True,
    )
    silence = b"\x00\x00" * int(SAMPLE_RATE * SILENCE_GAP_SECONDS)
    languages: dict[str, LanguageInfo] = {}

    async with AsyncClient() as client:

        @client.on(ServerMessageType.LANGUAGE_INFO)
        def handle_language_info(message):
            info = LanguageInfo(
                word_delimiter=message["word_delimiter"],
                writing_direction=message["writing_direction"],
            )
            languages[message["language"]] = info
            print(
                f"[LanguageInfo] language={message['language']!r} "
                f"word_delimiter={info.word_delimiter!r} "
                f"writing_direction={info.writing_direction} "
                f"partial={message.get('partial', False)}"
            )

        @client.on(ServerMessageType.ADD_PARTIAL_TRANSCRIPT)
        def handle_partial_transcript(message):
            result = TranscriptResult.from_message(message)
            text = render_transcript(result, languages)
            if text:
                print(f"[partial] {text}")

        @client.on(ServerMessageType.ADD_TRANSCRIPT)
        def handle_final_transcript(message):
            result = TranscriptResult.from_message(message)
            text = render_transcript(result, languages)
            if text:
                print(f"[final]   {text}")

        await client.start_session(
            transcription_config=transcription_config,
            audio_format=audio_format,
        )

        for clip in LANGUAGE_CLIPS:
            frames = load_clip_frames(str(Path(clip)))
            for offset in range(0, len(frames), audio_format.chunk_size):
                await client.send_audio(frames[offset : offset + audio_format.chunk_size])
            await client.send_audio(silence)

        await client.stop_session()


if __name__ == "__main__":
    asyncio.run(main())

Run:

 uv run main.py
[LanguageInfo] language='de' word_delimiter=' ' writing_direction=left-to-right partial=True
[partial] Die
[final]   Die
[partial] Einreise in das südliche Afrika mit dem Auto
[final]   Einreise in das südliche Afrika mit dem Auto
[partial] ist eine erstaunliche Möglichkeit, die
[final]   ist eine erstaunliche Möglichkeit, die
[partial] ganze Schönheit der Region zu sehen und an
[final]   ganze Schönheit der Region zu sehen und an
[partial] Orte abseits der normalen
[final]   Orte abseits der normalen
[partial] Touristenrouten zu gelangen.
[final]   Touristenrouten zu gelangen.
[LanguageInfo] language='fr' word_delimiter=' ' writing_direction=left-to-right partial=False
[final]   La partie extérieure que nous voyons lorsque nous regardons le soleil s'appelle la photosphère ce qui signifie boule de
[partial] lumière.
[final]   lumière.
[LanguageInfo] language='es' word_delimiter=' ' writing_direction=left-to-right partial=False
[final]   Es esencial que cuente cuando menos con calzado con suelas
[partial] apropiadas. Los zapatos de verano por lo general resbalan mucho en el hielo y la nieve.
[final]   apropiadas. Los zapatos de verano, por lo general, resbalan mucho en el hielo y la nieve.
[partial] Incluso hay botas de
[final]   Incluso hay botas de
[partial] invierno que no son adecuadas.
[final]   invierno que no son adecuadas.
[LanguageInfo] language='he' word_delimiter=' ' writing_direction=right-to-left partial=False
[final]   ‫טורקיה מוקפת ים משלושה כיוונים הים האגאי‬
[partial] ‫ממערב הים השחור מצפון והים התיכון מדרום.‬
[final]   ‫ממערב הים השחור מצפון והים התיכון מדרום.‬
[partial] ‫בוב ב'ק'ר שלושם לו קולודיה, 首都, שג'רג'לאבו,‬
[final]   ‫בו בי כה‬
[partial] ‫לו.‬
[final]   ‫לו.‬
[final]   ‫טו. סו.‬
[final]   ‫די.‬
[LanguageInfo] language='ja' word_delimiter='' writing_direction=left-to-right partial=False
[final]   薩格勒布在为貝尔格兰德游擊隊足球俱樂部效力時成名。
[final]   動物は地球上のいたるところに生息しています
[partial] 地面を掘ったり海を泳ぎ回ったり空を飛んだりしています。
[final]   地面を掘ったり海を泳ぎ回ったり空を飛んだりしています。