Skip to content

feat(p99lab): add Audrey STT plugin - #8145

Open
kollaikal-rupesh wants to merge 4 commits into
livekit:mainfrom
kollaikal-rupesh:p99lab-audrey-stt
Open

kollaikal-rupesh wants to merge 4 commits into
livekit:mainfrom
kollaikal-rupesh:p99lab-audrey-stt

Conversation

@kollaikal-rupesh

@kollaikal-rupesh kollaikal-rupesh commented Oct 26, 2026 •

Copy link
Copy Markdown

Summary

This PR adds livekit-plugins-p99lab, a speech-to-text plugin for Audrey, p99lab's speech-to-text model for
phone calls, meetings and accented speech. It is maintained by p99lab, the company that operates the API.

from livekit.plugins import p99lab

session = AgentSession(stt=p99lab.STT(language="en"), vad=silero.VAD.load(), llm=..., tts=...)

Self-serve sign-up at console.p99lab.com; maintainers can get a free key by DM.

What the plugin does

  • Streaming (STT.stream()): sends 16-bit mono PCM in 50 ms chunks over the API's /v1/stream WebSocket. A
    JSON configuration message goes first; flush() sends {"type": "flush"} and end_input() sends end and waits
    for done. Partials become INTERIM_TRANSCRIPT; each final becomes FINAL_TRANSCRIPT followed by
    END_OF_SPEECH (START_OF_SPEECH precedes the first partial of an utterance). Input at other rates is resampled
    to sample_rate (8, 16, 24 or 48 kHz) and stereo is mixed down. RECOGNITION_USAGE reports the seconds sent.
  • Files (STT.recognize()): posts the buffer as WAV to /v1/transcribe and returns one FINAL_TRANSCRIPT.
  • Options: language, prompt (up to 500 characters of context, such as names and product terms),
    endpoint_ms, partial_ms, sample_rate, base_url, stream_url, http_session. update_options()
    reconnects open streams with the new settings.
  • Configuration: P99LAB_API_KEY, plus an optional P99LAB_API_URL (the API base URL; the stream URL is
    derived from it). The public API serves 30 languages (English by default; "auto" detects English vs Spanish).

Errors and retries

  • Dropped connections, "server busy" and 503 reconnect under the session's APIConnectOptions.
  • Account refusals raise APIStatusError with retryable=False. body is a dict with a code:
    • 401: unknown or revoked key (unknown_key).
    • 402: the account cannot pay. body["reason"] is no_plan, no_credit or payment_failed, and
      body["console_url"] is the console billing page. When credit runs out mid-stream, the final of the turn in
      progress is delivered before the error.
    • 429 key_daily_limit. A 429 for too many parallel streams on one key remains retryable.
  • Exceptions are never chained from aiohttp errors, which can carry the request headers, so the API key does not
    reach logs or tracebacks.

Packaging

The layout follows the existing plugins: hatchling, version.py at the current workspace version, logger
livekit.plugins.p99lab, and Plugin.register_plugin. The workspace entry in the root pyproject.toml and the
p99lab extra in livekit-agents/pyproject.toml are one line each, and uv.lock is regenerated. Dependencies:
livekit-agents, aiohttp, numpy.

Happy to adjust naming, defaults or structure to match the repository's conventions.

Streaming and file speech-to-text with p99lab's Audrey API: livekit-plugins-p99lab, unit tests against a local stand-in server, workspace and extra registration.
@kollaikal-rupesh
kollaikal-rupesh requested a review from a team as a code owner October 26, 2026 22:29
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant