Repository navigation
feat(p99lab): add Audrey STT plugin - #8145
Open
kollaikal-rupesh wants to merge 4 commits into
Open
kollaikal-rupesh wants to merge 4 commits into
kollaikal-rupesh wants to merge 4 commits into
Conversation
Streaming and file speech-to-text with p99lab's Audrey API: livekit-plugins-p99lab, unit tests against a local stand-in server, workspace and extra registration.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds
livekit-plugins-p99lab, a speech-to-text plugin for Audrey, p99lab's speech-to-text model forphone calls, meetings and accented speech. It is maintained by p99lab, the company that operates the API.
Self-serve sign-up at console.p99lab.com; maintainers can get a free key by DM.
What the plugin does
STT.stream()): sends 16-bit mono PCM in 50 ms chunks over the API's/v1/streamWebSocket. AJSON configuration message goes first;
flush()sends{"type": "flush"}andend_input()sendsendand waitsfor
done. Partials becomeINTERIM_TRANSCRIPT; each final becomesFINAL_TRANSCRIPTfollowed byEND_OF_SPEECH(START_OF_SPEECHprecedes the first partial of an utterance). Input at other rates is resampledto
sample_rate(8, 16, 24 or 48 kHz) and stereo is mixed down.RECOGNITION_USAGEreports the seconds sent.STT.recognize()): posts the buffer as WAV to/v1/transcribeand returns oneFINAL_TRANSCRIPT.language,prompt(up to 500 characters of context, such as names and product terms),endpoint_ms,partial_ms,sample_rate,base_url,stream_url,http_session.update_options()reconnects open streams with the new settings.
P99LAB_API_KEY, plus an optionalP99LAB_API_URL(the API base URL; the stream URL isderived from it). The public API serves 30 languages (English by default;
"auto"detects English vs Spanish).Errors and retries
APIConnectOptions.APIStatusErrorwithretryable=False.bodyis a dict with acode:unknown_key).body["reason"]isno_plan,no_creditorpayment_failed, andbody["console_url"]is the console billing page. When credit runs out mid-stream, the final of the turn inprogress is delivered before the error.
key_daily_limit. A 429 for too many parallel streams on one key remains retryable.reach logs or tracebacks.
Packaging
The layout follows the existing plugins: hatchling,
version.pyat the current workspace version, loggerlivekit.plugins.p99lab, andPlugin.register_plugin. The workspace entry in the rootpyproject.tomland thep99labextra inlivekit-agents/pyproject.tomlare one line each, anduv.lockis regenerated. Dependencies:livekit-agents,aiohttp,numpy.Happy to adjust naming, defaults or structure to match the repository's conventions.