Conversation
The Azure provider only implemented the Responses API, so Foundry deployments that speak Chat Completions could not be used. On Chat Completions the deepseek/deepseek-v4-pro model sends `thinking` and `prompt_cache_key`, both of which Azure rejects with 400. The provider now dispatches openai-completions alongside azure-openai-responses. A stream wrapper resolves the Azure endpoint and deployment name onto the model first, since the shared openai-completions implementation reads both off the model. The request-shape differences are catalog compat rather than api code: reasoning_effort instead of DeepSeek's thinking field, no prompt cache parameters, reasoning_content kept on assistant turns so the prefix Foundry cached stays byte-identical, and effort clamped to the low/medium/high the deployment accepts. closes earendil-works#9645
jsanter27
force-pushed
the
feat/azure-chat-completions
branch
from
September 17, 2026 21:20
c5080a1 to
5f188f2
Compare
Contributor
|
@jsanter27 I want to change the provider name to azure as part of this change. things it will break:
I think these are the main things, and they are acceptable. adding provider aliases is lot of complexity for a narrow usecase. wdyt? makes sense to you? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements #9645.
azure-openai-responsesonly implemented the Responses API, so Foundry deployments that use Chat Completions (DeepSeek V4 Pro) couldn't work. Picks up @davidbrai's suggestion in #6228 to expand the Azure provider to other APIs.Only
deepseek-v4-prois in the built-in catalog, but the endpoint resolution is model-agnostic, so other Foundry deployments now route correctly when declared inmodels.jsonunder this provider withapi: "openai-completions": Llama 3.3 70B, Mistral Large 3, Grok 4.6, Kimi K3, Phi-4, etc. (full list)Decisions
model.api, asfireworksandcloudflare-ai-gatewayalready do, rather than a second provider: one Foundry resource, one key, one base URL.azureStreamsresolves the Azure endpoint onto the model before dispatch, sinceopenai-completionsreads it offmodel.baseUrland Azure models ship with an empty baseUrl.Impact
Azure users gain Chat Completions deployments. Nothing changes for the existing Responses path: same provider id, same auth, same env vars.
Verification
packages/ai/test/azure-openai-completions.test.ts.cacheRetentionvalue plusPI_CACHE_RETENTION=long, multi-turn reasoning replay, tool calling, etc.Follow Up
The
azure-openai-responsesprovider name is now not entirely accurate. A next piece of work could be to support provider aliases, so this can become a generalazureprovider without breaking existing configs.