Repository navigation
Python: Fix the auto_retry sample so it retries 429 errors - #10068
Sohom chakraborty (sohom-cs) wants to merge 2 commits into
Conversation
All three approaches in samples/02-agents/auto_retry.py retried on retry_if_exception_type(openai.RateLimitError). Chat clients raise ChatClientException and keep the RateLimitError on __cause__, so the predicate never matched and a 429 was raised on the first attempt. Add an is_rate_limit_error() predicate that checks the cause, and use it in all three approaches.
|
@microsoft-github-policy-service agree |
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are reported for the update to handle wrapped rate-limit errors across all three retry implementations.
No open issues 🎉
What changed in this PR
Fixes the Python auto-retry sample for #9904 by recognizing OpenAI rate-limit errors wrapped in ChatClientException, allowing the sample’s retry layer to handle 429 responses.
Changes:
- Adds a shared predicate that checks the exception type and its
__cause__. - Applies the predicate to all three retry implementations.
- Updates documentation to explain wrapped errors and retry behavior.
| File | Description |
|---|---|
python/samples/02-agents/auto_retry.py |
Corrects rate-limit detection across all retry approaches and clarifies documentation. |
Review effort: Balanced
|
#9904 was closed on 11-06, but |
Muhammad Abdullah Butt (1AbdullahButt)
left a comment
There was a problem hiding this comment.
Looks correct. Here's what I checked, plus one suggestion for the sample text.
- The predicate matches what the clients raise.
RawOpenAIChatClient._handle_request_error(openai/agent_framework_openai/_chat_client.py:874-889) re-raises existingChatClientExceptions unchanged and wraps everything else asraise ChatClientException(..., inner_exception=ex) from ex. So a 429 from the OpenAI SDK reaches the caller as aChatClientExceptionwhose__cause__isopenai.RateLimitError, which is exactly whatis_rate_limit_errorchecks.FoundryChatClient(used by all three approaches) goes through the same path for both the create and retrieve calls (:1045,:1070). - Errors that shouldn't be retried still aren't. A content-filter 400 becomes
OpenAIContentFilterException(aChatClientExceptionsubclass) with aBadRequestErrorcause, so it isn't retried, and neither is any other 400.reraise=Trueis unchanged, so after the last attempt the caller still gets the originalChatClientException, not a tenacityRetryError. - The three call sites are consistent.
retry_if_exception(is_rate_limit_error)is used in all three,retry_if_exception_typeis no longer imported, andRateLimitErroris still imported for the predicate. git merge-treeagainstmainis clean.
Suggestion: mention the SDK's own retries. The OpenAI Python SDK already retries 429s twice by default with backoff (openai-python README, "Retries"). So with this sample, a persistent 429 is attempted up to 3 × 3 = 9 times before failing. Your mock-transport check turned SDK retries off, which is why the sample's retry showed up right away. One sentence in the module docstring would make the layering explicit for readers copying this into production: "the OpenAI SDK retries 429s itself (max_retries, default 2); these approaches add an outer retry on top, so set max_retries=0 on the underlying client if you want tenacity to be the only policy".
I didn't run the sample.
Reviewed with help from Claude Code.
|
Added a sentence on the SDK's own retries to the module docstring, with |
Motivation & Context
None of the three approaches in
python/samples/02-agents/auto_retry.pyretries a 429. They retry onretry_if_exception_type(openai.RateLimitError), but the chat clients raiseChatClientExceptionwith theRateLimitErroron__cause__, so the predicate never matches. Reproduction in #9904.Description & Review Guide
is_rate_limit_error(), which matches aChatClientExceptionwhose__cause__is anopenai.RateLimitError, and use it throughretry_if_exceptionin all three approaches. The docstrings now say what is retried.FoundryChatClienton a mock transport (429, then 200; SDK retries off), each approach now retries and returns the response instead of raising on the first 429. A 400 is still not retried. There are no tests for these samples, so none are added. Not tested against a live rate limit or with streaming.__cause__is how you want the sample to show this. The predicate requiresChatClientException, so a bareRateLimitErroris not retried.Related Issue
Fixes #9904
AI Assistance
AI assistance details: Claude Code helped with the issue analysis, the reproduction, the change, and this description. I ran the reproduction before and after the change.
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.