Skip to content

Python: Fix the auto_retry sample so it retries 429 errors - #10068

Open
Sohom chakraborty (sohom-cs) wants to merge 2 commits into
microsoft:mainfrom
sohom-cs:fix/auto-retry-sample-429
Open

Sohom chakraborty (sohom-cs) wants to merge 2 commits into
microsoft:mainfrom
sohom-cs:fix/auto-retry-sample-429

Conversation

@sohom-cs

Copy link
Copy Markdown

Motivation & Context

None of the three approaches in python/samples/02-agents/auto_retry.py retries a 429. They retry on retry_if_exception_type(openai.RateLimitError), but the chat clients raise ChatClientException with the RateLimitError on __cause__, so the predicate never matches. Reproduction in #9904.

Description & Review Guide

  • What are the major changes? Add is_rate_limit_error(), which matches a ChatClientException whose __cause__ is an openai.RateLimitError, and use it through retry_if_exception in all three approaches. The docstrings now say what is retried.
  • What is the impact of these changes? Sample only. With FoundryChatClient on a mock transport (429, then 200; SDK retries off), each approach now retries and returns the response instead of raising on the first 429. A 400 is still not retried. There are no tests for these samples, so none are added. Not tested against a live rate limit or with streaming.
  • What do you want reviewers to focus on? Whether checking __cause__ is how you want the sample to show this. The predicate requires ChatClientException, so a bare RateLimitError is not retried.

Related Issue

Fixes #9904

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: Claude Code helped with the issue analysis, the reproduction, the change, and this description. I ran the reproduction before and after the change.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

All three approaches in samples/02-agents/auto_retry.py retried on
retry_if_exception_type(openai.RateLimitError). Chat clients raise
ChatClientException and keep the RateLimitError on __cause__, so the
predicate never matched and a 429 was raised on the first attempt.

Add an is_rate_limit_error() predicate that checks the cause, and use it
in all three approaches.
@sohom-cs

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the update to handle wrapped rate-limit errors across all three retry implementations.

No open issues 🎉

What changed in this PR

Fixes the Python auto-retry sample for #9904 by recognizing OpenAI rate-limit errors wrapped in ChatClientException, allowing the sample’s retry layer to handle 429 responses.

Changes:

  • Adds a shared predicate that checks the exception type and its __cause__.
  • Applies the predicate to all three retry implementations.
  • Updates documentation to explain wrapped errors and retry behavior.
File Description
python/​samples/​02-agents/​auto_retry.py Corrects rate-limit detection across all retry approaches and clarifies documentation.

Review effort: Balanced

@sohom-cs

Copy link
Copy Markdown
Author

#9904 was closed on 11-06, but auto_retry.py on main still retries only a bare RateLimitError in all three approaches, and the clients still raise ChatClientException, so the sample still doesn't retry a 429. This PR is the fix. Eduard van Valkenburg (@eavanvalkenburg) could you take a look when you have time?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks correct. Here's what I checked, plus one suggestion for the sample text.

  • The predicate matches what the clients raise. RawOpenAIChatClient._handle_request_error (openai/agent_framework_openai/_chat_client.py:874-889) re-raises existing ChatClientExceptions unchanged and wraps everything else as raise ChatClientException(..., inner_exception=ex) from ex. So a 429 from the OpenAI SDK reaches the caller as a ChatClientException whose __cause__ is openai.RateLimitError, which is exactly what is_rate_limit_error checks. FoundryChatClient (used by all three approaches) goes through the same path for both the create and retrieve calls (:1045, :1070).
  • Errors that shouldn't be retried still aren't. A content-filter 400 becomes OpenAIContentFilterException (a ChatClientException subclass) with a BadRequestError cause, so it isn't retried, and neither is any other 400. reraise=True is unchanged, so after the last attempt the caller still gets the original ChatClientException, not a tenacity RetryError.
  • The three call sites are consistent. retry_if_exception(is_rate_limit_error) is used in all three, retry_if_exception_type is no longer imported, and RateLimitError is still imported for the predicate.
  • git merge-tree against main is clean.

Suggestion: mention the SDK's own retries. The OpenAI Python SDK already retries 429s twice by default with backoff (openai-python README, "Retries"). So with this sample, a persistent 429 is attempted up to 3 × 3 = 9 times before failing. Your mock-transport check turned SDK retries off, which is why the sample's retry showed up right away. One sentence in the module docstring would make the layering explicit for readers copying this into production: "the OpenAI SDK retries 429s itself (max_retries, default 2); these approaches add an outer retry on top, so set max_retries=0 on the underlying client if you want tenacity to be the only policy".

I didn't run the sample.

Reviewed with help from Claude Code.

@sohom-cs

Copy link
Copy Markdown
Author

Added a sentence on the SDK's own retries to the module docstring, with max_retries=0 on the OpenAI client as the way to make tenacity the only retry policy.

This branch was successfully deployed

1 active deployment
github-app-auth — cb3eaa65 Deployed Nov 15, 2026 by sohom-cs via add_label #27878
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: auto_retry.py sample never retries a 429: the client raises ChatClientException, not RateLimitError

3 participants