Repository navigation
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. Pages changed in this PR: |
qgallouedec
left a comment
There was a problem hiding this comment.
Really cool. I haven't run myself, but very relevant considering our current effort around swarms
|
Thanks Kashif, great fit for an example. A few things
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 039f2ec. Configure here.
|
Thanks @kashif, great to have the first multi-agent example in TRL! A few things:
|

What does this PR do?
Adds
examples/async_grpo_s3_aggregation, a small replication of the information-aggregation task from s3. Four Qwen3-0.6B agents message each other to pool private numbers. OneAsyncGRPOTrainerrollout is a whole team episode, from a customAsyncRolloutLoop.After 50 updates of 128 teams, held-out team success goes from 9.4% to 49.5%. The README has the run commands and results.
Before submitting
AI writing disclosure
Who can review?
Anyone in the community is free to review the PR once the tests have passed.
Note
Low Risk
Adds an isolated examples package and documentation only; training logic lives in example subclasses and does not modify library training code paths.
Overview
Adds
examples/async_grpo_s3_aggregation, an end-to-end replication of the s3 information-aggregation task: four agents with private numbers coordinate via tooling on a message board, and oneAsyncGRPOrollout is a full team episode.The example includes a token-level harness (
harness/task.py,harness/team.py), training viaTeamRolloutLoop/TeamTrainer(whole-teamgenerate_one, length-weightedcompute_advantages, loss scaling so each team counts equally per the paper’s Eq. 3), plusevaluate.pyand a README with commands and reported results (~55% held-out success after 50 updates).Documentation is updated in
async_grpo_trainer.md(multi-agent pointer),example_overview.md(index row), andpaper_index.md(s3 section with run command). No core TRL trainer API changes—only the new example and docs.Reviewed by Cursor Bugbot for commit 072879f. Bugbot is set up for automated code reviews on this repo. Configure here.