SageMaker Just Trained a Search Agent with Multi-Turn RL — But One Benchmark Regressed
AWS recently published a workflow for fine-tuning search agents using multi-turn reinforcement learning (MTRL) on SageMaker AI. The results are striking, on BrowseComp-Plus, the agent's failure rate dropped from 22.89% to 0.68%.
SageMaker Just Trained a Search Agent with Multi-Turn RL — But One Benchmark Regressed
AWS recently published a workflow for fine-tuning search agents using multi-turn reinforcement learning (MTRL) on SageMaker AI. The results are striking: on BrowseComp-Plus, the agent's failure rate dropped from 22.89% to 0.68%.
But the same model regressed on FreshStack — nDCG@10 slipped from 0.4112 to 0.4089.
Why Multi-Turn RL Is Different
Traditional fine-tuning evaluates single responses. But search agents make decisions across turns — an early query might look suboptimal yet return a result that enables a precise follow-up. MTRL optimizes the entire trajectory, using the final outcome as the reward signal.
What the Numbers Show
| Benchmark | Base Model | Fine-Tuned | Change |
|---|---|---|---|
| BrowseComp-Plus | 0.5136 | 0.6354 | +23.7% |
| WixQA | 0.5725 | 0.6781 | +18.4% |
| Wands | 0.5762 | 0.6112 | +6.0% |
| FreshStack | 0.4112 | 0.4089 | -0.6% |
On BrowseComp-Plus, average turns dropped from 7.0 to 6.3 — the model learned when to stop.
Why FreshStack Regressed
FreshStack contains recent developer questions from Stack Overflow and technical docs — content that depends on version-specific terminology and rapidly changing facts. A policy trained on broader retrieval datasets doesn't necessarily transfer to this distribution. AWS did not publish an error analysis explaining the regression.
How You Actually Use It
MTRL is fully serverless — pay per token, no GPU cluster provisioning. You can connect agents running on Bedrock AgentCore, EKS, EC2, Fargate, or any custom framework.
Supported models include Qwen 3.6 27B, Nova Lite 2.0, GPT-OSS-20B, and Gemma 31B.
Only three hyperparameters needed changing from defaults:
max_epochs, global_batch_size, and rollout_max_concurrency .The Caveat
The FreshStack regression is a reminder: specialization gains don't guarantee general retrieval improvements. If your use case involves fast-moving developer questions, evaluate carefully before adopting.
# amazon-sagemaker# amazon-sagemaker-ai# amazon-sagemaker-studio# artificial-intelligence# performance
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article