What's New

Amazon SageMaker AI now offers multi-turn reinforcement learning (RL), a fully serverless model customization technique that fine-tunes foundation models on multi-step, agentic tasks by rewarding the full sequence of decisions an agent makes across a task. Unlike single-turn RL approaches, this capability trains models against a live agent environment, enabling smaller, lower-cost models to match or exceed the task accuracy of larger general-purpose models on targeted workloads. The feature is available today through SageMaker Studio and the SageMaker Python SDK, with no infrastructure to provision or manage.

How It Works

• Agent Environment Integration: You connect your agent environment to SageMaker AI — either via Amazon Bedrock AgentCore Runtime for fully managed hosting, or on Amazon EKS, Amazon EC2, AWS Fargate, or any custom infrastructure using your framework of choice.
• Full Training Loop Management: SageMaker AI orchestrates the entire training pipeline, including rollout orchestration, trajectory collection across multi-turn interactions, model training, and checkpoint management — eliminating the need to build custom RL infrastructure.
• Sequence-Level Reward Signal: Rather than grading individual responses, the system rewards the model based on the cumulative outcome of the full action sequence (e.g., tool calls, code execution, web searches), teaching the model which earlier decisions actually drove success.
• Built-in Observability with MLflow: Integrated MLflow tracking allows you to inspect agent trajectories, per-step rewards, and execution traces throughout training, providing full visibility into how the model is learning.
• Evaluation Jobs: Dedicated evaluation jobs compute reward, pass@k, and trajectory metrics, enabling rigorous benchmarking of a fine-tuned model before deployment to a SageMaker endpoint or Amazon Bedrock.
• Serverless, Token-Based Pricing: The capability runs fully serverless — you pay only for tokens processed during training, with no GPU instances to provision, configure, or manage.

Why It's Important

• Democratizes Agentic Model Training: Building multi-turn RL infrastructure from scratch typically takes weeks of custom engineering; SageMaker abstracts this entirely, making agentic fine-tuning accessible to teams without deep RL infrastructure expertise.
• Cost Optimization Through Model Specialization: By training smaller models to match the task accuracy of larger general-purpose models, organizations can significantly reduce inference costs for high-volume agentic workloads without sacrificing performance.
• Addresses a Critical Gap in Agent Reliability: Agents that complete multi-step tasks (e.g., customer support, code generation, data analysis) fail when models optimize for single-turn quality; multi-turn RL directly trains for end-to-end task completion, improving real-world reliability.
• Serverless Economics Lower the Barrier: Token-based pricing means teams can experiment with RL fine-tuning without committing to expensive reserved compute, making iterative experimentation financially viable.
• Tight Ecosystem Integration: Native connectivity to Bedrock AgentCore Runtime, MLflow, and SageMaker endpoints means fine-tuned models slot directly into existing AWS-based agent architectures with minimal friction.

How It's Different

• Multi-Turn vs. Single-Turn RL: Traditional RLHF/RLAIF assigns a reward to a single model output; multi-turn RL rewards the entire trajectory of decisions across many steps, training the model to optimize for cumulative task outcomes rather than isolated response quality.
• Serverless vs. Self-Managed Training Infrastructure: Competing approaches (e.g., running GRPO or PPO on self-managed GPU clusters) require weeks of infrastructure setup; SageMaker Multi-turn RL is fully managed and serverless, with no cluster provisioning required.
• Live Environment Rollouts vs. Static Datasets: The model trains against a real or simulated agent environment in real time, rather than learning from a pre-collected offline dataset, enabling more accurate credit assignment across complex action sequences.
• Integrated Evaluation Pipeline: Unlike generic RL frameworks, SageMaker includes built-in evaluation jobs with pass@k and trajectory metrics specifically designed for agentic task benchmarking, not just loss curves.
• Flexible Agent Hosting: The training loop is decoupled from agent hosting — you can run your agent on Bedrock AgentCore, EKS, EC2, Fargate, or any external infrastructure, whereas most managed RL offerings lock you into a specific runtime.

When to Prefer It

• Specialized Agentic Workflows: When you have a well-defined multi-step task (e.g., IT helpdesk automation, financial data retrieval and analysis, or software debugging pipelines) and want a smaller, cheaper model to perform it reliably at scale.
• Cost-Sensitive Production Deployments: When inference costs for a large general-purpose model are prohibitive and you need a smaller model fine-tuned to match its accuracy on your specific task.
• Teams Without RL Infrastructure Expertise: When your team lacks the experience or bandwidth to build and maintain custom PPO/GRPO training loops, rollout orchestration, and reward shaping pipelines.
• Iterative Agent Development: When you are actively developing an agent and want to continuously improve model behavior based on end-to-end task success signals rather than human annotation of individual responses.
• Existing AWS Agent Architectures: When your agent already runs on Bedrock AgentCore Runtime, EKS, or Fargate, and you want a training loop that integrates natively without re-architecting your stack.
• Benchmarking Before Production Promotion: When you need rigorous, quantitative evaluation (pass@k, trajectory metrics) of a fine-tuned model before promoting it to a production SageMaker or Bedrock endpoint.

Availability

• General Availability: Multi-turn RL is generally available as of June 3, 2026, through SageMaker Studio and the SageMaker Python SDK.
• Supported Regions: Available in us-west-2 (Oregon) and us-east-1 (N. Virginia); us-west-2 has broader model support.
• Supported Models (us-west-2): Qwen 3.6 27B, Nova Lite 2.0, GPT-OSS-20B, and Gemma 31B.
• Supported Models (us-east-1): Nova Lite 2.0 and GPT-OSS-20B only.
• Pricing Model: Fully serverless; you pay per token processed during training — no GPU instances to provision or pay for on an hourly basis.
• Access Methods: Available via SageMaker Studio UI and the SageMaker Python SDK as part of the broader SageMaker AI model customization suite.
• Documentation: Full details available at the [Amazon SageMaker AI Multi-turn RL documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/model-customize-mtrl.html).

Related Resources

• https://docs.aws.amazon.com/sagemaker/latest/dg/model-customize-mtrl.html