Fine-tune Small Language Models for Production-Grade Tool Calling with GRPO using Hugging Face TRL on Amazon SageMaker AI
Fine-tune Qwen3 (1.7B & 4B) with GRPO and verifiable rewards on SageMaker AI to build production-grade tool-calling agents. Using automated reward functions instead of labeled data, RL-tuned sub-4B models hit 0.96 exact match and 0.95 schema accuracy on financial APIs, outperforming GPT-OSS-120B and matching models 10–70× their size. Prototype with LLMs, deploy fine-tuned SLMs for enterprise-scale agentic workflows at a fraction of the latency and cost.
Authors: Pranav Murthy (AWS), Quentin Gallouédec (Hugging Face), Ben Burtenshaw (Hugging Face) Simon Pagezy (Hugging Face)
Introduction
Artificial Intelligent (AI) agents are reshaping how organizations execute work moving from rigid, scripted automation to intelligent systems that orchestrate multi-step workflows autonomously. From processing customer refunds to triaging support tickets, these agents assess context, break down tasks, and coordinate the tools needed to complete end-to-end workflows without human intervention.
This shift represents a qualitative change in what automation can accomplish. Knowledge workers routinely navigate between multiple different web applications during standard workflows, constantly switching contexts and manually transferring information between systems. AI agents eliminate this friction by orchestrating the entire process end-to-end, invisibly bridging the gaps between systems and handoffs that once required human effort. AI agents extend foundation models to understand user requests, break down tasks into smaller steps, collect additional information through natural conversation, take actions by making API calls to company systems, and augment performance by querying data sources, all without human intervention once initiated.
Gartner's research predicts that 15% of day-to-day work decisions will be made autonomously by agentic AI by 2028, and McKinsey finds that AI high performers are already realizing measurable gains in productivity, cost reduction, and revenue growth. But this value proposition rests entirely on a technical foundation that's often hard to get right: tool calling accuracy.
In this article, we demonstrate how to fine-tune small language models to power production-quality agents on Amazon SageMaker AI . SageMaker AI is a fully managed machine learning service that enables you to fine-tune models at any scale without managing infrastructure. Whether you prefer low-code interface or bring-your-own-script approaches, SageMaker AI provides complete flexibility, from pre-configured Deep Learning AMIs with optimized frameworks to full control over containers and training logic. The service automatically handles compute provisioning, distributed training orchestration, security, encryption, and resource cleanup, allowing data scientists to focus on model development rather than infrastructure management.
In this article, we focus specifically on Group Relative Policy Optimization (GRPO) , a reinforcement learning technique that trains models by scoring groups of candidate responses against each other, rewarding better outputs and penalizing weaker ones, to improve tool calling accuracy in agentic small language models (SLMs). GRPO can be effective for tool calling tasks when combined with Reinforcement Learning with Verifiable Rewards (RLVR). This combination uses automated verification of tool execution correctness rather than human feedback to provide clear training signals. Our approach shows that fine-tuned Small Language Models (SLMs) can match or exceed the tool calling performance of leading Large Language Models (LLMs) after just a few epochs of training on your organization's specific tools and APIs. This enables organizations to deploy agentic workflows that executes much faster and at a fraction of the cost compared to LLM-powered systems, while maintaining production-grade accuracy.
To validate this approach, we selected the smaller variants Qwen3 family of models (1.7B and 4B parameters) as our baseline and fine-tuned them using GRPO with RLVR on a representative set of tool calling tasks. We then benchmarked their performance against tool calling-optimized frontier LLMs like OpenAI GPT-OSS-120B, GPT-OSS-20B and Qwen3-next-80b-A3B. The results demonstrate that with domain-specific fine-tuning, even sub-2B and sub-8B parameter models can compete with frontier LLMs on complex agentic workflows, while maintaining the speed and cost advantages that make them viable for enterprise-scale deployment.
Why Tool Calling Accuracy Determines Return on Investment
When an agent automates a workflow, it must execute a precise sequence of tool calls where a single misstep breaks the entire chain and cascades into stalled onboarding, accounting discrepancies, or extended system downtime. In production environments processing thousands of workflows daily, these failures compound fast: failed workflows demand manual intervention, error rates drive up support costs, and incomplete tasks erode the efficiency gains that justified the investment. This is why tool calling accuracy is the foundation of agent ROI, where every percentage point improvement translates directly to fewer failed workflows, lower operational overhead, and greater confidence deploying agents at scale.
SLMs vs. LLMs for Agentic AI use cases
LLMs like Claude, GPT, and Nova deliver strong out-of-the-box tool calling capabilities thanks to broad training across diverse datasets, making them reliable for experimentation. But their computational demands introduce latency and cost that become real barriers in production: a workflow processing thousands of daily transactions cannot absorb 2-3 second inference times across 5-10 tool calls per sequence. SLMs, typically under 10 billion parameters, are fundamentally faster and cheaper to run, but out-of-the-box they struggle with the nuanced reasoning that LLMs handle naturally.
The key insight is that production workflows are narrow, not broad. A payment agent always calls the same financial APIs. An HR onboarding agent always orchestrates the same provisioning tools. Fine-tuning exploits this by training SLMs on domain-specific tool calling examples, teaching them the exact schemas, parameter formats, and error handling logic their target workflow requires. The result is an SLM that matches or exceeds LLM accuracy on its specific domain, while running orders of magnitude faster at a fraction of the cost.
This suggests a practical deployment pattern: use LLMs during prototyping when requirements are fluid, then swap in fine-tuned SLMs for production when workflows stabilize and performance matters. For enterprises deploying agents at scale, that efficiency gain is what makes agent ROI real.
Qwen3 Family of Model
For this article, we use the following Qwen3 models for fine-tuning:
Qwen3 models represent the cutting edge of small language model architecture, offering strong performance-to-parameter ratios that make them ideal for fine-tuning experiments. The Qwen3-1.7B and Qwen3-4B models are specifically designed with strong instruction-following capabilities and native tool-calling support, providing a solid foundation for specialized task optimization. Their compact size enables rapid iteration during fine-tuning where training cycles that would take hours on larger models complete in minutes while still maintaining the architectural sophistication needed to learn complex tool calling patterns.
Prerequisites
To fine-tune function calling models on SageMaker AI, you must have the following prerequisites:
- An AWS account that will contain your AWS resources.
- An AWS Identity and Access Management (IAM) role to access SageMaker AI. To learn more about how IAM works with SageMaker AI, see AWS Identity and Access Management for Amazon SageMaker AI .
- You can run the notebook provided in this post from your preferred development environment, including interactive development environments (IDEs) such as Visual Studio Code, provided your AWS credentials are properly set up and configured to access your AWS account. To set up your local environment, refer to Configuring settings for the AWS CLI . Optionally, we recommend using Amazon SageMaker Studio for straightforward development process on SageMaker AI.
- If you'd like to track your experiments in SageMaker with MLFlow, follow the instructions in the SageMaker documentation.
- If you're following along with this post, we use SageMaker fully managed training jobs and a single ml.g6e.8xlarge instance for training. You will need access to these SageMaker compute instances to run the example notebook presented in this post. If you're unsure, you can review the AWS service quotas on the AWS Management Console :
- View the SageMaker 'ml.g6e.8xlarge for training job usage' quota in the service quotas console.
- If the "Applied account-level quota value" is 0, request an increase at account level of 1.
- Access to GitHub Repo
To understand the costs associated with SageMaker Studio notebooks and SageMaker fully managed training jobs, refer to the SageMaker AI Pricing page .
Group Relative Policy Optimization (GRPO) and Reinforcement Learning with Verifiable Rewards (RLVR) on Amazon SageMaker AI
Traditional supervised fine-tuning requires extensive labeled datasets with thousands or even tens-of-thousands of examples showing the exact responses you want your model to produce. For tool calling tasks, this means manually curating examples of correct API invocations, properly formatted parameters, and successful workflow executions, a process that's expensive, time-consuming, and doesn't scale well when your tools or APIs evolve. Reinforcement learning offers an alternative approach where models learn by trying different actions and receiving feedback on which actions work. Group Relative Policy Optimization (GRPO) combined with Reinforcement Learning with Verifiable Rewards (RLVR) makes this practical for tool calling by replacing human labeling with automated verification.
GRPO leverages reinforcement learning with verifiable rewards using automated tool execution feedback to guide model optimization. This approach allows you to rapidly fine-tune models on your organization's actual tools and APIs, achieving production-grade accuracy without the complexity and cost of generating thousands of human-annotated examples. In this section, we provide a practical implementation guide using SageMaker AI that you can adapt to your own tool calling use cases.
What is GRPO?
Credit: Hugging Face GRPO Trainer
GRPO is a reinforcement learning algorithm that optimizes model behavior by comparing multiple candidate responses to the same prompt. Here's how it works: for each training prompt, the model generates several candidate completions (typically 4-8 responses). Each response receives a reward score based on its quality. GRPO then computes advantages relative to the group's performance where-in responses that perform better than the group average receive positive advantages, while below-average responses receive negative advantages. The model's parameters are updated to increase the probability of generating high-advantage responses.
The key innovation is that GRPO doesn't require a separate reward model like traditional reinforcement learning from human feedback (RLHF). In RLHF, you first collect human feedback, train a reward model to predict that feedback, and then use the reward model during training. GRPO skips the reward model entirely by using group statistics as the baseline. This makes training faster and simpler while maintaining effectiveness.
The advantage calculation in GRPO works as follows: for each response, the algorithm computes,
A_i = (R(s_i, a_i) - mean(R_group))/std(R_group)where
R(s_i, a_i) is the reward for that specific response and R_group correspond to all candidate responses generated for the same prompt. This relative scoring ensures that the model learns from the distribution of outcomes within each prompt's response group, encouraging it to favor strategies that consistently outperform the average for similar inputs. For example, if a prompt generates 8 candidate tool calls and 5 succeed (reward = 1.0) while 3 fail (reward = 0.0), the advantage would be 0.77 for successful attempts and -1.28 for failed attempts.To illustrate further,
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# Given: 8 candidate responses for the same prompt
# - 5 successful attempts (reward = 1.0)
# - 3 failed attempts (reward = 0.0)
# Step 1: Calculate mean reward
rewards = [1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0]
mean_reward = (5 × 1.0 + 3 × 0.0) / 8
mean_reward = 5.0 / 8 = 0.625
# Step 2: Calculate standard deviation
# Calculate squared differences from mean
variance_successful = (1.0 - 0.625)² = 0.140625
variance_failed = (0.0 - 0.625)² = 0.390625
# Calculate overall variance
variance = [(5 × 0.140625) + (3 × 0.390625)] / 8
variance = [0.703125 + 1.171875] / 8
variance = 1.875 / 8 = 0.234375
# Calculate standard deviation
std_dev = √0.234375 ≈ 0.484
# Step 3: Calculate advantages using GRPO formula
# A_i = (R(s_i, a_i) - mean(R_group)) / std(R_group)
# For successful attempts (reward = 1.0):
A_success = (1.0 - 0.625) / 0.484
A_success = 0.375 / 0.484 ≈ 0.77
# For failed attempts (reward = 0.0):
A_failure = (0.0 - 0.625) / 0.484
A_failure = -0.625 / 0.484 ≈ -1.28
# Interpretation:
# - Successful attempts: +0.77 (0.77 std devs above average)
# → Model increases probability of similar tool calls
# - Failed attempts: -1.28 (1.28 std devs below average)
# → Model decreases probability of similar tool calls
What is RLVR?
Reinforcement Learning with Verifiable Rewards (RLVR) provides the crucial missing piece: how do you compute reward scores without human labeling? The answer is RLVR. RLVR uses programmatic verification as the reward functions that can automatically check whether an output is correct.
For tool calling tasks, verification is straightforward. You can programmatically check whether the model invoked the correct tool, passed properly formatted parameters, and received a successful response. Unlike subjective tasks like creative writing or brand voice where quality depends on human judgment, tool calling has objective correctness criteria. Either the API call succeeds with the right parameters, or it doesn't.
This makes tool calling an ideal application for RLVR. You write a reward function that examines the model's tool calling behavior and returns a score (ex: 0.0 for incorrect and 1.0 for correct). The model generates multiple responses, receives automated feedback on each response, and learns to favor the patterns that lead to successful tool invocations.
How GRPO and RLVR Work Together
When you combine GRPO with RLVR for tool calling, the training loop looks like this:
- Sample multiple responses: For each training prompt (like "Calculate the mortgage affordability for a user with $80,000 income"), the model generates 'n' (between 4-16) candidate tool calling sequences.
- Verify each response: Your reward function checks each candidate. Did it call the right tool? Were the parameters properly formatted? Did the tool execution succeed? The reward function returns 1.0 for successful tool calls and 0.0 for failures. Optionally, it can also return a partial reward (ex: 0.1) if you choose to reward a model partial reward for certain types of outputs.
- Compute group advantages: GRPO calculates the baseline performance for this group of responses. If 5 out of 8 attempts succeeded, responses that succeeded get positive advantages while failed attempts get negative advantages.
- Update the model: The policy gradient increases the probability of generating successful tool calling patterns and decreases the probability of patterns that led to failures.
This process repeats across thousands of training examples, teaching the model to reliably invoke tools with correct parameters.
Sample Case Study: Simulating Real-World Tool Calling with Financial APIs
In this implementation assume you have a set of APIs or tools that you want to fine-tune an SLM to use accurately. To demonstrate the approach without requiring external dependencies, we created eight self-contained functions that simulate the structure and complexity of real-world financial APIs. These functions execute all computation internally and return either string or JSON responses, mimicking production API behavior.
The functions below represent common financial workflows: from portfolio allocation to mortgage calculations and intentionally use diverse parameter types (str, bool, int, float) and varying argument counts to test the model's ability to handle realistic API complexity. While these are simplified implementations, they capture the essential characteristics of production APIs: multiple required parameters, type constraints, conditional logic, and structured outputs. These include:
calculate_portfolio_allocation- total investment, risk tolerance (conservative to aggressive), time horizon, current/retirement age, income need %, existing allocations dict, ESG preference, tax-loss harvesting, rebalancing frequency, inflation protection, liquidity requirementcalculate_mortgage_affordability- annual income, monthly debts, down payment, credit score (FICO), loan type (conventional/fha/va/jumbo), property state, property tax rate, HOA fees, homeowners insurance, PMI flag, interest rate override, loan term (15/20/30 yr)optimize_tax_strategy- gross W-2 income, filing status (single/married/head), state, retirement contributions dict (401k/IRA/HSA), short/long-term capital gains, qualified/ordinary dividends, itemized deductions, dependents, self-employment income, rental incomecalculate_retirement_readiness- current/retirement age, current savings, annual contribution, employer match %, expected return, inflation rate, desired retirement income, Social Security estimate, pension income, annual healthcare cost, life expectancyanalyze_debt_payoff_strategy- debts list (balance, rate, minimum, type), monthly payment budget, strategy (avalanche/snowball/hybrid), extra payment allocation method, interest rate threshold, consolidation availability + rate, balance transfer fee, credit score impact weightcalculate_insurance_needs- age, annual income, dependents, mortgage + other debt balances, existing coverage dict (life/disability/LTC), health status, occupation risk (low/medium/high), years to retirement, spouse income, college funding need, final expensescalculate_education_funding- child's current age, college start age, years of college (2-6), current annual cost, education inflation rate, current 529 savings, monthly contribution, expected return, financial aid expected, student contribution %, state residency, school type (public in/out-of-state/private/community)calculate_currency_exchange_arbitrage- base/target currency (ISO codes), amount, exchange method (bank/forex broker/crypto/wire), transfer fee % + fixed, intermediate currency for triangular arbitrage, spot rate override, forward contract months (0-24), hedging strategy (none/forward/option/collar), tax reporting flag
Note: You can replace these sample APIs with your own domain-specific functions. For production APIs that make network calls or database queries, create mock versions for training to avoid latency and ensure faster convergence.
The full code for all these APIs can be found in this Hugging Face dataset here .
Dataset Preparation for GRPO RLVR Fine-Tuning
To begin fine-tuning, you need a domain-specific dataset containing prompts and their corresponding expected outputs. Your prompts should reflect the natural variation you expect in production, including:
- Verbosity: Range from terse requests ("Calculate mortgage for $80k income") to detailed descriptions with all parameters explicitly stated
- Specificity: Mix fully specified requests with underspecified queries that require the model to infer reasonable defaults
- Phrasing style: Combine formal business language, casual conversational tone, and technical terminology
- Parameter ordering: Vary whether parameters appear in API order or arbitrary sequences
- Implicit vs explicit intent: Some prompts clearly state which tool to use; others require inferring the appropriate tool from context
This diversity teaches the model to recognize intent and format tool calls correctly regardless of how users phrase requests. Training exclusively on short prompts produces models that confidently impute missing parameters, while detailed prompts teach accurate parameter extraction. A balanced dataset covering both extremes handles the full spectrum of production inputs. For instance, each training example consists of a prompt and an answer key. Here's an example from the financial tool calling dataset:
1
2
3
4
5
6
7
8
9
10
11
12
13
{
"prompt": [
{
"content": "You are a financial planning assistant with tools for portfolio allocation, mortgage affordability, tax optimization, retirement readiness, debt payoff strategies, insurance needs, education funding, and currency exchange. Analyze user requests and call the appropriate tool with all required parameters extracted from their query. Return concise answers with key metrics. Do not ask for clarification - use reasonable defaults if needed.",
"role": "system"
},
{
"content": "Single income $78k, $720 monthly debts, $30k down payment, credit score 690, FHA loan in Indiana with 0.85% property tax, $140 HOA, $105 insurance, PMI required, 30-year term.",
"role": "user"
}
],
"answer": "Max Price: $384,111, Monthly Payment: $2,950 (P&I: $2,285, Tax: $272, Ins: $105, HOA: $140, PMI: $148), DTI: 56.5%"
}Important: The answer field serves exclusively for reward computation, not direct model training. During training, the model receives only the prompt, generates a tool call with arguments, executes the function, and the output is compared against the answer key to compute the reward score. GRPO uses this reward to adjust model parameters. The model learns through reinforcement feedback, not by memorizing outputs. You have complete flexibility in reward design: exact matching, numeric tolerance, partial credit, or multi-field validation depending on your use case.
There are two primary approaches to building your training dataset:
- Real User Prompts (Recommended): If your organization already runs agentic AI workflows using frontier models to call internal tools, collect the user input prompts and corresponding tool calls from production logs. These natural human inputs represent authentic usage patterns and provide the highest-quality training signal. We recommend starting with at least 250 samples for initial fine-tuning, with 1,000+ samples being ideal. Dataset requirements vary based on prompt quality and tool complexity.
- Synthetic Data Generation: For organizations without existing production data, synthetic generation offers a practical starting point. Use a frontier model as a "teacher" to generate realistic prompts by providing context about your available tools, workflow requirements, and desired input variations. For this article, we used Kiro Agentic AI IDE to generate 1,000+ synthetic prompts by referencing our tool schemas and requesting diverse, human-like queries.
Important: Always review the model's license and terms of service regarding synthetic data generation and its use for fine-tuning before proceeding. Not all model providers permit this use case.
To make this dataset readily accessible, we've published synthetic GRPO RLVR sample dataset to the Hugging Face Hub: synthetic-financial-tool-calling-grpo-rlvr-1k .
Fine-tuning Qwen3 on Amazon SageMaker AI using training jobs
The diagram above illustrates how Amazon SageMaker AI enables rapid iteration from experimentation to production deployment. Data scientists develop and test in Studio notebooks, launch managed SageMaker Training Jobs on GPU infrastructure, and deploy fine-tuned models to either SageMaker Endpoints or Amazon Bedrock all without managing servers or infrastructure.
Development in SageMaker Studio
The workflow begins in Amazon SageMaker Studio, where you'll work within a JupyterLab notebook in your assigned user profile. Start by creating a JupyterLab space with at least 15GB of storage (learn how to launch a Studio Space ). You can run the notebook on minimal compute like ml.t3.medium since it only orchestrates the training job rather than executing the training itself. Within the notebook, prepare your training and validation datasets and upload them to Amazon S3. The notebook also allows you to configure GRPO training parameter used during Qwen3 fine-tuning runtime.
Launching and Scaling Training Jobs
When you execute the training cell, SageMaker launches a headless, non-blocking training job on GPU-accelerated instances. The job pulls your selected Qwen3 base model from Hugging Face, retrieves your dataset from S3, and begins GRPO fine-tuning with RLVR using prebuilt PyTorch Deep Learning Images with optimized frameworks. Scaling is seamless: moving from single-GPU training (ex: ml.g6e.8xlarge) to multi-node distributed training (ex: 4 x ml.g6e.8xlarge) requires changing only the instance_count parameter. SageMaker handles infrastructure complexity such as instance provisioning, networking, security, encryption, and cleanup. During training, fully managed MLflow on Amazon SageMaker AI automatically tracks experiments, logging metrics and hyperparameters. All checkpoints, safetensors, and artifacts are written to your S3 bucket for versioning and traceability.
Deployment Options
Once training completes, your fine-tuned model weights reside in S3 and are ready for deployment via two paths:
- SageMaker Endpoints: Deploy for real-time, low-latency serving with auto-scaling
- Amazon Bedrock Custom Model Import (CMI): Access through Bedrock's managed API
End users interact with your fine-tuned models through agentic AI workflows that invoke either endpoint. The fine-tuned SLM provides fast, accurate, cost-effective inference compared to frontier models while maintaining data sovereignty and security within your VPC.
Recipe-Based Fine-Tuning Configuration
This implementation uses a recipe-style workflow where training parameters and hyperparameters are defined in a simple YAML file. This approach enables easy versioning of different configurations, rapid experimentation, and minimal code changes between runs.
Here's a sample recipe for fine-tuning Qwen3-1.7B:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
model_name_or_path: Qwen/Qwen3-1.7B
model_revision: main
dtype: bfloat16
trust_remote_code: true
attn_implementation: kernels-community/flash-attn2
output_dir: /opt/ml/output/Qwen/Qwen3-1.7B/grpo-rlvr/
dataset_id_or_path: /opt/ml/input/data/training/grpo_financial_train.jsonl
# Generation settings
num_generations: 8 # recommended
max_grpo_completion_length: 512
# Training settings
learning_rate: 1.0e-6
lr_scheduler_type: "constant"
warmup_ratio: 0.1
num_train_epochs: 15
per_device_train_batch_size: 4
gradient_accumulation_steps: 2
gradient_checkpointing: true
# Checkpointing
save_strategy: "steps"
save_steps: 25
save_total_limit: 10
# Logging
logging_steps: 1
report_to:
- "mlflow" # logs to sagemaker mlflow
seed: 42
Launching Training on SageMaker
SageMaker makes it straightforward to reference your recipe file, supply your tools and reward functions, and launch training jobs. The code below configures a training job by specifying the recipe file, tool functions, reward logic, compute resources, output locations and then launches it on managed GPU infrastructure:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
# Define training arguments
args = [
"--config",
"hf_recipes/Qwen/Qwen3-1.7B--grpo.yaml",
"--tools_script",
"tools_funcs/financial_tools_complex.py",
"--reward_fn",
"rewards/financial_tools_reward.py",
]
# Configure compute resources
compute_configs = Compute(
instance_type="ml.g6e.8xlarge ", # or ml.g6e.8xlarge
instance_count=1,
keep_alive_period_in_seconds=1800,
volume_size_in_gb=450
)
# Set up source code and output paths
source_code = SourceCode(
source_dir="./sagemaker_code",
command=f"bash sm_accelerate_grpo_train.sh {' '.join(args)}",
)
base_job_name = f"{job_name}-finetune"
output_path = f"s3://{sess.default_bucket()}/{base_job_name}"
# Create the training job
model_trainer = ModelTrainer(
training_image="763104351884.dkr.ecr.{region}.amazonaws.com/pytorch-training:2.8.0-gpu-py312",
source_code=source_code,
base_job_name=base_job_name,
compute=compute_configs,
stopping_condition=StoppingCondition(max_runtime_in_seconds=18000),
output_data_config=OutputDataConfig(
s3_output_path=output_path,
),
checkpoint_config=CheckpointConfig(
s3_uri=os.path.join(
output_path,
"financial-api-for-tool-calling",
job_name,
"checkpoints"
),
local_path="/opt/ml/checkpoints"
),
role=role,
environment=training_env
)
Training Loop Mechanics
During training, the workflow operates as follows: the model receives a prompt as input and generates a tool call response. The TRL library parses the tool call's function name and arguments, invokes the actual function, and captures the response. This response is compared against the ground truth from the training data to compute the reward signal. GRPO uses this feedback to optimize the model's tool calling behavior, gradually teaching it to generalize correct tool invocations from input prompts.
All metrics are automatically logged to SageMaker MLflow if you have a tracking server configured, or stored locally for offline review. You can extend the training script to integrate with additional experiment tracking services like TensorBoard, Weights & Biases, or other experimentation tracking platforms as needed.
Evaluating Tool Calling Performance
Once training is complete its crucial to evaluate the model's performance on unseen samples to evaluate the a model's tool calling accuracy. In this section, we analyze the results of fine-tuning Qwen/Qwen3-1.7B and Qwen/Qwen3-4B on a synthetically generated financial tool calling dataset. While tool calling accuracy is often reported as a single aggregate metric, we decompose it into three distinct dimensions to isolate specific failure modes and measure improvements across different aspects of model performance:
- Response Validity (RV): Response Validity measures whether the model produces a usable, non-error response for a prompt with a valid ground-truth answer. This metric captures execution reliability rather than correctness, isolating failures caused by generation errors, tool invocation failures, or malformed outputs. Samples with invalid ground truth are excluded to avoid penalizing the model for annotation issues.
- Answer Exact Match (AEM): Answer Exact Match evaluates the strict correctness of the model's final answer by requiring an exact string match with the ground truth after normalization. Normalization removes superficial formatting differences (e.g., extra whitespace) while preserving semantic strictness. This metric reflects end-to-end task correctness and is commonly used in question answering and structured reasoning benchmarks.
- Tool Call Schema Match (TCSM): measures the correctness of the model's tool invocation by jointly evaluating tool selection and argument accuracy under a strict schema. Tool selection is scored as a binary match, while arguments are evaluated recursively against the ground-truth schema using exact matching (with controlled numeric tolerance). The metric emphasizes correct parameterization while still penalizing incorrect tool choice, producing a fine-grained score in 0 → 1.
Experimental Results
The table below details the hyperparameters and infrastructure used to fine-tune both Qwen3 models on SageMaker AI. All experiments were conducted on SageMaker AI's ml.g6e.8xlarge instances, which provide a single NVIDIA L40S GPU.
| Models | instance_type | epochs | num_generations | per_device_train_batch_size | gradient_accumulation_steps | max_completion_length | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3 1.7B | ml.g6e.8xlarge | 25 | 8 | 4 | 6 | 1024 |
| 2 | Qwen3 4B | ml.g6e.8xlarge | 15 | 8 | 2 | 4 | 1024 |
Results
In this section, we present the results from our fine-tuning experiments. Our results as presented in the table and chart below demonstrates substantial improvements across all evaluation metrics for both Qwen3 models. The fine-tuned Qwen3 1.7B and Qwen3 4B models achieve performance levels that are comparative or exceed significantly larger frontier models on this domain-specific financial tool calling tasks.
| # | Model | Response Validity | Answer Exact Match | Tool Call Schema Match |
|---|---|---|---|---|
| 1 | Qwen3-1.7B (base) | 0.78 | 0.62 | 0.90 |
| 2 | Qwen3-1.7B (rl-tuned) ** | 0.99 | 0.96 | 0.95 |
| 3 | Qwen3-4B (base) | 0.75 | 0.67 | 0.92 |
| 4 | Qwen3-4B (rl-tuned) ** | 0.98 | 0.96 | 0.95 |
| 5 | Qwen3-next-80b-a3b | 0.75 | 0.60 | 0.95 |
| 6 | GPT-OSS-20B | 0.94 | 0.71 | 0.83 |
| 7 | GPT-OSS-120B | 0.95 | 0.82 | 0.93 |
- GRPO fine-tuning delivers outsized gains on small models. The rl-tuned Qwen3-1.7B and 4B models achieve 0.99 and 0.98 response validity and 0.96 exact match on both - outperforming GPT-OSS-120B and matching or exceeding models 10-70x their size on every metric.
- Schema adherence is remarkably consistent post-tuning. All rl-tuned models hit 0.95 tool call schema match, on par with GPT-OSS-120B and ahead of GPT-OSS-20B, suggesting that structured financial tool-calling is a learnable skill that fine-tuning reliably instills regardless of model size.
- The answer exact match gap is where fine-tuning matters most. Base Qwen3 models score 0.62-0.67 on exact match, while their rl-tuned counterparts jump to 0.96 - a 43-55% relative improvement - and surpass all other models in the benchmark, including GPT-OSS-120B (0.82) and Qwen3-next-80b-a3b (0.60), demonstrating that precise numerical reasoning over complex financial APIs is highly sensitive to domain-specific training.
These results validate the core premise: with targeted fine-tuning using GRPO and verifiable rewards, sub-8B parameter models can achieve LLM-grade tool calling accuracy on domain-specific workflows, delivering the speed and cost advantages of SLMs without sacrificing the reliability needed for enterprise deployment.
Qwen3-1.7B (rl-tuned) achieves the highest response validity (0.99), outperforming GPT-OSS-120B by 4 points and all base models by over 20 points, likely attributable to its extended RL training duration relative to the 4B model.
Both rl-tuned Qwen3 models (1.7B and 4B) achieve the highest tool call schema match (0.95), demonstrating that GRPO fine-tuning effectively instills precise financial tool-calling structure regardless of model size.
The full code for evaluating GRPO + RLVR fine-tuned model can be found in this aws-samples/amazon-sagemaker-generativeai github repository.
Conclusion
The path to production-grade AI agents hinges on achieving reliable tool calling accuracy at enterprise scale. This article demonstrates that through Group Relative Policy Optimization and Reinforcement Learning with Verifiable Rewards, organizations can transform compact small language models into specialized agents that match frontier model performance. Our experimental results validate this conclusively: fine-tuned Qwen3 models at just 1.7B and 4B parameters achieved tool calling accuracy comparable to models 10 to 70x their size, with both reaching 0.99 response validity and 0.95 tool call schema match, placing them ahead of GPT-OSS-120B while retaining the speed and cost advantages inherent to small language models, making enterprise-scale deployment economically viable.
For organizations deploying AI agents, this approach offers a clear strategy: prototype with frontier LLMs to validate workflows, then fine-tune domain-specific SLMs for production deployment. This captures LLM flexibility during development while delivering the operational efficiency required at scale with agents that complete workflows order of magnitude faster, process requests at a fraction of the cost, and maintain the tool calling accuracy that determines whether automation delivers ROI. As AI agents move from experimentation to production, the ability to fine-tune specialized models for specific tool sets becomes a competitive advantage, enabling reliable automation across diverse workflows from financial planning to IT operations with confidence in execution, scalability, and measurable business value.
The fine-tuning workflow described in this article gives you full control over infrastructure and training configuration. For teams ready to move faster, Amazon SageMaker AI now offers serverless model customization that removes infrastructure management entirely, letting you fine-tune models like Qwen3 using RLVR with just a few clicks and evaluate against the base model, all without provisioning a single instance, get started here .
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article