Are We Merging the Right Models?
Model merging combines separately fine-tuned expert LLMs into a single model that handles multiple tasks at once. Standard practice freezes each expert at its best validation checkpoint before merging, but we find this is often the wrong choice. The optimal training duration turns out to be *method-dependent*: simple averaging wants undertrained experts, while sparsification-based methods like TIES reach their best performance with deliberately overfitted ones.
Paper: Are We Merging the Right Models? Impact of Expert Training Duration on Model Merging for LLMs , accepted to the ICML 2026 Workshop on Weight Symmetries.
The problem with "best checkpoint" merging
Model merging has become a practical way to build multi-task models without the cost of joint training. You fine-tune separate experts on different domains, then combine their weights into a single model that keeps the inference cost of one. It avoids the latency of mixture-of-experts, the memory of ensembling, and the data-sharing headaches of co-training, which makes it especially attractive for the small open-weight LLMs that enterprises increasingly fine-tune in production.
A common assumption in model merging is that each expert should be trained to its optimal validation loss before merging. Benchmarks and method comparisons often fine-tune models with early stopping or a fixed budget, freeze each expert at its best individual checkpoint, and then merge. We decided to test that assumption.
A random forest analogy
Our starting intuition came from ensemble learning. Random forests deliberately grow each tree deep because the averaging step that follows cleans up the noise that individual deep trees introduce. The aggregation tolerates and even benefits from individually "overfit" learners.
So we asked: do some merging methods behave like that averaging step? If so, the training duration that is optimal for a single expert might be very different from the duration that is optimal once the expert is going to be merged.
What we did
We ran a systematic sweep across three dimensions:
- 3 model sizes: Qwen 3.5 at 0.8B, 2B, and 4B
- 5 domains, each with its own fine-tuning dataset: Math ( DART-Math ), Code ( Magicoder ), Instruction Following ( TULU-3 ), Multilingual ( Aya ), and Safety ( BeaverTails )
- 5 merging methods: Simple Averaging, Task Arithmetic, TIES, DARE+TIES, and Greedy Soup
For every domain and size, we first find T*, the validation-optimal checkpoint. Then we keep training well past that optimum, saving experts from 0.25× to 5× T* (heavily undertrained to extremely overfitted). We merge at each duration and measure how the merged model does.
The key result
The picture above tells the core story. The x-axis is training duration as a multiple of T* (the green region is undertrained, the red region is overfitted), and each line is a merging method.
The methods split into clearly different regimes:
- Simple Averaging peaks early in the undertrained region and then falls off a cliff, losing accuracy as experts start to overfit.
- TIES and DARE+TIES do the opposite. They reach their best performance past T*, in the overfitted region, beating their own T* score.
- Task Arithmetic and Greedy Soup sit in between, tolerating moderate overfitting but without a strong preference either way.
Strikingly, the sweet spot for sparsification methods drifts further right as models get larger: TIES on the 4B model peaks at 3–5× T*. The same pattern shows up in safety behavior, where TIES on 4B reaches 68.5% refusal on HarmBench at 3× T*, more than 2.5× higher than simple averaging can manage at any duration.
Why it happens
The mechanism comes down to how each method handles the noise that overfitting introduces. We use a bias-variance-covariance decomposition: as an expert trains past its optimum, its bias drops (it captures more task-specific signal) but its variance rises (experts diverge and pick up idiosyncratic, noisy parameters).
Simple averaging has no defense against this. It weights every parameter of every expert equally, so the extra variance flows straight into the merged model and degrades it. TIES and DARE, by contrast, include explicit interference-resolution steps: they trim small-magnitude updates, resolve sign conflicts, and stochastically prune deltas before averaging. That sparsification acts exactly like the aggregation step in a random forest, suppressing the noise that high-variance experts bring while keeping the low-bias signal. This is why overtraining helps these methods rather than hurting them.
The practical takeaway
The choice of training duration and the choice of merging method are not independent decisions, and treating them as such leaves performance on the table. If you are merging with a sparsification-based method like TIES or DARE+TIES, do not early-stop your experts at their validation optimum. Train them past it, to roughly 2–5× T*, and the larger your base model, the more aggressively you can overtrain.
The full paper digs into the per-domain breakdowns, the bias-variance-covariance decomposition, and a mode-connectivity analysis that explains why LoRA keeps even heavily overtrained experts mergeable. If you work on model merging or fine-tune small LLMs in production, give it a read and let us know what you think.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article