
Efficient Code Edits with Agents
This post shows a new technique for editing code files when using coding assistants for better token efficiency and cost.
The issue with agents editing documents
The cost to generate with LLMs is dominated by the output tokens used. The price of a million output tokens can be an order of magnitude larger than the input token price. At the time of this post, for example, GPT-5.6 Sol has an output to input token price ratio of 6 (output tokens are 6x the price of the input tokens). This difference presents an opportunity.
Naive implementations of code/file editing with LLMs requires the model to generate the document with any changes, or generate the changes and ask the user to implement them. Wouldn’t it be wonderful if we could only edit the pieces of a file that need to be edited?
An approach that currently exists is a search and replace tool. The user asks the model to make an edit to a file, and the model returns the substring in the file that will be replaced, and then the text that should replace it.
<<<<<<< SEARCH
def calculate_total(items):
total = 0
for item in items:
total += item.price
return total
=======
def calculate_total(items):
“”“Calculate the sum of all item prices.”“”
total = sum(item.price for item in items)
return total
>>>>>>> REPLACEThis approach has a number of limitations, we are doing a string search/replace (ambiguity between the same text), just knowing the edits made, doesn’t allow us to reverse the operation when ambiguity is present (changing ‘r‘ to ‘i‘, doesn’t allow us to then do the reverse of the operation without switching ‘i’ characters that existed previously), and chiefly, to make an edit, the model has to generate the entire sting that it’s replacing. There has to be a better way. I want a structured way to make edits to a file, and I want the model (or agent) to be able to make multiple edits at the same time. Finally, for the observability-minded folks, I want to be able to rewind any file changes as needed.
An idea for better editing
The genesis of this idea comes from how text editors make edits to a document internally. Instead of doing a string replace, let’s add some structure to our documents and perform edits against that structure (like an index or “tokenization” of our document).
Now, we have some hurdles... LLMs don’t (natively) distinguish between documents, instructions, tools, anything really (with small caveats). But if we’re going to make edits to a document, we need a way to separate the document from general context. There are a number of options for how we can add structure to a document. We can view a document as a list of characters, words, sentences, lines, pages, etc. The “best” choice here will depend on the use-case, but when we consider the popularity of coding assistants, seems like a good middle ground is a line (I.e. instead of viewing a document as unstructured text, we will convert any document into a list of lines). To implement this (since again, LLMs don’t distinguish), we will add lines directly to the document, and this is fed into the model.
import logging 0: import logging
import os 1: import os
2:
DEFAULT = Decimal("2") 3: DEFAULT = Decimal("2")
4:
class Reconciler: 5: class Reconciler:
def __init__(self): --> 6: def __init__(self):
self.tolerance = os.getenv("TOL", DEFAULT) 7: self.tolerance = os.getenv("TOL", DEFAULT)
8:
def run(self, movements, publisher): 9: def run(self, movements, publisher):
for movement in movements: 10: for movement in movements:
publisher.publish(movement) 11: publisher.publish(movement)
return len(movements) 12: return len(movements)Additionally, we need a way to actually edit the document. If we have a number of edits we want to make we can view the problem as denoting the edits to go from document → document*. And we can do this using 3 operations [insert, replace, delete]. Since we have structured our document into lines, we just need the model to designate the line range to perform the edit, the specific operation, and the text to be replaced. This actually works pretty well as a tool we can give to our model (or agent).
Example code file structure
0: import logging
1: import os
2:
3: DEFAULT = Decimal("2")
4:
5: class Reconciler:
6: def __init__(self):
7: self.tolerance = os.getenv("TOL", DEFAULT)
8:
9: def run(self, movements, publisher):
10: for movement in movements:
11: publisher.publish(movement)
12: return len(movements)Example code edit
{
"edits": [
{
"range": "1,2",
"lines": []
},
{
"range": "6,8",
"lines": [
" def __init__(self, tolerance: Decimal = DEFAULT):",
" self.tolerance = tolerance"
]
},
{
"range": "10,12",
"lines": [
" publisher.publish_batch(movements)"
]
}
]
}So what does this get us?
We should see a number of benefits with this approach. If we think about the minimum amount of information we have to generate to perform edits of any document, we need to know:
- Where we start the edits
- Where we end the edits
- And the actual text (the edit)
Our approach (Line operations) has these components. So can we say it’s the “best?” Well not quite. Remember we decided to structure our code into lines. So if every edit needed to change things at the scale of a line, then we might be able to start talking about “best”, but if we, for example, only wanted to change a word, our current approach would still have the LLM generate an entire line (doesn’t seem optimal to me). But we do have a number of benefits here. We don’t have any ambiguous edits (since we have the line or index). We can easily reverse any operation/edit we make. And we don’t have to generate the line(s) we are replacing.
So let’s see it in action…
Let’s evaluate
I have a few evaluations that I have performed. It sometimes gets a bit tricky, since I want to evaluate both performance (speed and token efficiency) and correctness (doesn’t matter if it’s the fastest method ever if it’s wrong). I’m not going to do much with correctness in this post since I wanted to try it out on some realistic examples (but maybe another post).
So I have a number of code files (they were generated because I couldn’t find a good dataset) of about 100 lines long, and I have a number of edits that I will ask an LLM to make (20). I’ll do this for Python and Typescript (apparently the only languages that matter anymore… aside from Rust). And I will run this evaluation as passing the code file to a model as well as an edit command (e.g. “Add FX conversion so cross-currency movements compare in USD.“). I will test 3 approaches, an LLM without any edit tool, an LLM with the search and replace tool, and an LLM with the line operations tool. So across 10 Python/Typescript files and 20 edits for each, and 2 trials each, we’ll record the latency and tokens used.
Example line operations tool spec
"toolSpec": {
"name": "line_operations",
"description": """Perform multiple line-based file edits (insert, replace, delete) in a single call. Line indices are 0-based; ranges are [start, end) — inclusive of start, exclusive of end.
insert : start == end, lines non-empty. Existing lines at/after start shift down.\nreplace: start < end, lines non-empty.\ndelete : start < end, lines == [].
\n\nExample:
file_edits([\n {"range": "2,2", "lines": ["from fastapi import FastAPI", "", "print(torch.__version__)"]},\n {"range": "3,4", "lines": []},\n {"range": "5,6", "lines": ["model.load_state_dict(torch.load('/opt/ml/model/weights.pt'))"]}\n ])""",
"inputSchema": {
"json": {
"type": "object",
"properties": {
"edits": {
"type": "array",
"description": "List of edit operations to apply.",
"items": {
"type": "object",
"properties": {
"range": {
"type": "string",
"description": "Start line index (inclusive) and the End line index (exclusive). Equal to start for an insert (comma separated). All edits, using ranges from the original document"
},
"lines": {
"type": "array",
"items": { "type": "string" },
"description": "Replacement/inserted lines without trailing newlines. Empty list means delete."
}
},
"required": ["range", "lines"]
}
}
},
"required": ["edits"]
}
}
}Note: Since our Line operations method requires the line numbers, we will keep those, but not include them for the other two methods. There is a small input token increase when adding the lines, and I want to remove that as a potential biasing point from the other approaches.
Full results




Line operations vs. search and replace
We can look at the two file edit methods specifically. In this case we can compare the difference in output tokens used. So for the same code file, and the same code change instructions. This direct comparison shows the “distribution” of tests where the line operations method produced fewer/more output tokens compared to search and replace.

In summary, across 400 unique edit cases, line operations reduced the median output tokens by 86%, and the median latency by 71% compared with full-file generation. Compared with search and replace specifically, line operations used 50% fewer output tokens, and responded 32% faster at the median.
In closing
Well, there we have it. A neat little method for LLMs/agents editing files that gives us token efficiencies of around 71% over the naive approach, and even 50% over the search and replace editing method. I didn’t give much of the cost difference between the methods. This is mostly because it depends on the model used and it’s pricing. I have been thinking about this off and on for more than a little while. So I’m glad to finally be able to remove it from my memory (until I have some more inspiration).
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article