AWS Builder Center

Builder Center

Textract Couldn't Read My Bilingual Warranty Cards. Geometry and Claude Haiku Fixed It

Textract Couldn't Read My Bilingual Warranty Cards. Geometry and Claude Haiku Fixed It

Textract's AnalyzeExpense mangled two-column warranty cards. I rebuilt the columns from bounding boxes with code, used Claude Haiku only for the leftovers, and measured about 1.4 cents per receipt.

I'm building Reclaima , an Android app that scans a receipt, tracks every item's warranty and return deadline, and generates a claim PDF when something breaks. The whole app depends on one step: reading the receipt correctly.
Amazon Textract's AnalyzeExpense API does most of that well. It returns the vendor, total, dates and line items as structured fields. Then I scanned a real bilingual warranty card, and the result looked like this:
1
Warranty Warranty does does not not cover cover for for physical damages
Here is why that happened, and the two-step fix that made it readable.

Why the text came out doubled

Many warranty cards are printed in two side-by-side columns: English on one side, another language on the other, saying the same thing. OCR reads a page line by line, left to right, across the whole width. So it reads a bit of the left column, then a bit of the right, then the next line of the left, and so on. The two columns get interleaved.
You get two symptoms:
  1. Duplicated words, where the same phrase appears in both columns.
  2. Orphan fragments: lonely lines like "the", "only" or "above" left over from a column break.

The obvious fix, and why I didn't start with it

The tempting answer in 2026 is "just send the whole image to an LLM." I didn't, for two reasons:
  • AnalyzeExpense already gives me structured, typed fields (total, date, line items) with confidence scores. Rebuilding that with prompts would be slower and harder to trust.
  • Most of the problem isn't language. It's geometry. The model shouldn't have to guess at a layout problem I can solve exactly.
So the fix has two steps: rebuild the layout with code first, then use an LLM only for what's left.

Step 1: Rebuild the columns from bounding boxes (free and deterministic)

Textract returns a BoundingBox for every LINE block. If a region has two columns, the left edges of its lines cluster into two groups with a clear gap between them. Find the largest gap, and if it's wide enough, split at it:
1
2
3
4
5
6
7
8
9
10
11
left_vals = sorted({round(ln["left"], 3) for ln in lines})
gaps = [(left_vals[i + 1] - left_vals[i], (left_vals[i] + left_vals[i + 1]) / 2)
for i in range(len(left_vals) - 1)]
max_gap, split_x = max(gaps, key=lambda g: g[0])

if max_gap >= 0.15: # 15% of page width -> two columns
left_col = sorted([l for l in lines if l["left"] < split_x], key=lambda l: l["top"])
right_col = sorted([l for l in lines if l["left"] >= split_x], key=lambda l: l["top"])
ordered = left_col + right_col # read each column top to bottom
else:
ordered = sorted(lines, key=lambda l: l["top"])
That one change turns "interleaved soup" into "left column, then right column." It costs nothing, needs no API call, and gives the same answer every time.

Step 2: Let Claude Haiku clean up the leftovers

Geometry can't fix everything. Some lines are still duplicated, fragments remain, and the non-English half still needs to be removed. For that I call Claude Haiku 4.5 on Amazon Bedrock with a narrow prompt:
1
2
3
4
5
6
SYMPTOM 1 - Word duplication: "Warranty Warranty does does not not cover cover"
must become: "Warranty does not cover"
SYMPTOM 2 - Orphan fragments: absorb into the neighbouring sentence or drop.
- Preserve ALL factual warranty details exactly: time periods, day counts,
coverage conditions, exclusions.
- Return ONLY the cleaned text. No explanation.
Three settings matter more than the prompt wording:
  • temperature: 0.1. This is a repair job, not creative writing.
  • A tight max_tokens. 512 for warranty notes, 50 for a store name. That caps cost and stops the model from rambling.
  • Fail open. If Bedrock throws an error, the code logs it and returns the original OCR text. A slow or failed LLM call never blocks a user from saving a receipt.
What the cleanup should produce, taken from the example in my prompt:
1
Warranty does not cover for physical damages

What one receipt costs

I took the unit prices from the AWS Price List API (aws pricing get-products, us-east-1) and the token counts from CloudWatch's AWS/Bedrock metrics for my real calls.
Unit prices (us-east-1, on-demand):
  • Textract AnalyzeExpense: $0.01 per page (first 1M pages a month)
  • Claude Haiku 4.5, via the us. cross-region profile: $1.10 per 1M input tokens, $5.50 per 1M output tokens
Tokens per call (14 real calls): an average of 365 input and 25 output. The output is tiny because most calls return one short field, like a store name or a phone number. Most of the input is my prompt's instructions, not the receipt text.
So one Claude call costs about:
1
365 × $1.10/1M + 25 × $5.50/1M ≈ $0.00054
The number of calls depends on the receipt, because the app only cleans fields that actually exist. A fully loaded receipt with a store name, phone, email, address, warranty notes and 3 products needs 8 calls:
StepCost
Textract AnalyzeExpense (1 page)$0.0100
Claude Haiku 4.5 (8 calls × $0.00054)$0.0043
Total per receipt≈ $0.014
Even in that worst case, about 70% of the cost is Textract, not the LLM. It would take roughly 18 Claude calls to cost as much as one Textract page. In a pipeline like this, the "AI" step is the cheap part.

What I'd change

I review my own code honestly, so here is what I'm fixing next:
  • One call per field is wasteful. Store name, phone, email, address, notes and every product name each get their own call, and each one repeats a few hundred tokens of instructions. One call that returns all fields as JSON would send the instructions once and cut latency.
  • The global. inference profile is about 9% cheaper. It costs $1.00 / $5.00 per 1M tokens instead of $1.10 / $5.50, and switching is just a different model ID.
  • Some of these calls don't need an LLM at all. Phone numbers and emails have strict formats. A regex is cheaper, instant, and can't hallucinate.
  • My prompts assume the US. The phone prompt defaults to +1 and the address prompt uses US formats, which breaks for receipts from anywhere else. My email example also teaches the model to merge parts of a domain (best . buy . com → bestbuy.com), which is risky for multi-part domains like .co.uk.

The lesson

Fix structure with code, and fix meaning with a model. Bounding boxes solved most of my problem for free. The LLM handles the small, messy remainder, under tight limits and with a fallback.

Your turn

Have you hit OCR problems with multi-column or multi-language documents? How did you handle them: geometry, a bigger model, or something else? Tell me in the comments.
Reclaima: https://reclaima.sujaya.dev/ · Code: https://github.com/sujayamindev/reclaima
# amazon-textract# amazon-bedrock# generative-ai# python# ai
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article