AWS Builder Center
The Regex That Ate My Weekend: Parsing Indian Government Documents with Amazon Textract

The Regex That Ate My Weekend: Parsing Indian Government Documents with Amazon Textract

By Tejas Shinde — Team NxtTech What a single misplaced asterisk taught me about OCR, regex, and the surprisingly hard problem of reading Indian identity documents.

The Bug That Shouldn't Have Existed
On day three of the AWS First Commit hackathon, our pipeline worked. Sort of. The user uploaded a rejection letter, Textract read it, and the app produced a diagnosis.
There was just one problem: the diagnosis was wrong.
Not obviously wrong. Subtly, insidiously wrong. Here's what our app output:
Applicant Name:
Ram Patil Father Name
And here's what the document actually said:
Applicant Name: Ram Patil
Father Name: Shyam Patil
Somewhere in my regex, the value for "Applicant Name" was bleeding into the next line and grabbing the label of the following field. The applicant was named "Ram Patil Father Name." His father had vanished entirely.
I had written the regex ninety minutes earlier. I had tested it. I was sure it worked.
It didn't.
This is the story of finding that bug, fixing it, and the deeper lessons I learned about parsing Indian documents with Amazon Textract.



Why Indian Documents Are Hard

Before I show the fix, I want to explain the problem, because it's not obvious to anyone who hasn't worked with Indian government paperwork.
A typical rejection letter from a state welfare department looks like this:
GOVERNMENT OF MAHARASHTRA
Department of Social Justice and Special Assistance
Mantralaya, Mumbai - 400 032

Ref. No.: SJSA/PMAY/2025/11876
Date: 12 September 2025

To,
Suraj Balasaheb Khanase
Ganjibhoyare, Parner Taluka
Ahilyanagar, Maharashtra - 414302

Subject: Rejection of Application under Pradhan Mantri Awas Yojana
Notice the problems:
  1. The applicant's name is not labeled. It appears in the address block. There's no "Applicant Name:" prefix. You have to infer it from position.
  1. Names have variable length. Some use one name, some use three. Some add a middle name (patronymic) that isn't consistent across documents.
  1. There are honorifics and abbreviations. "Shri," "Smt," "Mr.", "Late" all appear before names.
  1. Multiple date formats. "12 September 2025" and "12/09/2025" and "12-09-2025" are all valid.
  1. Devanagari, Latin, and mixed scripts. Aadhaar cards often show both. Regional documents mix them mid-line.
  1. The rejection reason is often in a box or table. Textract returns table content differently from inline text.
  1. No consistent field order. Every department has its own letterhead format.
When I started building the extraction Lambda, I assumed this was a solved problem. I was wrong. It's a solved problem in the general case — for standardized invoices, receipts, and English-language forms. It is emphatically not solved for Indian welfare documents.



My First Attempt: The Greedy Regex

Here's the naive pattern I wrote for extracting the applicant's name:
import re

def extract_applicant_name(text):
match = re.search(r'Applicant\s*Name\s*[:\-]\s*(.*)', text, re.IGNORECASE)
return match.group(1).strip() if match else None
The problem is  .* . In Python's  re  module,  .  matches any character except newline by default. But when you use  re.DOTALL  — or when the extracted text from Textract has unusual line endings —  .  matches across lines.
More subtly: even without  re.DOTALL , if the input text uses  \r\n  line endings (Windows), the  .  will match  \r  but stop at  \n . So the captured group includes a trailing carriage return, which then confuses the next regex.
My test input had  \n  line endings. Textract returned  \r\n . The regex worked in testing. It failed in production.
This is a classic class of bug: a bug that depends on the line ending convention of your input source. It's the kind of thing that works perfectly until it doesn't.



The Fix: Anchor Everything

Here's what I replaced it with:
import re

def extract_field(text, label_variants):
"""
Extract a labeled field from Indian government document text.
Handles:
- Multiple label variants (Applicant Name, Applicant's Name, Name of Applicant)
- Optional colon, hyphen, or no separator
- Windows (\r\n) and Unix (\n) line endings
- Multi-word values on a single line
"""
pattern = (
r'^\s*'
+ r'(?:' + '|'.join(label_variants) + r')'
+ r'\s*[:\-]?\s*'
+ r'([^\n\r]+)'
)
match = re.search(pattern, text, re.IGNORECASE | re.MULTILINE)
if not match:
return None
return match.group(1).strip()
Three critical changes:
1.  [^\n\r]+  instead of  .* 
The capture group explicitly excludes both  \n  and  \r . It cannot bleed into the next line, no matter what line-ending convention the input uses. This is the fix.
2.  re.MULTILINE 
The  ^  anchor now matches the start of any line, not just the start of the string. This lets the regex find the field label wherever it appears in the document.
3. Label variants as a list
Different documents write the same field differently. "Applicant Name," "Applicant's Name," "Name of Applicant," "Name of the Applicant." Passing a list of variants makes the extraction tolerant.
Using it looks like this:
applicant = extract_field(text, [
r'applicant\s*name',
r"applicant'?s\s*name",
r'name\s*of\s*applicant',
])

father = extract_field(text, [
r"father'?s\s*name",
r'father\s*name',
r"father'?s\s*name",
])
The bug disappeared.  Ram Patil  extracted cleanly.  Shyam Patil  extracted cleanly. The correction form generated correctly.



The Deeper Lesson: Test With Real Line Endings

The root cause wasn't the regex itself. It was that my test fixture didn't match the real input.
I tested with this:
TEST_TEXT = "Applicant Name: Ram Patil\nFather Name: Shyam Patil"
Textract returned this:
TEXTRACT_TEXT = "Applicant Name: Ram Patil\r\nFather Name: Shyam Patil\r\n"
The difference is invisible when you read the strings. It's obvious when you use  repr() :
>>> repr(TEST_TEXT)
"'Applicant Name: Ram Patil\\nFather Name: Shyam Patil'"

>>> repr(TEXTRACT_TEXT)
"'Applicant Name: Ram Patil\\r\\nFather Name: Shyam Patil\\r\\n'"
I now write every test fixture by copying  repr()  output from actual Textract responses. It's tedious. It catches bugs like this one every time.



How Amazon Textract Actually Works

For anyone who hasn't used Textract, here's what I learned building with it.
Textract's  detect_document_text  API returns a list of  Block  objects. Each block is one of three types:
  • PAGE — represents a page
  • LINE — a line of text
  • WORD — a single word
To reconstruct the document, you iterate over lines:
import boto3

textract = boto3.client('textract')

response = textract.detect_document_text(
Document={'S3Object': {'Bucket': bucket, 'Name': key}}
)

lines = [
block['Text']
for block in response['Blocks']
if block['BlockType'] == 'LINE'
]

full_text = '\n'.join(lines)
Textract joins lines with  \n  in the response array. But when you concatenate manually, you control the separator. If I had used  '\r\n'.join(lines)  or preserved the block-level structure, I would have caught the bug earlier.
For documents with tables (like the rejection reason box in our sample PMAY letter),  analyze_document  with the  TABLES  feature returns structured table data. We didn't need it for the demo — the reason appears in inline text in most letters we tested — but for production, it's the right tool.



What I Learned About Testing

Three weeks before this hackathon, I thought testing was about coverage percentage. Building Adhikar AI taught me it's about test input fidelity.
A 100% coverage test suite that uses synthetic input catches the bugs you already understand. A minimal test suite using real extracted data catches the bugs you didn't anticipate.
The four test fixtures I ended up building:
  1. Clean .txt document — the ideal case, all fields present, standard format
  1. Real Textract output from an image — with  \r\n  line endings and OCR artifacts
  1. Document with missing fields — no rejection reason, no date
  1. Document with Devanagari names —  सुरज खनसे  alongside  Suraj Khanase 
The .txt fixture caught the regex bug the moment I ran it against real data. The image fixture caught an unrelated issue where Textract occasionally reads  1  as  l  in reference numbers. The missing-field fixture taught me to return  None  cleanly instead of raising  KeyError . The Devanagari fixture taught me that our fuzzy matching could handle cross-script comparison for common names — an unexpected bonus that improved our matching accuracy.
Every one of those bugs would have shipped. Every one was caught because the test input was real.



What I Would Do Differently

1. Log every Textract response. In production, I would log the raw  Blocks  array for every extraction. When a document fails to parse, the response is the only way to know why. We added this halfway through day three and it immediately revealed a class of documents Textract was reading as images-only (returning zero text blocks).
2. Set a timeout and a fallback. Textract on a large or blurry image can take 10+ seconds. Our extract Lambda had a 15-second timeout. On a few documents, we hit it. Setting the timeout to 30 seconds and adding a fallback to  detect_document_text  (which is faster than  analyze_document ) made the Lambda resilient.
3. Sanitize input before parsing. Stripping BOM characters, normalizing line endings, and collapsing multiple whitespace characters before running any regex saved us hours of debugging. This should happen once, at the top of the Lambda, not scattered across the parsing functions.
4. Test with the actual IAM permissions. We spent 30 minutes debugging a Textract  AccessDeniedException  that was purely an IAM issue. The Lambda's role had  textract:DetectDocumentText  but not  textract:AnalyzeDocument . When we switched APIs for table extraction, we forgot the permission. Test in the exact permission environment you'll deploy in.



Why This Matters Beyond Our Project

India processes an estimated three billion document verifications per year across welfare schemes, financial services, and government services. Each one requires a machine or a human to read an Indian document and extract structured information from it.
For decades, the standard answer was "hire clerks." Today, it's increasingly "use OCR." But OCR alone is not enough. The hard work is in the parsing layer — the code that takes raw text and produces structured fields that downstream systems can act on.
That parsing layer is almost always brittle. It works for the documents the developer tested with and fails for everything else. The reason is simple: the input space is enormous and undocumented. There are no standards for Indian welfare letters. Every department invents its own format. Every state has its own conventions.
Building software that handles this well is not glamorous work. It's regex, edge cases, and testing against real data. But it's the work that determines whether a welfare delivery system works or silently fails.



The Fix, Summarized

If you're building document extraction for Indian government documents, this is the checklist I wish I'd had on day one:
import re

def normalize_document_text(text):
"""Call this once on every Textract response."""
text = text.replace('\r\n', '\n') # normalize line endings
text = text.replace('\r', '\n') # handle old Mac endings
text = re.sub(r'\ufeff', '', text) # strip BOM
text = re.sub(r'[ \t]+', ' ', text) # collapse horizontal whitespace
return text

def extract_field(text, label_variants):
"""Never use .* in a capture group."""
pattern = (
r'^\s*'
+ r'(?:' + '|'.join(label_variants) + r')'
+ r'\s*[:\-]?\s*'
+ r'([^\n\r]+)'
)
match = re.search(pattern, text, re.IGNORECASE | re.MULTILINE)
return match.group(1).strip() if match else None
Two functions. Ten lines of actual logic. That's what stood between our pipeline working and not working.



Try It

The extraction Lambda is at  backend/src/extract/handler.py . The sample documents are in  samples/ . The test fixtures — including a real Textract response with the  \r\n  line endings — are also there. Steal them. Improve them. Tell me what you find.


Team NxtTech — Suraj Khanase (Lead), Sumit Jagtap, Nandkishor Tamkhade, Tejas Shinde
Built at AWS First Commit — Bharat Builds Tour, Event 01, September 2026.
If you're working on Indian language document processing or civic tech, I'd love to compare notes. Reach me through the GitHub repo.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article