
From Scanned PDFs to Smart Insights: Building an Intelligent Document Processing System on AWS
Build an automated document processing pipeline on AWS. Combine Amazon Textract OCR with Amazon Bedrock LLMs to extract clean, structured JSON from unstructured PDFs and scanned forms.
The Problem
Extracting structured data from raw documents manually or with basic rule-based parsers poses severe challenges:
- Unstructured Formats: Important fields are scattered across multi-page tables, variable headers, and nested footers.
- Handwriting and Low Quality: Scanned files with noise, skewness, or handwritten annotations break traditional template matching.
- Lack of Semantic Understanding: Basic OCR retrieves plain strings like
"Total: 150.00"without knowing if it represents an invoice balance, a medical lab reading, or a tax field.
An automated AI pipeline bridges this gap by extracting raw text geometries and using Large Language Models (LLMs) to map unstructured text directly into normalized JSON models.
How DocuMind AI Works
The application follows a streamlined serverless workflow:
Document (PDF / Image)
│
▼
Document Upload
│
▼
Amazon S3 Ingestion
│
▼
AWS Lambda Trigger
│
▼
Amazon Textract (OCR & Layout)
│
▼
Raw Text & Forms
│
▼
Amazon Bedrock (Claude / LLM)
│
▼
Structured JSON & Insights
│
▼
Amazon DynamoDB / User Dashboard
The user uploads a PDF or scan, which is picked up automatically by background services to process geometry, extract text, run LLM normalization, and store structured records.
Why Combine Textract and Amazon Bedrock?
Traditional OCR engines only output raw text strings without contextual awareness. By combining specialized document vision with generative AI, each service handles what it does best:
- Amazon Textract: Handles structural parsing, table extraction, multi-column bounding boxes, and document geometry.
- Amazon Bedrock: Performs semantic understanding, entity classification, missing field inference, and standardized JSON output formatting.
A simplified pipeline structure looks like this:
Raw Document
│
▼
Textract Optical Parsing
│
▼
Key-Value & Table Extraction
│
▼
Prompt Formatting
│
▼
Bedrock Foundation Model
│
▼
JSON Entity Extraction
Bringing the Project to AWS
Building a production-ready ML solution requires reliable cloud infrastructure for orchestration, security, storage, and API access. A production AWS cloud architecture for DocuMind AI is structured as follows:
User / Frontend
│
▼
Amazon API Gateway
│
▼
AWS Lambda
│
┌─────┴────────────────┐
▼ ▼
Amazon S3 Amazon Textract
│ │
└───────┬──────────────┘
▼
Amazon Bedrock (LLM Endpoint)
│
▼
Amazon DynamoDB
│
▼
Amazon CloudWatch (Logs & Metrics)
Amazon S3 for Storage
Amazon S3 handles incoming uploaded documents and stores processed JSON outputs. Organizing files into logical prefix paths keeps ingestion predictable:
documind-storage-bucket/
│
├── raw-uploads/
│ ├── invoices/
│ └── medical-reports/
├── textract-outputs/
└── processed-json/
S3 bucket notification triggers allow AWS Lambda to run processing functions instantly as soon as a new file lands in
raw-uploads/ .
Amazon Textract for Document Analysis
Amazon Textract processes raw document images or multi-page PDFs to retrieve lines, words, key-value pairs, and tabular layouts using dedicated API calls:
-
DetectDocumentText: Quickly extracts raw strings and line positions.
-
AnalyzeDocument: Identifies structural elements usingFORMSandTABLESfeature types.
AWS Lambda for Serverless Processing
AWS Lambda orchestrates the end-to-end pipeline asynchronously without requiring persistent server infrastructure.
import boto3
import json
textract = boto3.client('textract')
bedrock = boto3.client('bedrock-runtime')
def lambda_handler(event, context):
bucket = event['Records'][0]['s3']['bucket']['name']
key = event['Records'][0]['s3']['object']['key']
# 1. Extract text via Textract
response = textract.analyze_document(
Document={'S3Object': {'Bucket': bucket, 'Name': key}},
FeatureTypes=['FORMS', 'TABLES']
)
raw_lines = [b['Text'] for b in response['Blocks'] if b['BlockType'] == 'LINE']
extracted_text = "\n".join(raw_lines)
# 2. Extract structured entities via Bedrock
prompt = f"Human: Extract document_type, issuer, date, total_amount, and key_summary as valid JSON from:\n{extracted_text}\nAssistant:"
bedrock_response = bedrock.invoke_model(
modelId='anthropic.claude-v2',
body=json.dumps({"prompt": prompt, "max_tokens_to_sample": 500, "temperature": 0.1}),
contentType='application/json'
)
result = json.loads(bedrock_response['body'].read().decode('utf-8'))
return {"statusCode": 200, "body": result.get('completion')}
Amazon API Gateway
Amazon API Gateway exposes RESTful HTTPS endpoints to connect web/mobile user interfaces with Lambda execution logic, separating backend processing from frontend presentation.
Amazon CloudWatch for Monitoring
Monitoring operational health is essential once deployed. Amazon CloudWatch tracks:
- API request volume and invocation latency.
- Textract document parsing success rate.
- Bedrock token consumption and response latency.
- Lambda failure alerts and retry counts.
Understanding the Extraction
Suppose a user uploads a messy invoice scan. The system processes the file through the pipeline:
- Textract Parsing: Reads raw text blocks and identifies key-value positions.
- LLM Ingestion: Passes text to Bedrock with strict schema prompt constraints.
- Structured JSON Output:
{
"document_type": "Invoice",
"issuer": "Acme Logistics Inc.",
"date": "2026-08-08",
"total_amount": "$1,250.00",
"confidence_score": 0.94,
"key_summary": "Invoice for cloud infrastructure consulting services."
}
Confidence scores and validation checks prevent downstream database pollution when files are unreadable.
Going Beyond Raw Text Extraction
An enterprise-ready IDP system should deliver actionable workflow intelligence alongside basic extraction:
Classification & Tagging
Automatically categorizes incoming files into buckets like Receipts, Medical Scans, Tax Forms, or Legal Contracts.
Schema Validation
Verifies whether essential required fields (e.g., Total Amount, Invoice ID) exist before saving.
PII Redaction
Identifies and masks sensitive Personal Identifiable Information (SSN, medical ID) before storing records.
Human-in-the-Loop Routing
Flags low-confidence extractions for manual review by human operators before updating records.
Challenges
Building an intelligent document pipeline introduced key real-world engineering challenges:
Scanned Document Quality
Extremely blurry scans, skewed angles, or heavy handwriting reduce Textract accuracy. High-quality image preprocessing (deskewing, contrast adjustments) is critical before feeding files to OCR.
Large PDF Context Windows
Multi-page documents can exceed LLM prompt context windows. Splitting documents into chunked sections or pre-filtering irrelevant pages is required for multi-page document processing.
Prompt Hallucination
Foundation models can sometimes invent missing values if a field isn't explicitly in the source document. Enforcing strict system prompt rules (
Return null if value is not explicitly present ) prevents halluncinated fields.
Future Improvements
- Multi-Modal Foundation Models: Utilizing vision-language models directly on document images to preserve spatial layout without needing prior OCR steps.
- Multi-Lingual Processing: Translating foreign language invoices automatically into standardized English JSON outputs.
- Automated Fine-Tuning: Continuously fine-tuning lightweight models on domain-specific document types to lower inference latency and costs.
Lessons Learned
Building this project highlighted key operational insights:
- OCR quality dictates LLM success: Garbage in results in garbage out; structured OCR output is crucial for LLM accuracy.
- Serverless handles unpredictable spikes: S3-triggered Lambda architecture scales smoothly regardless of document submission volume.
- Cost optimization matters: Using smaller, optimized models for basic documents saves money compared to invoking massive LLMs for every simple file.
The ultimate takeaway is that combining specialized cloud vision services with generative AI turns static document files into structured, actionable database assets at scale.
Conclusion
Intelligent Document Processing is a practical demonstration of how modern cloud architectures and generative AI combine to solve real business bottlenecks. By linking Amazon Textract, Amazon Bedrock, AWS Lambda, and S3, you can build a scalable, production-grade cloud system that converts messy physical documents into structured data ready for downstream applications.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article