AWS Builder Center
From Scanned PDFs to Smart Insights: Building an Intelligent Document Processing System on AWS

From Scanned PDFs to Smart Insights: Building an Intelligent Document Processing System on AWS

Build an automated document processing pipeline on AWS. Combine Amazon Textract OCR with Amazon Bedrock LLMs to extract clean, structured JSON from unstructured PDFs and scanned forms.

Backend and AI Engineer | Python & Cloud Developer



The Problem

Extracting structured data from raw documents manually or with basic rule-based parsers poses severe challenges:
  • Unstructured Formats: Important fields are scattered across multi-page tables, variable headers, and nested footers.
  • Handwriting and Low Quality: Scanned files with noise, skewness, or handwritten annotations break traditional template matching.
  • Lack of Semantic Understanding: Basic OCR retrieves plain strings like  "Total: 150.00"  without knowing if it represents an invoice balance, a medical lab reading, or a tax field.
An automated AI pipeline bridges this gap by extracting raw text geometries and using Large Language Models (LLMs) to map unstructured text directly into normalized JSON models.



How DocuMind AI Works

The application follows a streamlined serverless workflow:
Document (PDF / Image)
│
▼
Document Upload
│
▼
Amazon S3 Ingestion
│
▼
AWS Lambda Trigger
│
▼
Amazon Textract (OCR & Layout)
│
▼
Raw Text & Forms
│
▼
Amazon Bedrock (Claude / LLM)
│
▼
Structured JSON & Insights
│
▼
Amazon DynamoDB / User Dashboard

The user uploads a PDF or scan, which is picked up automatically by background services to process geometry, extract text, run LLM normalization, and store structured records.



Why Combine Textract and Amazon Bedrock?

Traditional OCR engines only output raw text strings without contextual awareness. By combining specialized document vision with generative AI, each service handles what it does best:
  • Amazon Textract: Handles structural parsing, table extraction, multi-column bounding boxes, and document geometry.
  • Amazon Bedrock: Performs semantic understanding, entity classification, missing field inference, and standardized JSON output formatting.
A simplified pipeline structure looks like this:
Raw Document
│
▼
Textract Optical Parsing
│
▼
Key-Value & Table Extraction
│
▼
Prompt Formatting
│
▼
Bedrock Foundation Model
│
▼
JSON Entity Extraction




Bringing the Project to AWS

Building a production-ready ML solution requires reliable cloud infrastructure for orchestration, security, storage, and API access. A production AWS cloud architecture for DocuMind AI is structured as follows:
User / Frontend
│
▼
Amazon API Gateway
│
▼
AWS Lambda
│
┌─────┴────────────────┐
▼ ▼
Amazon S3 Amazon Textract
│ │
└───────┬──────────────┘
▼
Amazon Bedrock (LLM Endpoint)
│
▼
Amazon DynamoDB
│
▼
Amazon CloudWatch (Logs & Metrics)




Amazon S3 for Storage

Amazon S3 handles incoming uploaded documents and stores processed JSON outputs. Organizing files into logical prefix paths keeps ingestion predictable:
documind-storage-bucket/
│
├── raw-uploads/
│ ├── invoices/
│ └── medical-reports/
├── textract-outputs/
└── processed-json/

S3 bucket notification triggers allow AWS Lambda to run processing functions instantly as soon as a new file lands in  raw-uploads/ .



Amazon Textract for Document Analysis

Amazon Textract processes raw document images or multi-page PDFs to retrieve lines, words, key-value pairs, and tabular layouts using dedicated API calls:
  •  DetectDocumentText : Quickly extracts raw strings and line positions.
  •  AnalyzeDocument : Identifies structural elements using  FORMS  and  TABLES  feature types.



AWS Lambda for Serverless Processing

AWS Lambda orchestrates the end-to-end pipeline asynchronously without requiring persistent server infrastructure.
import boto3
import json
textract = boto3.client('textract')
bedrock = boto3.client('bedrock-runtime')
def lambda_handler(event, context):
bucket = event['Records'][0]['s3']['bucket']['name']
key = event['Records'][0]['s3']['object']['key']
# 1. Extract text via Textract
response = textract.analyze_document(
Document={'S3Object': {'Bucket': bucket, 'Name': key}},
FeatureTypes=['FORMS', 'TABLES']
)
raw_lines = [b['Text'] for b in response['Blocks'] if b['BlockType'] == 'LINE']
extracted_text = "\n".join(raw_lines)
# 2. Extract structured entities via Bedrock
prompt = f"Human: Extract document_type, issuer, date, total_amount, and key_summary as valid JSON from:\n{extracted_text}\nAssistant:"
bedrock_response = bedrock.invoke_model(
modelId='anthropic.claude-v2',
body=json.dumps({"prompt": prompt, "max_tokens_to_sample": 500, "temperature": 0.1}),
contentType='application/json'
)
result = json.loads(bedrock_response['body'].read().decode('utf-8'))
return {"statusCode": 200, "body": result.get('completion')}




Amazon API Gateway

Amazon API Gateway exposes RESTful HTTPS endpoints to connect web/mobile user interfaces with Lambda execution logic, separating backend processing from frontend presentation.



Amazon CloudWatch for Monitoring

Monitoring operational health is essential once deployed. Amazon CloudWatch tracks:
  • API request volume and invocation latency.
  • Textract document parsing success rate.
  • Bedrock token consumption and response latency.
  • Lambda failure alerts and retry counts.



Understanding the Extraction

Suppose a user uploads a messy invoice scan. The system processes the file through the pipeline:
  1. Textract Parsing: Reads raw text blocks and identifies key-value positions.
  1. LLM Ingestion: Passes text to Bedrock with strict schema prompt constraints.
  1. Structured JSON Output:
{
"document_type": "Invoice",
"issuer": "Acme Logistics Inc.",
"date": "2026-08-08",
"total_amount": "$1,250.00",
"confidence_score": 0.94,
"key_summary": "Invoice for cloud infrastructure consulting services."
}

Confidence scores and validation checks prevent downstream database pollution when files are unreadable.



Going Beyond Raw Text Extraction

An enterprise-ready IDP system should deliver actionable workflow intelligence alongside basic extraction:

Classification & Tagging

Automatically categorizes incoming files into buckets like Receipts, Medical Scans, Tax Forms, or Legal Contracts.

Schema Validation

Verifies whether essential required fields (e.g., Total Amount, Invoice ID) exist before saving.

PII Redaction

Identifies and masks sensitive Personal Identifiable Information (SSN, medical ID) before storing records.

Human-in-the-Loop Routing

Flags low-confidence extractions for manual review by human operators before updating records.



Challenges

Building an intelligent document pipeline introduced key real-world engineering challenges:

Scanned Document Quality

Extremely blurry scans, skewed angles, or heavy handwriting reduce Textract accuracy. High-quality image preprocessing (deskewing, contrast adjustments) is critical before feeding files to OCR.

Large PDF Context Windows

Multi-page documents can exceed LLM prompt context windows. Splitting documents into chunked sections or pre-filtering irrelevant pages is required for multi-page document processing.

Prompt Hallucination

Foundation models can sometimes invent missing values if a field isn't explicitly in the source document. Enforcing strict system prompt rules ( Return null if value is not explicitly present ) prevents halluncinated fields.



Future Improvements

  • Multi-Modal Foundation Models: Utilizing vision-language models directly on document images to preserve spatial layout without needing prior OCR steps.
  • Multi-Lingual Processing: Translating foreign language invoices automatically into standardized English JSON outputs.
  • Automated Fine-Tuning: Continuously fine-tuning lightweight models on domain-specific document types to lower inference latency and costs.



Lessons Learned

Building this project highlighted key operational insights:
  • OCR quality dictates LLM success: Garbage in results in garbage out; structured OCR output is crucial for LLM accuracy.
  • Serverless handles unpredictable spikes: S3-triggered Lambda architecture scales smoothly regardless of document submission volume.
  • Cost optimization matters: Using smaller, optimized models for basic documents saves money compared to invoking massive LLMs for every simple file.
The ultimate takeaway is that combining specialized cloud vision services with generative AI turns static document files into structured, actionable database assets at scale.



Conclusion

Intelligent Document Processing is a practical demonstration of how modern cloud architectures and generative AI combine to solve real business bottlenecks. By linking Amazon Textract, Amazon Bedrock, AWS Lambda, and S3, you can build a scalable, production-grade cloud system that converts messy physical documents into structured data ready for downstream applications.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article