AWS Builder Center

Builder Center

From PDFs to EBCDIC: Building a hallucination-free ETL pipeline for IBM mainframes with Amazon Nova

From PDFs to EBCDIC: Building a hallucination-free ETL pipeline for IBM mainframes with Amazon Nova

How I used Amazon Bedrock Data Automation and Amazon Nova 2 Lite to build a deterministic, zero-hallucination ETL pipeline. Anathis bridges the gap between modern unstructured data and legacy z/OS systems by translating PDFs directly into compiled EBCDIC copybooks.

The $3 trillion friction point: The community we serve

The tech industry is obsessed with building consumer-facing chatbots, but the backbone of the global economy (regional banks, logistics conglomerates, and insurance firms) still runs on IBM Z-Series mainframes. These systems process roughly $3 trillion in daily transactions. They are rigid, incredibly secure, and require strict EBCDIC-encoded data.
Meanwhile, the vendors feeding them operate in the real world with messy, unstructured PDF invoices.
Currently, the bridge between these two worlds relies on brittle OCR templates that break when a logo moves, or armies of offshore back-office clerks acting as human data-entry machines. I built Anathis to solve this. Anathis helps IT infrastructure teams, compliance officers, and logistics operators eliminate manual reconciliation by using Agentic AI to bridge the modern cloud and legacy metal.

The architecture: Constraining AI with deterministic math

My core constraint was simple: Large Language Models are incredible reasoning engines, but they are terrible calculators. A hallucinated decimal point in a banking ledger is unacceptable.
To solve this, Anathis uses a "Zero-Trust AI" architecture orchestrated by AWS Step Functions:
  1. Extraction: A raw PDF invoice is ingested via an Amazon API Gateway (HttpApi) and dropped into an S3 bucket. AWS Step Functions triggers Amazon Bedrock Data Automation (BDA) to asynchronously extract the unstructured text into a structured JSON schema.
  2. Context & routing: The payload is passed to Amazon Nova 2 Lite. To completely eliminate SDK bloat and cold-start latency, I bypassed standard Boto3 integrations and built a zero-dependency, native Python urllib bridge to hit the Nova Developer API directly. Nova reads the context and routes the variables.
  3. The deterministic enforcer: The AI does not do the accounting. Nova passes the extracted integers to a hard-coded Python Lambda function that enforces the math (Subtotal + Tax + Shipping = Grand Total). If the variance is off by even a penny, the pipeline fails the transaction. Zero hallucinations.
  4. Legacy compilation: Once verified, the standard ASCII JSON is cross-compiled into an IBM CP500 EBCDIC hexadecimal format, complete with COBOL PIC $$$,$$$.99 copybook mapping.

"V2" - Scaling for the enterprise (The roadmap)

Anathis was built to prove the deterministic AI pipeline. To scale this for Tier-1 financial institutions, our "V2" AWS architecture roadmap focuses on massive parallel throughput and zero-trust security:
  • Event-driven scale: Transitioning the current BDA polling loop to an Amazon EventBridge integration utilizing .waitForTaskToken. This eliminates idle compute waste and allows the Step Functions pipeline to utilize Distributed Map for highly parallel, 50,000+ file nightly batch processing.
  • Strict VPC enclaving & pre-processing redaction: Moving the AI reasoning engine from the default network into a strictly isolated VPC. By routing outbound inference calls through AWS PrivateLink endpoints, we guarantee zero public internet exposure. Additionally, we will deploy Amazon Macie and Amazon Bedrock Guardrails at the S3 ingestion layer to automatically redact PII (SSNs, Account Numbers) before the AI ever sees the data.
  • Air-gapped mainframe ingestion: Integrating AWS Mainframe Modernization (M2) Dataset APIs and AWS Transfer Family to securely push the generated EBCDIC hex payloads directly into legacy IBM z/OS environments via secure SFTP.
  • Multimodal exception swarms: When a transaction fails the deterministic math check, it will trigger an asynchronous workflow using Amazon Nova Pro. The agent will ingest the raw PDF, visually highlight un-reconciled variables (e.g., handwritten adjustments), and route the visual context to a human Compliance Officer for manual override.
Agentic AI doesn't have to be probabilistic. By combining Amazon Nova's extraction capabilities with strict programmatic constraints, we can finally modernize legacy ingestion without compromising ledger integrity.
# generative-ai# amazon-bedrock# amazon-nova# aws-step-functions# aws-lambda
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article