AWS Builder Center

Bedrock Data Automation: Parse PDFs to JSON in Python

Amazon Bedrock Data Automation turns PDFs into structured JSON with one async API call — no OCR pipeline. Copy-paste Python, cost notes, and cleanup inside.

Series: Bedrock Builder's Log (6 articles)

  1. …
  2. 6
    Bedrock Data Automation: Parse PDFs to JSON in Python This article
Lead Data Scientist

Bedrock Data Automation: Parse PDFs to JSON in Python

Every RAG project I've shipped hits the same wall on day one: how do we turn these PDFs into clean text? The usual answer is a pipeline — Textract for OCR, a layout heuristic for tables, a summarizer for figures, glue code to stitch it together. It works, and it's a maintenance tax you pay forever.
Amazon Bedrock Data Automation (BDA) collapses that pipeline into a single async API call. You hand it an S3 URI, it hands back structured JSON — text, Markdown, tables, and page-level metadata — for documents, images, audio, and video. Best part for getting started: there's a public default project, so you can extract standard output today without creating or configuring anything.
Here's the whole thing, copy-pasteable — and the full runnable version lives in the companion repo,   bedrock-data-automation-pdf-quickstart  .

Prerequisites

  • An AWS account with Bedrock Data Automation enabled in a supported Region (I'm using  us-east-1 ).
  • An S3 bucket you own, with a PDF uploaded ( s3://YOUR-BUCKET/input/sample.pdf ) and an output prefix ( s3://YOUR-BUCKET/output/ ).
  •  boto3  installed ( pip install boto3 ), and an IAM identity that can call  bedrock-data-automation-runtime  plus read/write your bucket.
BDA uses two clients:  bedrock-data-automation  for build-time work (creating projects and blueprints) and  bedrock-data-automation-runtime  for invoking jobs. We only need the runtime client today.

Submit the document

python
import boto3, time, json
from urllib.parse import urlparse

REGION = "us-east-1"
ACCOUNT_ID = boto3.client("sts").get_caller_identity()["Account"]

bda = boto3.client("bedrock-data-automation-runtime", region_name=REGION)
s3 = boto3.client("s3", region_name=REGION)

# The public default project gives you standard output with zero setup.
PROJECT_ARN = f"arn:aws:bedrock:{REGION}:aws:data-automation-project/public-default"
PROFILE_ARN = f"arn:aws:bedrock:{REGION}:{ACCOUNT_ID}:data-automation-profile/us.data-automation-v1"

resp = bda.invoke_data_automation_async(
inputConfiguration={"s3Uri": "s3://YOUR-BUCKET/input/sample.pdf"},
outputConfiguration={"s3Uri": "s3://YOUR-BUCKET/output/"},
dataAutomationConfiguration={"dataAutomationProjectArn": PROJECT_ARN},
dataAutomationProfileArn=PROFILE_ARN,
)
invocation_arn = resp["invocationArn"]
print("Submitted:", invocation_arn)
Two ARNs trip people up, so note them:
  •  dataAutomationProjectArn  —  public-default  is an AWS-owned project. Point at it and you skip project creation entirely.
  •  dataAutomationProfileArn  — required, and it's your account's profile ( us.data-automation-v1 ), not an AWS-owned one. Leaving it out is the most common first-call error.

Poll for completion

 invoke_data_automation_async  is fire-and-forget. Poll  get_data_automation_status  until it leaves  InProgress :
python
def wait_for_bda(arn, timeout=600, interval=5):
deadline = time.time() + timeout
while time.time() < deadline:
status = bda.get_data_automation_status(invocationArn=arn)
state = status["status"]
if state in ("Success", "ServiceError", "ClientError"):
return status
print(" ...", state)
time.sleep(interval)
raise TimeoutError("BDA job did not finish in time")

result = wait_for_bda(invocation_arn)
if result["status"] != "Success":
raise RuntimeError(result)

# On success, BDA points you at a job_metadata.json in your output prefix.
meta_uri = result["outputConfiguration"]["s3Uri"]
print("Metadata at:", meta_uri)
In production, swap polling for an EventBridge notification — BDA supports it via  notificationConfiguration  on the invoke call, so you don't burn a Lambda on a  time.sleep  loop.

Read the structured output

The status response points at  job_metadata.json , which references the actual per-asset result file. Follow one hop:
python
def read_s3_json(uri):
p = urlparse(uri)
body = s3.get_object(Bucket=p.netloc, Key=p.path.lstrip("/"))["Body"].read()
return json.loads(body)

meta = read_s3_json(meta_uri)
segment = meta["output_metadata"][0]["segment_metadata"][0]
standard = read_s3_json(segment["standard_output_path"])

# Document standard output: page-level text plus a document-wide Markdown view.
doc = standard["document"]
print(doc["description"][:500]) # summary of the document
for page in standard.get("pages", []):
print(f"\n--- page {page['page_index']} ---")
print(page["representation"]["markdown"][:800]) # tables preserved as Markdown!
That  markdown  field is the payoff: tables come back as pipe-delimited Markdown, headings stay headings, and reading order is sane — exactly the shape a chunker or an LLM wants. No bounding-box math, no table reconstruction.

Cost and cleanup

  • Cost: standard document output runs about $0.010 per page; custom output (your own blueprint with typed fields) is roughly $0.040 per page. A 1,000-page batch of standard extraction is around $10. Confirm current numbers on the  Bedrock pricing page  — per-page beats token-based OCR math for predictability.
  • Cleanup: BDA itself leaves nothing running — there's no endpoint to delete. Just remove the S3 artifacts:  aws s3 rm s3://YOUR-BUCKET/output/ --recursive . If you created a custom project or blueprint later, delete those too.

Why this matters — what I'd do

The strategic shift here isn't "another OCR API." It's that document parsing became a managed, per-page line item instead of a service you operate. For a RAG ingestion path, I'd default to BDA standard output and only reach for a custom blueprint when I need typed fields (invoice totals, claim numbers) validated at extraction time. Where I'd stay on Textract: high-volume, latency-sensitive form extraction where I already have tuned queries and want the cheaper per-page rate. BDA earns its premium when documents are messy and multimodal — mixed tables, figures, and prose — which is most real-world PDFs.
Start with  public-default , prove the output shape against your ugliest document, then decide whether a blueprint is worth building.
What's still forcing you to run your own document pipeline — table fidelity, cost, latency, or something BDA doesn't touch yet? I'd like to hear where it breaks for you.
.
.
.
#AWS #Bedrock #GenAI #MachineLearning #RAG #Python

Series: Bedrock Builder's Log (6 articles)

  1. …
  2. 6
    Bedrock Data Automation: Parse PDFs to JSON in Python This article
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article