Analyzing Insurance Churn with SageMaker Data Agent, Amazon S3 Tables and Faker
Learn how to use Amazon SageMaker Data Agent, Amazon S3 Tables, and the Python Faker library to generate realistic synthetic insurance data for churn analysis, without exposing live policyholder records. This post walks through generating synthetic tables from scratch and creating synthetic twins from existing tables, then shows how to use plain-language prompts to analyze churn patterns across products, demographics, payment behavior, and customer interactions.
Insurance providers rely on high quality data to understand why customers churn, which products are at risk, and where to focus retention efforts. At the same time, policy and interaction data is some of the most sensitive information an insurer holds, so copying it into ad hoc sandboxes is rarely an option.
In this post, we show how to use Amazon SageMaker Data Agent , Amazon S3 Tables , and the Python Faker library to generate realistic synthetic insurance data for churn analysis, without exposing live policyholder records. You will walk through two patterns: generating synthetic tables from scratch using plain-language prompts, and creating a synthetic twin from an existing table while preserving its schema and distributions.
Once the synthetic data is in place, we explore how SageMaker Data Agent can be used to analyze churn patterns across products, demographics, payment behavior, and customer interactions, all within a governed lakehouse environment built on Apache Iceberg.
Why prompt synthetic data?
Teams exploring customer churn typically need to join policy, billing, claims, and interaction data at scale, to iterate on data pipelines and dashboards. Doing that with production data introduces privacy risk and slows you down, because every new environment requires careful approvals and data masking. In the analytics lifecycle, synthetic data is most valuable before pipelines and dashboards reach production and then serves as a safe sandbox once they do.
For Prototyping, you can use synthetic data to design schemas and pipelines, test calculations, and sketch out churn or revenue dashboards without waiting for access to production tables. Because it mirrors real structure and rough distributions, charts and filters behave realistically while still protecting customer information.
In Development and Test, synthetic data powers day to day work on data pipelines and dashboards. You can build views, wire dashboard visuals, drill downs, and create row level security rules against synthetic tables. You can push these tables to production‑like volumes to check performance and edge cases. If the row-level security is not right, or there are issues with data pipelines or a dashboard, it is only a fake dataset, and you have not exposed any real records.
In Pre‑Production, you can introduce real data in a controlled way. Data pipelines and dashboards that already behave correctly on synthetic data can be pointed to masked or tightly governed production tables. You can use this stage to validate numbers, compare results, and run user acceptance testing before anything is exposed to a wider audience.
Once pipelines and dashboards are run in Production, real data drives decisions, but synthetic data still helps you evolve the content. You use it to prototype new pipelines and dashboards, rehearse schema changes, and train users on realistic examples, all without touching live customer data.
And now with SageMaker Data Agent you can describe the synthetic dataset you need in plain language, and the agent generates the code and SQL to build it on top of Amazon S3 Tables. From there, analysts can connect with their existing tools and start exploring churn patterns within minutes, without waiting for one-off exports.
Why use Faker?
Faker is a lightweight Python library that generates realistic values such as names, addresses, phone numbers, dates, amounts, and many other domain specific fields. In a previous blog , I walked through how you could use Faker to create synthetic data pipelines, but with the Data Agent in SageMaker Unified Studio, it’s even easier.
You can use Faker in two main ways. First, you can create synthetic data from scratch. By selecting the right Faker providers and seeding the random number generator, you can produce thousands or millions of rows that look like real insurance data, including plausible policy numbers, premium amounts, and contact details.
Second, you can start from an existing table and build a synthetic twin. In this pattern, you read the schema and simple distributions from the real table, then use Faker to generate replacements for each column, keeping data types and high-level patterns intact. The result behaves like production data from the perspective of pipelines and dashboards but contains no live customer information. Because both patterns use the same library and prompting style, you can standardize how you ask for synthetic tables across teams and projects.
Note: To add the Faker library to your SageMaker Unified Studio notebook, choose the Packages menu, and select Add to install the Faker package into your environment.
Creating synthetic data from scratch
To walk through the first pattern, open a new notebook in SageMaker Unified Studio (IAM Domain) and select the icon to open the Data Agent pane. Here, you can provide a prompt that defines the synthetic dataset you want the agent to generate. For an insurance churn use case, the prompt might look like:
Prompt: Create a fully synthetic dataset for policyholder churn and premium revenue analysis for an Australian insurance company.
Create up to three relational tables in the insurance database in the policy_churn schema stored as tables on Amazon S3 Tables, with each table having fewer than 20 columns. Include keys to join customers to policies, claims or billing, and a churn or lapse flag, and generate data for about 10,000 distinct policyholders.
Using the Faker library, create realistic Australian context (states, phone formats, AUD), and include fields for policy tenure, monthly or annual premium, total written premium, product line, policy type, churn or lapse date, and churn reason.
Lastly, analyze the data generated and use the notebook’s interactive table to explore retention rate by state.
This prompt includes four important elements:
The first sentence sets the business objective and domain. It specifies that the dataset is fully synthetic, focused on churn and premium revenue, and based on an Australian insurance portfolio. This context steers the agent toward appropriate entities and attributes such as policy tenure, product lines, and lapse behavior.
The second part applies structural constraints. It limits the solution to three relational tables with fewer than twenty columns each, created in insurance database, in the policy_churn schema on Amazon S3 Tables. It also asks for join keys between policyholders, policies, claims or billing, and a churn or lapse flag. These constraints encourage a compact, easy to understand schema that still supports common churn analytics, and they target a volume of about ten thousand policyholders for modelling and performance testing.
The third part introduces local realism and feature requirements. It calls out Faker explicitly, then asks for Australian states, phone formats, and AUD currency to keep the data geographically consistent. It lists the key fields that data analysts care about, including tenure, premium amounts, product line, policy type, churn date, and churn reason. Connecting features directly to the business questions at prompt time is what makes the final dataset genuinely useful, rather than just a collection of generic insurance fields.
The final sentence focuses on how you plan to validate the output. Asking for an interactive table that highlights retention by state gives the agent a clear first analysis to support and gives you a natural starting point once the tables are created.
From prompt to Amazon S3 Tables
When you submit this prompt, SageMaker Data Agent translates it into a sequence of steps. The agent generates the code and DDL to create S3 Tables in the policy_churn schema, defines primary and foreign keys, and chooses data types that match the fields you described. It then produces Python and PySpark code that uses Faker to populate those tables with synthetic policyholders, policies, claims or billing records, and churn outcomes that respect the requested volume and structure.
Because Amazon S3 Tables are built on Apache Iceberg, the synthetic tables immediately benefit from Iceberg features such as schema evolution, partitioning, and time travel, while keeping all data in S3.
In this example, the SageMaker Data Agent has created 3 tables in the policy_churn schema (Billing, Customers and Policies):
Now these tables are in place, you can connect to them in SageMaker Unified Studio through notebooks or the query editor , through Amazon Athena , Amazon Redshift , and other Iceberg-aware engines without additional data movement. This makes it straightforward to validate that the schema matches your expectations, review distributions, and, if needed, adjust the prompt and regenerate the dataset.
Working with notebook interactive tables
In the last step of the prompt, it asks Data Agent to surface the synthetic tables in an interactive table view inside the notebook. This view provides an immediate way to explore the data without writing SQL. You can filter by state, average annual premium, sort by retention or churn rates, etc. to help you understand the data that has been generated.
Interactive tables also support lightweight visualizations. You can build bar charts, line charts, scatter plots and other chart types directly over the synthetic data, such as a bar chart of retention rate by state. These views help you confirm that the generated data looks sensible before investing time in building data pipelines and dashboards. If the visuals reveal unrealistic patterns, you can refine the prompt or tighten the Faker configuration and regenerate the tables.
Creating synthetic data from an existing table
In the second pattern we are going to look at, you can start from an existing table, with real data, rather than a blank slate. The goal is to preserve the shape and behavior of the base table while removing any sensitive information. For example, you might have a table in AWS Glue Data Catalog called customersupport.customer_interactions that records calls, emails, and chat sessions from policyholders. You want to be able to use the data in this table with the rest of your synthetic data set you have created, and integrate Customer Interactions with Billing, Customers and Policies
To ensure that all of the tables can be joined together, you can create a new synthetic version of customer_interactions, but you can reuse the customer_id values from the existing synthetic Customers table (insurance.policy_churn.customers) that was created in the first prompt. That way, every interaction still joins cleanly back to a customer.
A prompt for this pattern could be:
Prompt: Create a fully synthetic version of the customersupport.customer_interactions table using the Faker library.
When the synthetic table is ready, write it as an Amazon S3 Table in the insurance database and policy_churn schema, with the table name customer_interactions.
Use the existing synthetic insurance.policy_churn.customers table as the source of truth for keys: sample its customer_id values and use only those values in the synthetic interactions table so that joins on customer_id remain valid.
For every other column in customersupport.customer_interactions, generate realistic synthetic values with Faker that match the original data types and patterns.
In response, Data Agent inspects the source schema and then generates code that uses Faker to build a new interactions table with the same structure. The columns have kept their names and formats, but the values are replaced with synthetic values. In the case of the customer_id column, it is the same column definition as the original table, but it used values from the pure synthetic Customers table that was created by the first prompt.
The resulting Iceberg backed S3 Table can be joined back to your synthetic customers and policies and used in churn models that rely on interaction frequency, complaint types, or resolution outcomes.
Viewed side by side, the original and synthetic tables share the same columns and data types and show similar distributions for channels and outcome codes, but every record in the synthetic table is fabricated. That makes the synthetic version a drop in replacement for many development, testing, and analytics scenarios, while keeping sensitive customer support data in tightly controlled locations.
With all of the synthetic tables created, we now can get into the analysis of the actual data, using SageMaker Data Agent.
Example Prompts for Churn Analysis
Below are some example prompts you can use with SageMaker Data Agent to analyze customer churn and retention using the insurance.policy_churn schema in S3 Tables. Each prompt demonstrates different analytical capabilities.
Basic Churn Rate Analysis
Prompt: Calculate the overall churn rate from the policies table in the insurance database.
Show me the total number of policies, number of churned policies, and the churn
percentage.
This prompt helps you understand the fundamental churn metrics for your insurance portfolio. The agent will query the
policies table to calculate the churn rate by counting policies where churn_flag = 1 versus total policies. This is typically the first metric stakeholders want to know and establishes a baseline for deeper analysis. Churn Analysis by Product Line
Prompt: Analyze churn rates across different product lines using the insurance S3 Tables.
Group by product_line and show the churn rate, average premium, and number of
customers for each product type.
Different insurance products often have different churn characteristics. This prompt segments churn by product line, revealing which products have retention challenges. Understanding product-specific churn helps target retention strategies where they're needed most and can inform product development priorities.
Customer Demographics and Churn
Prompt: Join the customers and policies tables from the insurance database and analyze
how customer age and state affect churn rates. Create a visualization showing
churn patterns across age groups and geographic regions.
Demographics significantly influence customer behavior. This prompt combines customer demographic data with policy churn information to identify high-risk segments. The visualization helps spot patterns in the data. For example, younger customers in certain states might churn more frequently, informing targeted marketing and retention campaigns.
Payment Behavior and Churn Correlation
Prompt: Using the billing and policies tables, analyze the relationship between payment
status (Late, Paid, Pending) and churn. Calculate the churn rate for customers
with late payments versus on-time payments.
Payment behavior is often a leading indicator of churn. This analysis reveals whether customers with late or missed payments are more likely to cancel their policies. Understanding this correlation helps create early warning systems and allows customer service teams to proactively reach out to at-risk customers before they churn.
Customer Interaction Impact on Retention
Prompt: Analyze the customer_interactions table to understand how customer service
interactions correlate with churn. Compare the average number of interactions,
sentiment scores, and interaction channels between churned and retained customers.
Customer service interactions can either strengthen or weaken customer relationships. This prompt examines whether frequent interactions, negative sentiment, or specific channels (phone, email, chat) correlate with higher churn rates. Insights help improve service quality, train agents, and optimize channel strategies.
Churn Timeline Analysis
Prompt: Create a time-series analysis showing monthly churn trends over the past two years
using the policies table. Include policy_start_date and churn_date to identify
seasonal patterns or concerning trends.
Churn often follows patterns relating to what is happening at the time, including seasonal shifts, economic events, or competitive actions. This time-series analysis reveals when churn spikes occur, helping you understand whether issues are recent or long-standing. Identifying seasonal patterns enables proactive retention campaigns during high-risk periods.
High-Value Customer Retention Analysis
Prompt: Identify high-value customers (top 20% by annual_premium_aud) from the policies
table who have churned. Calculate the revenue impact and analyze their churn
reasons to understand why valuable customers are leaving.
Losing high-value customers has disproportionate business impact. This prompt focuses specifically on premium customers who churned, quantifying the revenue loss and identifying common churn reasons. This targeted analysis helps prioritize retention efforts for the most valuable customer segments and reveals if premium customers have unique concerns.
Churn Reason Deep Dive
Prompt: Create a comprehensive analysis of churn_reason from the policies table. Show the
distribution of churn reasons, average customer tenure before churning for each
reason, and the associated revenue loss per reason category.
Understanding why customers leave is crucial for prevention. This prompt categorizes all churn reasons and quantifies their impact by frequency, timing, and revenue loss. The insights directly inform which retention initiatives to prioritize to keep customers. For example, if "Better Price Elsewhere" is a top reason, pricing strategy needs attention.
Tips for prompting with SageMaker Data Agent:
- Be specific: Mention the database name (insurance) and schema (policy_churn) when needed
- Request visualizations: Add phrases like "create a chart" or "visualize the results" for graphical outputs
- Iterate: Start with basic prompts and refine based on results
- Combine insights: Use findings from one prompt to inform follow-up questions
- Ask for explanations: Request the agent to explain patterns it finds in the data
Summary
In this post, we covered two approaches to synthetic data generation for insurance churn analytics using Amazon SageMaker Data Agent, Amazon S3 Tables, and the Faker library. The first approach generates relational tables from scratch using a plain-language prompt, producing realistic policyholder, policy, and billing data stored directly in S3 as Iceberg tables. The second approach creates a synthetic twin from an existing table, preserving its schema and distributions while replacing every value with fabricated data and maintaining valid join keys across the broader dataset.
With all four synthetic tables in place, we demonstrated how SageMaker Data Agent can be used to analyze churn rates by product line, correlate payment behavior with cancellations, assess the impact of customer service interactions on retention, and identify revenue loss from high-value customers leaving.
Treat each prompt as a lightweight, evolving specification. Over time, a shared library of prompts becomes a reusable foundation for synthetic data generation and analysis across teams, helping you iterate faster, protect customer privacy, and fully leverage your lakehouse environment.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article