Connecting to Salesforce Data Using AWS Glue
Integrating Salesforce data into your AWS analytics ecosystem is an essential step in building a comprehensive view of your customers, sales, and operations. With the growing number of options available in AWS for ingesting and transforming external data, it’s important to understand which approach best suits your needs—especially when comparing traditional Glue ETL jobs with newer Zero-ETL features.
Integrating Salesforce data into your AWS analytics ecosystem is an essential step in building a comprehensive view of your customers, sales, and operations. With the growing number of options available in AWS for ingesting and transforming external data, it’s important to understand which approach best suits your needs—especially when comparing traditional Glue ETL jobs with newer Z ero-ETL features.
In this post, we’ll first compare Glue ETL and Zero-ETL approaches, then walk through how to connect to Salesforce using AWS Glue Studio and the native Salesforce Connector, a low-code, scalable option for building powerful ETL pipelines.
Glue ETL Job vs. Zero ETL: What's the Difference?
Before diving into the implementation details, it’s helpful to understand the difference between two prominent strategies: Glue ETL Jobs and Glue Zero-ETL.
Glue ETL Jobs refer to the traditional approach where you build extract-transform-load (ETL) jobs using AWS Glue. These jobs can be authored in Python or Scala (Spark-based), or created visually using AWS Glue Studio. With Glue ETL, you have full control over data extraction, transformation logic, and how/where the data is stored (e.g., S3, Redshift).
Zero-ETL, on the other hand, refers to a set of fully managed integrations that provide automatic data replication from source systems (like operational databases) into analytics destinations, with minimal setup and no code. Zero-ETL aims to reduce operational complexity and make near real-time analytics more accessible.
Pros and Cons
Feature | Glue ETL Jobs | Glue Zero-ETL |
|---|---|---|
Control | Full control over transformation, schema, scheduling | Easy to use, out of the box, |
Development | Creation of Glue ETL jobs required | No ETL jobs required |
Latency | Batch-oriented | Near real-time |
Complexity | Higher setup, but customizable | Simple setup, but less flexible |
Best For | Complex pipelines | Replication |
So Glue Zero-ETL is a good choice for getting Salesforce data, but when you need to create more complex workflows or data pipelines, you may want to use Glue ETL jobs.
Connecting to Salesforce with AWS Glue Studio
With the Salesforce Connector now available in AWS Glue, it’s easier than ever to integrate Salesforce data into your analytics workflows. The connector supports both standard and custom Salesforce objects, allows for SOQL-based querying, and handles schema inference—all within the familiar Glue environment.
To get started, follow these steps:
- You will need to create a Salesforce connection within the AWS Glue Console. Navigate to the Glue service, then go to the “Connections” tab and select Add connection. Choose “Salesforce” as the connection type. You’ll be asked to provide authentication credentials, which can be set up using either OAuth or username-password. If using OAuth, ensure you’ve configured a connected app in Salesforce with the correct client ID, secret, and refresh token. You'll also need to specify the correct login URL (e.g., https://login.salesforce.com or a sandbox URL). As part of creating the connection, you will be able to test your connection to ensure that it can connect to Salesforce.
- Once your connection is successfully created, you can begin building your ETL job in AWS Glue Studio. In Glue Studio, choose Create job and select the visual option that starts with a source and target.
- Under the source node, choose Salesforce as your source type and select the connection you just created. Glue Studio will then allow you to browse available Salesforce objects, including both standard entities like Account, Contact, Opportunities, etc. as well as any custom objects/tables in your instance.
- You can either load entire tables or select specific columns you want to retrieve. You also have the ability to specify a WHERE clause for the data to be retrieved. This gives you a high degree of control and allows you to avoid over-fetching unnecessary data. For example, your might want to retrieve only contacts created in the past 30 days or filter by specific opportunity stages.
Note: There are some notes available here on the TIMESTAMP format you will need to use for the WHERE clause.
- After defining your source, you can optionally apply transformations using Glue Studio’s visual flow editor. This is useful if you want to clean or reshape your data—for instance, by renaming columns, dropping unused fields, or joining with another dataset. If your goal is simply to export data into S3 for storage or downstream querying, you can skip transformations and go straight to the target node.
- For the target, select Amazon S3 and specify the output path where your Salesforce data will be stored. Choose a format that aligns with your analytics needs—Parquet is often a good choice due to its compact size and query efficiency. You can also configure partitioning, compression (e.g., Snappy or Gzip), and schema output settings.
- Before running the job, review the Job details section. Assign a meaningful name to the job, select an IAM role with access to Salesforce and S3, and choose the appropriate worker type and number of DPUs based on expected job size. Optionally, enable job bookmarks to support incremental loading—useful when tracking changes via the SystemModstamp field in Salesforce.
Conclusion
AWS Glue provides a robust and flexible solution for integrating Salesforce data into your AWS ecosystem.
This low-code, visual approach empowers data engineers and analysts alike to securely extract, transform, and load Salesforce data without managing infrastructure or writing Spark code. Whether you're building a customer 360 view, feeding a data lake, or preparing CRM data for machine learning, Glue Studio offers the flexibility and scalability you need to succeed.
By connecting Salesforce to AWS Glue, you ensure your most valuable business data becomes a seamless part of your analytics strategy.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article