Building a Student 360 View with SageMaker Data Agent, Databricks Unity Catalog and S3 Tables -- Without Moving a Single Row of Data
Universities can build a Student 360 view, combining student information system data in Databricks Unity Catalog with learning management data in Amazon S3 tables — without moving or duplicating data. Using catalog federation and Amazon SageMaker Unified Studio, institutions can query across federated datasets through natural language with Data Agent, enabling early intervention, improved retention, and personalized learning at scale.
What is Student 360?
In higher education, student data is rarely in one place. Records live in student information systems, learning activity sits in LMS platforms, and engagement data is scattered across multiple tools. The result is a fragmented picture that makes it difficult for institutions to act quickly and confidently.
A Student 360 view changes that by bringing together data from across the institution into a single coherent model, including academic history, course engagement, assessment results, support interactions, and more. The outcomes this view can drive are significant. It enables earlier identification of at-risk students, improved retention, personalized learning pathways, and richer institutional research.
Historically, building this view required substantial data engineering effort to extract, transform, and load data into a central store, keep that data updated, and onboard any new data sources as they emerged. That approach is expensive, introduces latency, and creates governance challenges around duplication and lineage. With modern data catalog federation, none of that is necessary. The data stays where it is, and the unified view comes to you.
Federation Without Duplication
To see how this works, we are going to walk through an example of how this data might come together in practice using synthetic data. In this scenario, a university has two primary data sources.
The first is a Databricks schema called student_info that contains a table called student_details. This table is sourced from the university's Student Information System (SIS) and holds the authoritative record of each student, including their identifiers, enrolment status, program of study, demographic information, and other administrative attributes.
The second data source consists of tables stored in Amazon S3 Tables , sourced from the university's Learning Management System (LMS), which is also running on AWS. These tables live in a schema called student_lms and include two tables. The course_enrollments table holds enrollment identifiers, course codes, course names, semesters, credit points, and enrollment status. The learning_activities table captures granular student interactions with course materials, such as logins, content views, and assessment events.
Because catalog federation is in place, both the Databricks-managed data and the S3 Tables are visible within Amazon SageMaker Unified Studio and can be used together for analysis. There is no ETL pipeline copying data between systems. The data stays exactly where it is. What changes is the ability to query and reason across it as if it were one coherent dataset.
Note: This walkthrough assumes that catalog federation between Databricks Unity Catalog and the AWS Glue Data Catalog has already been configured in AWS Lake Formation . If you are setting this up for the first time, there are resources provided at the end of this post to help. The federation setup establishes a link to the Databricks Unity Catalog so that metadata is shared and the data can be queried in SageMaker Unified Studio without requiring any physical movement of the underlying data.
Surfacing the Data in Amazon SageMaker Unified Studio
SageMaker Unified Studio provides a collaborative environment where data engineers, analysts, and data scientists can work together. With catalog federation in place, navigating to the data catalog within SageMaker Unified Studio will surface both the federated Databricks catalog and the Glue-registered S3 Tables in the same interface, as shown below.
Analysts can browse student_info.student_details (from Databricks) alongside student_lms.learning_activities and student_lms.course_enrollments (from S3 Tables) without needing separate credentials, separate tools, or any awareness of where the data physically resides.
This unified catalog view is the foundation for everything that follows. When a Data Agent is asked a question about student performance, it can reason across all three tables simultaneously, joining on the common student_id key that links the student information system records to the learning management system records.
Using Data Agent to Build the Student 360 View
Data Agent in Amazon SageMaker Unified Studio (IAM domain) is a conversational interface that allows users to interact with their data using natural language. Rather than requiring analysts to write complex SQL queries from scratch, Data Agent interprets intent, understands the schema of the available tables, and generates the appropriate queries to answer the question.
For a Student 360 use case, this is particularly powerful because it lowers the barrier to insight for non-technical stakeholders such as academic advisors, student success managers, and faculty.
The following sections walk through a series of example prompts, starting with understanding the dataset and progressing through to the kinds of analysis that drive real Student 360 outcomes.
To get started with SageMaker Data Agent, open a notebook in SageMaker Unified Studio (IAM Domain) and follow these steps:
- Open the agent chat panel from the notebook interface.
- Enter a prompt describing your data analysis needs.
- Review the agent's plan and suggested approach.
- The SageMaker Data Agent will then generate a collection of cells to implement the analysis plan.
- Review and execute the generated cells as needed.
Note: If you run a cell and encounter an error, there is a "Fix with AI" button at the bottom of the notebook. This will correct any errors in the cell and allow you to re-run it.
With that in place, let's get into the prompts.
Phase 1: Understanding the Dataset
Before performing any meaningful analysis, it is worth using Data Agent to understand the structure and content of the available tables. This is especially useful when working with federated data sources where schema documentation may be incomplete or spread across different systems.
Prompt: "What tables are available to me in the student_info and student_lms schemas, and what are the key columns in each?"
This prompt asks Data Agent to describe the schema landscape. It will return a summary of student_info.student_details, student_lms.learning_activities, and student_lms.course_enrollments, including column names, data types, and any available descriptions. This gives the analyst an immediate understanding of what they are working with before writing a single query.
Prompt: "Using Spark SQL, how many students are in the student_info table, and what is the breakdown by program of study?"
You will notice in this example, we are explicitly specifying in the prompt to use Spark SQL. This choice is deliberate. Spark SQL provides a unified query interface that can seamlessly access data across multiple federated Iceberg data stores without requiring separate connections or complex data movement operations. By leveraging Spark SQL's distributed processing capabilities, we can query large-scale datasets efficiently while maintaining consistency across heterogeneous data sources.
This is a simple exploratory prompt that establishes the population size and composition. It helps analysts understand whether the data is complete and representative across all federated sources, and gives a baseline for subsequent analysis. This foundational step is particularly valuable when working with distributed data ecosystems, as it ensures you have full visibility into the scope of your data before proceeding with more complex analytical queries.
Prompt: "Show me a sample of 10 rows from the learning_activities table so I can understand what a typical record looks like."
Understanding the granularity of the LMS activity data is important before performing any aggregation. This prompt surfaces the raw structure of the data, whether activities are recorded at the session level, the content item level, or the event level, which informs how subsequent analysis should be framed.
Prompt: "Are there any students in the student_lms tables that do not have a corresponding record in the student_details table? How many are there?"
Data quality and referential integrity checks are an essential part of working with federated data from multiple source systems. This prompt asks Data Agent to perform a left join analysis to identify orphaned records, which might indicate data ingestion issues, students who have withdrawn, or mismatched identifiers between systems.
Prompt: "What is the date range of activity records in the learning_activities table, and how many distinct students have at least one activity recorded?"
This prompt establishes the temporal scope of the LMS data and confirms the active student population within it. It is a useful sanity check before building any time-based analysis.
Phase 2: Building the Student 360 Analysis
With a solid understanding of the dataset in place, the analysis can move toward the kinds of questions that drive genuine Student 360 outcomes. The prompts below reflect the types of questions that student success teams, academic advisors, and institutional researchers typically need to answer.
Prompt: "Using Spark SQL, create a combined dataset with the information from the student_info database and the student_lms database that could be used for Student 360 analysis."
This is the foundational Student 360 query. It brings together identity and enrolment context from the student information system with engagement and performance signals from the LMS. The result is a single row per student that captures the essential dimensions of their experience. Critically, this query is executing across federated data sources, the Databricks-managed table and the S3 Tables, without any prior data movement.
Prompt: "Using the combined dataset, which students have not logged any activity in the learning_activities table in the last 14 days but are still enrolled as active students?"
This prompt targets one of the most common early warning indicators for student disengagement. A student who has gone quiet in the LMS is often a student who is struggling. By combining enrolment status from the student information system with recency of activity from the LMS, this query surfaces a list of students who may benefit from proactive outreach from an advisor.
Prompt: "Show me a graphical distribution of course result scores for each program of study. Which programs have the highest proportion of students scoring below 50?"
This prompt supports program-level analysis, helping academic quality teams identify where students are struggling at scale. The program context it provides is what makes the results actionable rather than simply informative.
Prompt: "For students who have failed at least one course, what is their average weekly LMS activity count compared to students who have passed all courses?"
This is a comparative analysis that begins to explore the relationship between engagement behavior and academic outcomes. The insight it generates, whether high-activity students are still failing or whether low activity is a reliable predictor of failure, can inform both curriculum design and the thresholds used in early alert systems.
Prompt: "Identify students who are in their first year of study, have completed fewer than 20 LMS activities this semester, and have an average course result below 55. Rank them by lowest average result."
This prompt operationalizes a multi-factor at-risk model. By combining year of study from the student information system, engagement volume from the LMS activities table, and academic performance from the course results table, it produces a prioritized list of students for advisor intervention. This is exactly the kind of query that, without catalog federation, would have required a data engineer to build and maintain a dedicated pipeline.
Prompt: "What are the top 10 most accessed learning resources or activities in the LMS, and how does engagement with those resources correlate with course result scores?"
This prompt shifts the lens from students to content, asking which learning materials are most used and whether engagement with them is associated with better outcomes. It is the kind of analysis that supports learning design decisions and helps institutions understand which resources are genuinely driving value.
Prompt: "For each student, calculate an engagement score based on their total LMS activities and recency of last activity, and combine it with their average course result to produce a composite Student 360 summary. Flag students where either the engagement score or the academic score falls below the bottom quartile."
This is a more sophisticated prompt that asks Data Agent to construct a composite metric. It demonstrates how conversational AI can be used not just to retrieve data, but to perform multi-step analytical reasoning across a federated dataset, producing outputs that are directly usable by student success teams.
Prompting Tips for SageMaker Data Agent
When prompting, be specific and clear because the more specific your prompt the better the results. Also provide context around your data, including schema, table or column names if you know them, and clearly define any metrics or calculations you want to be calculated or aggregated. A good strategy is to use iterative prompting, where you start broad and refine your prompts as you go along. You can see a bit of that strategy in the example prompts provided above.
In addition, you can also use the @ command in the prompt to reference an existing Data Frame to provide context. You can find the Data Frame name at the bottom left of the cell in your notebook, and then use the @ command to reference it.
For multi-step tasks, breaking your request into numbered steps tends to produce better results. For example, you may want to join a data set together, then do some filtering, then summarization and then statistical analysis. Structuring your prompt into numbered steps will help the agent understand the flow and work more efficiently.
If you are prompting to create a chart or graph, be explicit and name the chart type you want, such as a bar chart, line graph, or heatmap. Specify what should appear on each axis, for example "x-axis should be month and y-axis should be total LMS activities". You can also request color coding or grouping when it would make the output easier to read.
Also, telling the agent how to handle edge cases upfront saves time. Mention how you want missing values treated, for example "treat null values as 0" or "exclude rows with missing data". Specify whether deduplication is needed, such as "remove duplicate entries based on enrollment_id". If your data contains ambiguous date formats, clarify those as well.
The Key Point: No Data Was Moved
It is worth pausing to emphasize what has happened across the analysis above. Every query, from the simple schema exploration through to the multi-table Student 360 aggregation, has been executed against data that remains in its original location. The student_details table continues to live in the Databricks-managed catalog. The learning_activities and course_enrollments tables continue to live in Amazon S3 Tables, registered in the Glue Data Catalog. No ETL pipeline was built. No data warehouse was populated. No copies were made.
Catalog federation means that the metadata, including schema definitions, table locations, and column types, is shared across catalog boundaries, and the query engine handles the cross-system joins at execution time. For universities, this has meaningful implications. The authoritative source of record for student data remains the student information system, governed by the teams responsible for it. LMS data remains under the governance of the teams who manage that platform. And when either dataset is updated, the Student 360 view reflects those updates immediately, without any synchronization lag.
Conclusion
Catalog federation between Databricks Unity Catalog and the AWS Glue Data Catalog, surfaced through Amazon SageMaker Unified Studio and made accessible through Data Agent, represents a meaningful shift in how universities can approach the challenge of unified student data. The Student 360 view that once required months of data engineering effort can now be assembled through natural language queries against federated metadata, with the underlying data staying exactly where it belongs.
The outcomes this enables, including earlier identification of at-risk students, more personalized support, better program quality insights, and richer institutional research, are not new aspirations for universities. What is new is the ability to pursue them without the overhead, latency, and governance complexity that data duplication has historically imposed.
Related Resources
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article