The Semantic Foundation — Why Your Data Agent Needs More Than RAG
The promise of "ask your data anything" has been around for years. The reality? Most implementations hallucinate metrics, confuse business terms, and break the moment questions get domain-specific. Explore why the semantic layer — not the model — is the missing piece, and see how AutoScout24 built a production-grade data agent (Omnio) that answers "why" questions across 13 data domains using governed context, ontologies, and evaluation-gated releases
This episode of the Let's Talk About Data Show, hosted by Lakshmi Nair (Principal Specialist Solutions Architect at AWS), featured guests Aleksandar Marjanovic (Senior Engineering Manager at AutoScout24, Berlin) and Norman Weisenburger (Senior Solutions Architect at AWS, working with retail and consumer goods customers in Germany). The discussion centered on a critical evolution in data agents: the shift from answering "what" questions to answering "why" questions — multi-domain investigations that require reasoning across data boundaries. Norman explained that RAG alone gets you ~30% accuracy on real business questions; the remaining 70% requires structured semantic context.
Norman introduced the concept of the Semantic Spine. He highlighted that semantic meaning is not a single layer but is scattered along the entire stack: schema descriptions and metric layers at the data substrate (how to compute), ontologies at the tools/retrieval layer (what things are and how they relate), and enterprise context at the agent layer (how to think about the business). He demonstrated this via live whiteboard, showing a three-tier architecture: data substrate (gold-layer curated data), tools & retrieval (SQL, search, graph traversals, APIs — 95%+ via MCP servers), and agent layer (multi-agent orchestration with routing).
Aleksandar shared AutoScout24's journey building Omnio — a conversational data agent that sits on top of their data platform. They started with simple YAML files capturing business knowledge ("moving people's brains to YAML"), graduated to Postgres + PGVector when governance became the bottleneck, and now use LLMs to detect contradictions across context documents. Their biggest accuracy breakthrough came from reverse-engineering their five most critical QuickSight dashboards into deterministic query packs — a "dashboard ontology" that ensures consistent answers. The team emphasised that evaluations are not optional: every prompt change, model swap, or context addition runs against a benchmark suite of real business questions, and failures become new test cases that gate future releases.
Bullet Point Highlights
- "What" vs "Why" questions — "What was revenue?" is a single SQL query. "Why did revenue drop?" is a multi-domain investigation crossing sales, traffic, marketing, and campaign data — requiring the agent to reason like a human analyst using the five-whys technique
- Semantic Spine (not layer) — Semantic meaning is distributed across the full stack: metric layer (bottom), ontology (middle), enterprise context (top). Not a single component — a spine running through the architecture
- Metric Layer — Predefined KPIs with encapsulated computation, dimensions, and descriptions. ~80% of queries are repetitive and answerable from metrics/templates without generating SQL from scratch
- Ontology — Formal specification of domain concepts (like OOP class definitions). Defines entities, attributes, relationships, constraints. Example: "supplier" vs "vendor" vs "partner" — humans clarify ambiguity, LLMs just execute with whatever meaning they pick
- Enterprise Context Layer — Documents specifically designed as agent context: how the business works, how domains relate, company goals. Not general documentation but purpose-built for LLM consumption
- Start with YAML — AutoScout24's first ontology was not a graph database, it was business knowledge written in YAML files. The hard part was not technology; it was getting people to agree on definitions
- Governance = data mesh for context — Domain owners responsible for their agent subdomains. Context organized hierarchically: company → domain → team → technical metadata. No uncontrolled document buckets in production
- AI-assisted governance — LLMs detect contradictions across context documents ("you defined 'lead' differently here vs there") and surface them as alerts
- Dashboard Ontology (biggest accuracy unlock) — Reverse-engineered five critical Amazon QuickSight dashboards into deterministic query packs: data sources, filters, columns, visuals mapped to backend queries. Consistent answers because they follow deterministic paths
- Evaluations from day one — Not technical benchmarks but real business question protection. Offline benchmarks after every change + runtime evaluation (prompt quality, routing verification, metadata-based reliability scoring)
- Feedback → test case → release gate — User failures become benchmark cases. Every serious failure becomes a test. The evaluation suite is the product specification
- Deterministic + LLM-as-judge evaluation — Mix of hard checks (right tool? right scope? right model?) and AI-based checks (prompt quality, routing, answer coherence)
- Race car analogy — "A race car isn't fast because of one component. Engine, tires, brakes, telemetry, and driver work together. Omnio is the visible AI experience; the real performance comes from the platform underneath."
- Two weeks to first users — Basic version with 10 users giving feedback in under two weeks. Day one it didn't know the date; now it's a production product
- Architecture — AWS Lambda (99% of conversations) + Fargate (long-running). Bedrock + OpenAI APIs. Postgres + PGVector. 95%+ tools via MCP servers
- Key insight — "The problem is no longer the model. Data platforms do not carry enough meaning for the model to reason correctly."
- The difference between demo and production — Not the model. It's the data foundation, governed context, and evaluations that keep the system honest
Hosts of the show 🎤
Lakshmi Nair - Principal Specialist Solutions Architect @AWS
Guests 🎤
Aleksandar Marjanovic - Senior Engineering Manager@AutoScout24
Norman Weisenburger - Senior Solutions Architect @AWS
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article