Modern Data Pipelines with KIRO: Architecture, Optimization, and Analytics
Explore how to design, tune, and leverage high-performance data pipelines using KIRO to drive actionable insights. Learn practical architectural patterns and optimization techniques straight from a presentation delivered at the AWS Office last July 24, 2026.
I am deeply grateful for the invitation from KIRO Ambassador and AWS Community Builder James Torzar to speak at KIROVERSE Week 6: Data Pipeline & Analytics Dashboard with KIRO. Delivering my very first workshop session at the AWS Office last July 24, 2026, was a rewarding milestone.
It was inspiring to present to over 80 participants, including students, industry professionals, and career shifters, all eager to learn and master KIRO.
80 Participants comes from Student, Professional and Career-shifters
As an active Tech Contributor for Data Engineering Pilipinas and AWS User Group Philippines, this session was a personal commitment fulfilled. As an applicant for the AWS Community Builder program under the Data category, giving back through technical workshops like this serves as concrete proof of my dedication to the community as I prepare for the next application cycle next year.
Using Medallion Architecture in KIRO Workshop
Using the Medallion Architecture to demonstrate how modern data engineering works in practice, we walked through a complete end-to-end pipeline using a public dataset from Better Gov PH as our source:
- PRD-Driven Data Cleaning: Ingesting raw data and cleaning it strictly in alignment with Data Engineering Product Requirement Documents (PRD).
- Pre-Quality Validation: Executing rigorous checks to catch orphan records, anomalies, and data discrepancies before promotion.
- Post-Quality Validation & Indexing: Applying strategic indexing after data validation to drastically optimize query performance and cut execution times, ensuring cost efficiency so you do not run out of KIRO credits.
- Gold Phase Readiness: Curating validated, high-performance data in the Gold layer, ready for Data Scientists, AI/ML Engineers, and analytics platforms.
General Appropriations Act of 2026 Operational Dashboard
To bring these engineering concepts to life, I presented my work on the General Appropriations Act (GAA) of 2026. I showcased how clean data feeds directly into an operational dashboard, featuring:
- Department Spending Highlights: Identifying top departments with the largest expenditures.
- Granular Budget Line Items: Mapping granular appropriations for deep-dive tracking.
- Regional Budget Allocation Maps: Visualizing geographic distribution of government funds.
- Executive Analysis & Strategic Insights: Delivering C-suite ready reporting for FY2026 budget decision-making.
One of the participants presented his project on flood control during the session
A major highlight of the session was seeing the participants apply these concepts in real time. The KIRO moderators selected 5 participants to showcase their own projects using the Better Gov PH datasets. They built impressive dashboards covering critical public topics like flood control and regional analytics, perfectly demonstrating how strong data engineering underpins meaningful data visualization.
Certificate of Appreciation: Awarded by the KIRO Community for serving as a guest speaker.
At the end of the session, I was honored to be officially recognized by the KIRO Community for serving as a speaker for this event.
As a first-time workshop speaker, this was an unforgettable experience. While it was a huge milestone, it also gave me valuable insights into how I can continuously improve as a presenter for future KIRO events covering data engineering and data analytics. Building resilient, cost-effective data pipelines is where true business value begins, and I look forward to sharing more with the community soon!
Resources & Links
To explore the code, live dashboard, and presentation slides used during the workshop, feel free to check out the links below:
- GitHub Repository: https://github.com/CardinalSeen/kiro-ph-data-dashboard
- Live Dashboard (Vercel): https://kiro-ph-data-dashboard.vercel.app/?fbclid=IwY2xjawTJtktleHRuA2FlbQIxMABicmlkETE0WUNaTUlMb0psajl4U1l5c3J0YwZhcHBfaWQQMjIyMDM5MTc4ODIwMDg5MgABHvxKodn1BwCK0zMeKFWJmQ5198JrwAFRWK3U8dboKpGvL7qKJRRq_nH27HF4_aem_e5zzafSlCK9gt06YcWG4_Q
- Presentation Deck: https://www.canva.com/design/DAHQCJmgQho/rlmc0-nqKNGZh2FPCy-9aw/edit
- Books:
- Stephen Few - Information Dashboard Design: Displaying Data for At-a-Glance Monitoring
- Edward Tufte - The Visual Display of Quantitative Information
- Google's HEART / Lean Analytics Frameworks - Core Concept: Defining actionable metrics vs. vanity metrics in product and business analytics.
- Joe Reis & Matt Housley (2022), Fundamentals of Data Engineering: Plan and Build Robust Data Systems
- DAMA-DMBOK: Data Management Body of Knowledge (2nd Edition / DMBOK2)
- Martin Kleppmann (2017), Designing Data-Intensive Applications
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article