GIDS news media, articles, insights and virtual events educate and illuminate its audiences so they can be fully prepared to deal with the new realities at work and in their professions.
Get the latest event updates, and insights from today's leading voices.
< session />
Tue, April 21DataTech OpsTech Architecture
Organizations repeatedly face the same data integration challenge: many data sources, inconsistent schemas, and extensive manual effort to transform data into analytics-ready formats. This session explores an alternative approach using open-source tooling and AI to reduce that effort significantly.
In this hands-on session, you will build intelligent data pipelines that automatically transform inconsistent data uploads into unified, analytics-ready tables. The session demonstrates how Apache NiFi can be used for orchestration, how LLM-powered schema inference can automate schema mapping and validation, and how Apache Iceberg’s schema evolution capabilities allow pipelines to adapt to changing data structures while preserving backward compatibility. The focus is on practical implementation and real-world architecture patterns suitable for enterprise-scale environments.
What You will Learn
How to build self-adapting data pipelines that infer semantic meaning across different schemas
Practical techniques for integrating LLMs into data pipelines for automated schema mapping and validation
How Apache Iceberg schema evolution supports compatibility while adapting to new data structures
Who Should Attend
Data Engineers
Platform Engineers
Analytics Engineers
Software Architects working on data platforms
< speaker_info />
Kamesh is a veteran tech innovator, author, and Lead Developer Advocate at Snowflake, India, with over 20 years in the trenches and more than a decade of open source contributions. As an author and developer advocate, he’s on a mission to demystify data engineering, cloud architecture, and emerging AI technologies. His expertise spans data cloud platforms, serverless computing, and distributed systems.
Passionate about empowering developers to harness the power of cutting-edge tools and frameworks, Kamesh has been actively contributing to the open source landscape for more than 10 years. With a track record of crafting robust enterprise solutions across diverse industries and participating in community-driven projects, he brings battle-tested insights on everything from integrating AI into enterprise data ecosystems to building scalable cloud-native architectures.
Kamesh is always eager to collaborate, innovate, and push the boundaries of what’s possible in tech, bringing the power of AI to enterprise data challenges.
