< session />
Building Adaptive ETL Pipelines with Apache NiFi, LLMs, and Apache Iceberg
Tue, 21 April
Organizations repeatedly face the same data integration challenge: many data sources, inconsistent schemas, and extensive manual effort to transform data into analytics-ready formats. This session explores an alternative approach using open-source tooling and AI to reduce that effort significantly.
In this hands-on session, you will build intelligent data pipelines that automatically transform inconsistent data uploads into unified, analytics-ready tables. The session demonstrates how Apache NiFi can be used for orchestration, how LLM-powered schema inference can automate schema mapping and validation, and how Apache Iceberg’s schema evolution capabilities allow pipelines to adapt to changing data structures while preserving backward compatibility. The focus is on practical implementation and real-world architecture patterns suitable for enterprise-scale environments.
What You will Learn
-
How to build self-adapting data pipelines that infer semantic meaning across different schemas
-
Practical techniques for integrating LLMs into data pipelines for automated schema mapping and validation
-
How Apache Iceberg schema evolution supports compatibility while adapting to new data structures
Who Should Attend
-
Data Engineers
-
Platform Engineers
-
Analytics Engineers
-
Software Architects working on data platforms
< speaker_info />
About the speaker
Kamesh Sampath
Lead Developer Advocate, Snowflake
Kamesh is a veteran tech innovator, author, and Lead Developer Advocate at Snowflake, India, with over 20 years in the trenches and more than a decade of open source contributions. As an author and developer advocate, he’s on a mission to demystify data engineering, cloud architecture, and emerging AI technologies. His expertise spans data cloud platforms, serverless computing, and distributed systems.
Passionate about empowering developers to harness the power of cutting-edge tools and frameworks, Kamesh has been actively contributing to the open source landscape for more than 10 years. With a track record of crafting robust enterprise solutions across diverse industries and participating in community-driven projects, he brings battle-tested insights on everything from integrating AI into enterprise data ecosystems to building scalable cloud-native architectures.
Kamesh is always eager to collaborate, innovate, and push the boundaries of what’s possible in tech, bringing the power of AI to enterprise data challenges.
