GIDS news media, articles, insights and virtual events educate and illuminate its audiences so they can be fully prepared to deal with the new realities at work and in their professions.
Get the latest event updates, and insights from today's leading voices.
< session />
Fri, April 30Reliability, Observability & SecurityAI-Native Software
Once agentic systems reach production, the model quickly stops being the hardest engineering problem. The greater challenge is understanding why the system made particular execution decisions, observing how its behaviour changes over time, and establishing the operational controls needed to keep adaptive agent systems reliable, inspectable, and safe.
In this session, you will explore what it takes to operate agentic software in production by looking beyond whether the final answer was correct to the runtime decisions that produced it. You will examine why a system selected a particular tool, plan, memory, model, handoff, or recovery strategy, and how engineering teams can evaluate those decisions and detect changes in behaviour. The session covers execution traces, evaluation signals, policy evolution, safety boundaries, observability, and debugging practices, with the model treated as one component within a larger agent harness. You will leave with a practical framework for deciding what to observe, what to evaluate, where runtime controls belong, and how to keep adaptive behaviour inspectable, governable, and understandable as systems evolve.
What You Will Learn
How to observe and evaluate runtime decisions made by intelligent software systems
How execution traces, evaluation signals, observability, and runtime controls support production operations
How to keep adaptive system behaviour inspectable, governable, and understandable over time
Who Should Attend
AI Engineers, Platform Engineers, SREs and Reliability Engineers, Software Architects, Staff and Principal Engineers, Technical Leads, and teams responsible for operating intelligent software in production.
< speaker_info />
Nicole Koenigstein is an AI Researcher and Practitioner in Agentic Systems, working across research, consulting, teaching, and direct system implementation to build reliable, production-ready AI systems. Her work focuses on multi-agent architectures, evaluation, safety, and long-term system behavior.
She served as an external evaluator for a European Commission AI Grand Challenge and has advised IOSCO on generative AI in regulated environments. She also serves on advisory boards for leading AI and quantitative finance conferences. Nicole regularly delivers invited talks and technical workshops across academia, industry, and international events. She is the author of Math for Machine Learning and Transformers in Action with Manning Publications. Her books Transformers: The Definitive Guide: Applications Beyond NLP and AI Agents: The Definitive Guide have been published by O’Reilly Media, and her forthcoming book Harness Engineering for AI Agents will also be published by O’Reilly Media.
