# How Enterprises Can Architect Generative AI Apps for Production Scale

This session explains why generative AI applications often work in proof-of-concept or staging environments but fail in production, and what enterprise teams can do to fix that gap. Harshit Kohli argues that the real issue is often not the model itself but the surrounding API architecture: AI agents behave very differently from human users, generating bursty, parallel traffic, hitting many downstream APIs, and creating “thundering herd” and authentication-sprawl problems through short-lived tokens and repeated authorization checks. He outlines production patterns designed for these workloads, including an AI-aware API gateway, chain-aware rate limiting, token broker and cache, request classification, circuit breakers, and stronger observability around full agent chains rather than single requests. The talk also covers real-time data pipelines using Kafka and stream processors, policy-as-code governance, and readiness stages for deployment. The main advice is to start with API behavior, design for 10x expected traffic, make governance operational, and plan for ongoing model evolution.