Secure and Govern AI Traffic with Multi-LLM AI Gateway

Revisit one of our sessions from API Summit 2024!

As the use of large language models (LLMs) in AI applications continues to grow, organizations are faced with the challenge of effectively managing and securing the influx of AI-driven traffic. In this breakout session, we'll explore a novel approach to building a multi-LLM AI gateway using Kong Gateway and Amazon Bedrock. The session will begin by discussing (along with practical demonstration and real-world use cases) the need for a robust and scalable solution to handle the complexities of AI-driven workloads. We'll then dive into the architecture and key features of the multi-LLM AI gateway, showcasing how Kong Gateway can be leveraged as the central control plane to route, secure, govern, and observe AI traffic from various LLM providers, including Amazon Bedrock.