New providers, smarter routing, stronger guardrails — because AI infrastructure should be as robust as APIs

We know that successful AI connectivity programs often start with an intense focus on how you govern and protect LLM and MCP traffic. This is why Kong was first to market with an enterprise-grade AI gateway, and it’s why, in our last release, we added enterprise MCP proxy support to our already-existing LLM proxy in the Kong AI Gateway.

And we’re still innovating.  

Today, we're proud to announce one of the most significant updates to Kong AI Gateway yet. This release is all about helping you move from “we’re experimenting with LLM and MCP-powered workflows” to “let’s ship our first production-grade agent to market,” with a special focus on the hard problems to solve around agentic security, dev productivity, and resilience. 

Kong AI Gateway 3.13 unlocks a wave of new capabilities that power enterprise-grade AI infrastructures — from MCP tool-level access control and circuit breakers to expanded provider support, advanced generation workflows, and load balancing improvements. If you’re building multi-model, agentic, real-time, or mission-critical AI systems, you’ll want to upgrade.

The full changelog can be found here.

With the rise of agent architectures and multi-tool workflows, controlling what tools an agent (or user) can call — and when — is essential. Since the rise of MCP, organizations (and often different teams within organizations) have approached this problem in different, often non-standardized ways. Kong AI Gateway solves for this, giving AI and Infra teams the ability to standardize how their org:

• Filters MCP tools: Restrict which tools or capabilities are exposed to an agent or consumer by default.
• Enforces access control at the MCP layer: Use ACLs to enforce least-privilege access across MCP servers.
• Dynamically manages ACL configuration: ACLs can be managed via declarative config (e.g., decK / Terraform) or via the control plane.

Let’s consider exposing an MCP interface for fictitious airline KongAir's API. In the example configuration below, we can ensure that only users in the booking-agents consumer group will be allowed to consume KongAir MCP tools by default. Users that have been mapped to the developers consumer group will only be permitted access to the tools that allow flight details to be fetched. 

These features help you ensure that AI traffic through the Gateway remains secure, governed, and compliant — even in highly dynamic, multi-agent environments.

Say goodbye to lock-in. This release adds first-class support for several new providers:

• Anthropic (native SDK support) — optimized for code generation, developer tooling, and agent-driven code workflows.
• xAI / Grok — enabling reasoning workloads and inference use-cases at scale.
• Aliyun (Alibaba Cloud / Qwen) — ideal for deployments in Asia-Pacific with compliance and low-latency needs.
• Cerebras — optimized large-model inference endpoints.

With these additions, Kong AI Gateway becomes a truly multi-cloud, multi-model AI platform — giving teams flexibility to choose providers based on cost, compliance, performance, or capability.

We're excited to empower organizations to govern a tool that nearly every developer uses, Anthropic’s Claude Code. With one line of configuration, access to Claude can be secured, governed, and observed end-to-end–finally adding that governance layer to Claude-powered dev productivity.

Views the docs here.

AI infrastructure isn’t just about performance and scale — safety and compliance matter. That’s why this release brings built-in integration with Lakera.ai guardrails. With AI Gateway 3.13, you can:

• Enforce content safety, block unsafe or toxic generations, and prevent leakage of PII or sensitive data.
• Apply guardrails uniformly, regardless of the underlying model provider.
• Seamless integration: no code changes required — policies enforced at the gateway layer.

Combined with existing plugins (such as prompt guard, PII sanitizer, and semantic caching), this makes Kong AI Gateway ideal for enterprise deployments where safety and compliance are non-negotiable.

Learn how to integrate with Lakera here.

Modern AI workflows are complex. Think: batch inference, file-based generation, multi-modal inputs (text, image, audio), streaming outputs, and long-lived agent loops. This release expands support for:

• Batch operations — efficient parallel inference for large workloads
• File-based generation / file uploads — supporting document processing, ingestion, and rich context workflows across every provider that offers file-based operations.
• Multi-modal support — mixing text, images, audio, whatever your use-case needs
• Real-time / streaming APIs — we’ve long supported the OpenAI Realtime API, and with this release, we’re adding Gemini Live, expanding real-time and event-driven capabilities for chatbots, live agents, voice assistants, and continuous streaming applications.

All providers (old and new) that support these advanced text generation patterns (old and new) can now benefit from a unified, consistent interface — reducing complexity and speeding up development.

The support matrix can be viewed here.

AI workloads increasingly demand high availability, low-latency scaling, and resilience. This release adds:

• Native circuit-breaker support — automatically detect and shed traffic from unhealthy or failing upstreams to prevent cascading failures
• Enhanced semantic load balancing — extended to support classification-group aware failover, so routing maintains semantic correctness when managing fallback or failover logic
• Least connections algorithm — ideal for long-lived streaming, real-time, or high-concurrency workloads

With these enhancements, Kong AI Gateway strengthens its role as a purpose-built load-balancing and traffic-management layer for GenAI and agent workloads.

AI systems have moved beyond single-model, one-off requests. What organizations need now is infrastructure that handles:

• Multi-model strategies (vendor diversification, specialization per provider)
• Agentic workflows with complex tool interactions
• Real-time, streaming, multi-modal generation at scale
• Safety, compliance, and governance — without slowing down innovation
• Resilience, failover, and performance under load

This release empowers you to build systems that meet — and exceed — those needs.

Ready to try out the new release of Kong AI Gateway? You can get started for FREE with Konnect Plus. If you already have a Konnect account, visit the official product page or dive straight into the demos and tutorials.

If you want to learn more, check out the updated docs, provider blueprints, and example configurations on the website. Declarative config (decK / Terraform) tooling is already updated with new plugin settings for ACLs, guardrails, load balancing, and provider configuration.

Ready to build safer, smarter, multi-model AI infrastructure at scale? Dive in today and let us know what you build.