More easily manage AI spend, build AI agents and chatbots, get real-time AI responses, and ensure content safety
We're introducing several new Kong AI Gateway capabilities in Kong Gateway 3.7 and Kong Gateway Enterprise 3.7, including enterprise-only and OSS improvements. Read on for a full rundown of the new AI-focused features.
With the new Kong Gateway 3.7 release, we're promoting Kong AI Gateway to GA status.
Starting today, AI developers can now focus on building AI-specific use cases — like LLM RAG chatbots or AI integrations — without having to build the underlying infrastructure to establish a secure and observable lifecycle for AI applications in production. This is fully supported by Kong at scale on both Kong Konnect and Kong Gateway Enterprise.
Kong AI Gateway can also be provisioned entirely in the cloud as a dedicated SaaS service with Kong’s new Konnect Dedicated Cloud Gateways offering.
Kong AI Gateway supports a wide range of use cases to help accelerate the adoption and rollout of new AI applications into production.
Kong AI Gateway provides one API to access all of the LLMs it supports. To accomplish this, we've standardized on the OpenAI API specification. This will help developers to onboard more quickly by providing them with an API specification that they’re already familiar with.
In this new release, we're making it even easier to build AI agents and applications using Kong AI Gateway by natively supporting the OpenAI SDK client library. You can start using LLMs behind the AI Gateway simply by redirecting your requests to a URL that points to a route of the AI Gateway.
If you have existing business logic written using the OpenAI SDK, you can re-use it to consume every LLM supported by Kong AI Gateway, removing the need to alter your code, given it will be 100% compatible.
Streaming in the “ai-proxy” plugin when consuming every LLM provider is now natively supported by Kong AI Gateway. This unlocks more real-time experiences, rather than having to wait for the full response to be processed by the LLM before sending it back to the client.
The response will now be sent token-by-token in HTTP response chunks (SSE). The capability can be enabled in the plugin configuration by setting the following property of “ai-proxy”:
Which then allows the clients to request streaming by making requests like:
With this capability, Kong AI Gateway users can create more compelling and interactive AI experiences.
We're introducing a new enterprise-only AI capability to rate-limit the usage of any LLM by the number of request tokens. By enabling the new “ai-rate-limiting-advanced” plugin, customers can better manage AI spend across the board by specifying different levels of consumption for different teams in the organization. For self-hosted LLM providers, customers will be able to better scale their traffic on the AI infrastructure when the AI traffic increases across the applications.
Kong already provides API rate-limiting capabilities which rate-limits based on the number of requests that are being sent to an API. The new ai-rate-limiting-advanced plugin instead focuses on the number of AI tokens requested, regardless of the number of raw HTTP requests being sent to it. If the customer wants to rate-limit both raw requests and AI tokens specifically, the ai-rate-limiting-advanced plugin can work in combination with the standard Kong rate-limiting plugin.
The ai-rate-limiting-advanced plugin is the only rate-limiting plugin available today for AI.
The new enterprise plugin “ai-azure-content-safety” allows customers to seamlessly integrate with the Azure AI Content Safety service to validate prompts that are being requested via the AI Gateway across every LLM supported (not only Azure AI).
For example, the customer may want to detect and filter out all violence, hate, sexual, and self-harm content across all prompts sent to any LLM provider in Kong AI Gateway using Azure’s native services.
It's now possible to configure the requested model dynamically via the URL path requested by the client. Additionally, users can consume a model by hard coding its name in the plugin configuration. By enabling this capability, it becomes easier to scale Kong AI Gateway across the teams that want to experiment with a wide variety of models, without having to pre-configure them in the “ai-proxy” plugin.
By allowing “ai-proxy” to set up the LLM route using the URL requested by the client, it's possible to apply the “ai-proxy” plugin once and then support all models available by the underlying AI provider by parsing the URL path requested instead.
This capability can be configured with the new “config.route_source” configuration parameter in “ai-proxy”.
Kong AI Gateway provides one API interface to consume models across both cloud and self-hosted providers. We've expanded our unified API interface to also support the Anthropic Claude 2.1 Messages API typically used to create chatbots or virtual assistant applications. The API manages the conversational exchanges between a user and an Anthropic Claude model (assistant).
Kong AI Gateway will continuously add support for more LLMs and models based on user demand.
With Kong AI Gateway going into GA, we've updated our analytics logging format for all AI requests processed by Kong.
With this new logging format, we can now measure consumption across every model that has been requested by “ai-proxy,” “ai-request-transformer,” and “ai-response-transformer.”
This new analytics log format replaces the old one.
Get started today with Kong AI Gateway and accelerate the rollout of AI applications in production in a secure, observable, and scalable way.
In the latest Kong Gateway 3.12 release , announced October 2025, specific MCP capabilities have been released: AI MCP Proxy plugin: it works as a protocol bridge, translating between MCP and HTTP so that MCP-compatible clients can either call exi
MCP ACLs, Claude Code Support, and New Guardrails New providers, smarter routing, stronger guardrails — because AI infrastructure should be as robust as APIs We know that successful AI connectivity programs often start with an intense focus on how
For the last couple of years, Retrieval-Augmented Generation (RAG) architectures have become a rising trend for AI-based applications. Generally speaking, RAG offers a solution to some of the limitations in traditional generative AI models, such as
APIs have quietly powered the global shift to an interconnected economy. They’ve served as the data exchange highways behind the seamless experiences we now take for granted — booking a ride, paying a vendor, sending a message, syncing financial rec
Companies are charging headfirst into AI, with research around agentic AI in the enterprise finding as many as 9 out of 10 organizations are actively working to adopt AI agents. LLMs are being deployed, agentic workflows are getting created left
The Requirement : Article 10 of the EU AI Act mandates strict data governance for high-risk AI systems. This includes error detection, bias monitoring, and arguably most critically for enterprise use — ensuring that sensitive personal data (PII) is