Today we're announcing Kong Operator 2.3. Building on 2.2's expansion into supporting Event Gateway and Dev Portal, this release significantly broadens our feature set. While 2.2 introduced foundational event-driven capabilities, 2.3 brings Kong **AI Gateway** fully into the Kubernetes-native fold, enabling teams to manage LLM routing and policies through familiar GitOps workflows. This update also rounds out Gateway API coverage with `TCPRoute`, `UDPRoute`, and `GRPCRoute`.
We have full Kubernetes CRD support for Kong AI Gateway. You can now declare an entire AI Gateway deployment; control plane, providers, models, policies, authentication, and consumers as Kubernetes resources, and manage it with the same GitOps workflows you already use for the rest of your Kong estate.
A `KonnectAIGateway` provisions the AI Gateway control plane in Konnect and acts as the controller for everything else. From there, `AIGatewayModelProvider` and `AIGatewayModel` connect upstream LLM providers; OpenAI, Anthropic, Azure, Bedrock, Gemini, and a dozen others and exposes them as routes, with support for weighted round-robin, priority, latency usage-based, and semantic load balancing across targets:
apiVersion: aiconfiguration.konghq.com/v1alpha1
kind: AIGatewayModel
metadata:
name: gpt-4o-mini
spec:
aiGatewayRef:
type: namespacedRef
namespacedRef:
name: my-ai-gateway-cp
apiSpec:
type: model
model:
name: gpt-4o-mini
formats:
- type: openai
capabilities:
- generate
config:
route:
paths:
- /v1
targets:
- name: gpt-4o-mini
provider:
name: openai-provider
config:
type: openai
openai:
upstreamURL: https://api.openai.com/v1/chat/completionsBecause Kong Operator natively understands these AI Gateway CRDs, you can easily configure provider-specific rules, such as setting up Azure OpenAI latency-based load balancing or configuring dynamic Bedrock model routing directly within your Kubernetes manifests.
`AIGatewayPolicy` applies guardrails like prompt injection detection and topic allowlisting before a request ever reaches a model. AIGatewayAuthStrategy and the `AIGatewayConsumer` family add per-team API key authentication, consumer grouping, and usage attribution, all enforced at the Gateway.
Get started with the [_AI Gateway on Kubernetes get-started guide_](https://developer.konghq.com/operator/get-started/ai-gateway/install/)_AI Gateway on Kubernetes get-started guide_.
Kong Operator has supported `HTTPRoute` and `TLSRoute` for a while. 2.3 fills out the rest of the Gateway API route kinds, `TCPRoute`, `UDPRoute`, and `GRPCRoute `are now supported both for on-prem gateways and for Konnect based Gateway deployments.
Learn more in the [_Gateway API reference_](https://developer.konghq.com/operator/dataplanes/gateway-api/)_Gateway API reference_.
Platform teams running multi-tenant clusters need a way to answer "whose traffic is this?" without leaving Kubernetes. Kong Operator 2.3 extends Kong Operator's existing konghq.com/tags annotation to Gateway API resources under Hybrid Gateway: tag an HTTPRoute, GRPCRoute, TLSRoute, TCPRoute, or UDPRoute or the backend Service etc. and that tag now propagates through to Konnect.
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: checkout-service
annotations:
konghq.com/tags: "team-checkout,cost-center-4210"
spec:
# ...Running AI Gateway and Event Gateway data planes in production means being able to scale them the same way you'd scale anything else in Kubernetes. 2.3 adds native `HorizontalPodAutoscaler` support for AI and Event Gateway Dataplanes. Just set `spec.deployment.scaling.horizontal`, and the operator creates and manages the HPA for you, removing it cleanly if you switch back to a static replica count, which prevents the common issue where the Kong Operator HPA gets deleted or orphaned when replicas are fixed manually.
With 2.3, Kong Operator now manages API Gateway, AI Gateway, Event Gateway, and Dev Portal resources through the same declarative, Kubernetes-native model, version-controlled, reviewed, and deployed through the same GitOps pipelines you already run everything else through.
Whether you're routing HTTP, TCP, UDP, or gRPC traffic, proxying to an LLM, or exposing MCP tools, it's all the same operational pattern within Kubernetes.
**Q: How do I deploy Kong AI Gateway on Kubernetes?**
A: Deploying Kong AI Gateway on Kubernetes is done via the Kong Operator. You apply Custom Resource Definitions (CRDs) like `AIGatewayModel` and `AIGatewayPolicy `to define your control plane, providers, and models. For full instructions, refer to our[ _AI Gateway on Kubernetes get-started guide_](https://developer.konghq.com/operator/get-started/ai-gateway/install/) _AI Gateway on Kubernetes get-started guide_.
**Q: What are the main differences between Kong Operator 2.2 vs 2.3?**
A: While Kong Operator 2.2 introduced support for Event Gateway and Dev Portal, version 2.3 expands heavily into AI and advanced routing. The biggest differences in 2.3 are native Kubernetes AI Gateway support (enabling GitOps for LLM routing) and full Gateway API protocol coverage, including TCPRoute, UDPRoute, and GRPCRoute.
**Q: Why is my AIGatewayPolicy not enforcing the topic allowlist?**
A: If your topic allowlist isn't being enforced, verify that your `AIGatewayPolicy `is correctly bound to the targeted route or service. Also, ensure the LLM provider configuration in your `AIGatewayModelProvider` supports the necessary inspection capabilities. Misconfigurations here are the most common reason a request bypasses guardrails.
**Q: What is semantic load balancing in an LLM gateway?**
A: Semantic load balancing is an advanced routing strategy where Kong AI Gateway analyzes the actual meaning or context of an incoming prompt—not just traditional HTTP metrics—to route the request to the most appropriate, cost-effective, or performant LLM target.
**Q: How do I fix a "Gateway API GRPCRoute unsupported" error in Kong Operator?**
A: This error occurs when using an older version of the Kong Operator that only supported HTTPRoute and TLSRoute. To fix it, upgrade to Kong Operator 2.3, which introduces full native support for GRPCRoute, as well as TCPRoute and UDPRoute.
AI is not just producing different answers. It is starting to do different kinds of work. JEV introduces a different model for using AI: instead of generating text, it returns typed, probabilistic decisions that software can act on directly. By ret
When an AI application fails, knowing that an individual model call returned a 200 isn’t enough. You need to know what led to it. Kong stitches the interactions associated with an AI session together, giving developers a view across the sequence of
Consider a bank using an AI agent to investigate potentially fraudulent transactions. A transaction is processed and the bank’s fraud detection system identifies suspicious activity. That event is published to Kafka: fraud.suspected The bank’s Fraud
Most AI cost management starts with infrastructure. A provider can tell you that you consumed a certain number of input and output tokens on a particular model. An observability platform can show requests, latency, tokens, and traces. A cloud cost p
Traditional applications generally have well-defined boundaries. They interact with a known set of services using credentials provisioned specifically for those connections. Agents are different. Their value comes from their ability to work across s
