Bringing Gateway API for Mesh to Kuma
The release of Kuma 2.3 brings experimental support for GAMMA (Gateway API for Mesh Management and Administration) resources. Kuma has long supported Gateway API with the built-in gateway for ingress traffic but with GAMMA support, users can specify how to route and modify in-mesh traffic using the well-known HTTPRoute resource from Gateway API.
Gateway API is a project focused on improving the APIs around networking between services in Kubernetes clusters. The new API consists of resources like Gateway and HTTPRoute that model infrastructure and behavior around ingress traffic, routing, and traffic modification.
GAMMA in particular is a subproject focused on mesh implementations of Gateway API and extending the Gateway API resources to mesh use cases.
Gateway API and GAMMA's goals are to create an API and specification implementable by any ingress infrastructure or mesh. Portability of APIs and expected behavior allows users to learn just one API, reduces barriers when moving between implementations and means less work for tools that integrate with implementations. These APIs are intended to be as expressive as possible for core functionality but provide extension points for implementation-specific behavior and specification.
Read on for more information about using GAMMA with Kuma.
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
...
spec:
parentRefs:
- ...
rules:
- matches: ...
filters: ...
backendRefs: ...
It's important to remember that the GAMMA specifications for using HTTPRoute for mesh traffic are still in the implementable phase, so the semantics are still very subject to evolution and change. However, the resource itself is quite powerful and stable, already in beta and soon to graduate to GA.
The central concept behind HTTPRoute for mesh is that instead of specifying and attaching to a parentRef of kind Gateway, we attach to a Service parent. Note again that GAMMA semantics are still experimental and in particular having a Service as parent is still under review in Kubernetes. Using Service enables standardized usage of HTTPRoute across meshes but the Service resource is already overloaded so it’s still possible that an alternative arises before GAMMA graduates from experimental.
The effect of attaching to a Service parent is that any requests to what's called the Service "frontend", i.e. the DNS name defined by the parent Service, are first filtered and then routed as specified by the HTTPRoute. Instead of routing to the Service endpoints, or "backends", as controlled by kube-proxy, requests can be routed by the mesh to the backendRefs as defined in the HTTPRoute resource.
Let's look at an example of how to use Kuma with HTTPRoutes.
Assume we have a backend service that's being consumed by a number of
services in our Kubernetes cluster. Now one of these consumers wants to enable an
experimental feature of the backend service that's guarded by an HTTP header.
This kind of traffic modification is something HTTPRoute was designed to do:
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
name: enable-catalog-feature
namespace: ui-backend
spec:
parentRefs:
- name: catalog
namespace: products
port: 80
kind: Service
rules:
- filters:
- type: RequestHeaderModifier
requestHeaderModifier:
add:
- name: x-dev-feature
value: "enabled"
- backendRefs:
- name: catalog-v1
port: 80
weight: 90
Most importantly, we note that spec.parentRefs[0].kind is Service, which means Kuma interprets the HTTPRoute as a GAMMA resource.
The second piece to understanding how exactly this resource affects mesh traffic is to check
the Namespace of HTTPRoute itself as well as the Namespace of the parentRef.
GAMMA supports two kinds of routes.
- Consumer routes apply to a single Namespace of consumers of the parent Service and they’re created by setting the route resource’s Namespace to be different from the Namespace of the parent. The assumption is that the owners of consumers of the parent in this route’s Namespace are the same users who can modify routes in this Namespace.
- Producer routes apply to all requests to a parent Service. These routes live in the same Namespace as their parent. Here the assumption is that the owners of the parent Service are the same users who can modify routes in the parent’s Namespace.
The HTTPRoute we just created doesn't live in the same Namespace as the api Service so it’s a consumer route and its effects are isolated to requests sent from workloads in the Namespace of the HTTPRoute, ui-backend.
For this demo, the catalog service simply echoes our requests back to us as JSON:
$ curl http://catalog.products
{
"path": "/",
"headers": {
"host": "catalog.products",
"x-request-id": "ab05140b-9a13-445a-94f1-76980d6e4c92",
"x-dev-feature": "enabled"
},
...
}
If we instead look at requests from the products Namespace:
$ curl http://catalog.products
{
"path": "/",
"headers": {
"host": "catalog.products",
"x-request-id": "72adb8e3-e75f-4ef7-838a-910d097ef607"
},
...
}
We see the header is absent.
What if we're the backend team though, and we want to configure routing once, by default for all requests in the mesh? Then we can create a producer route.
Let’s look at a traffic splitting use case. We’ll have two additional Services that select different versions of our catalog app that have been tagged through their Deployment:
apiVersion: v1
kind: Service
metadata:
name: catalog-v1
namespace: products
spec:
selector:
app: catalog
version: v1
---
apiVersion: v1
kind: Service
metadata:
name: catalog-v2
namespace: products
spec:
selector:
app: catalog
version: v2
We can now create an HTTPRoute that directs traffic to these two different services but only send 10% to v2.
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
name: enable-catalog-feature-for-all
namespace: products
spec:
parentRefs:
- name: catalog
port: 80
kind: Service
rules:
- backendRefs:
- name: catalog-v1
port: 80
weight: 90
- name: catalog-v2
port: 80
weight: 10
Starting with ui-backend:
$ for ((n=0;n<10;n++)); curl http://catalog.products | jq -r .os.hostname; done
catalog-v1-8568c9bddc-pztrm
catalog-v1-8568c9bddc-pztrm
catalog-v1-8568c9bddc-pztrm
catalog-v2-688ff89fdd-8mz9l
catalog-v1-8568c9bddc-pztrm
catalog-v1-8568c9bddc-pztrm
catalog-v1-8568c9bddc-pztrm
catalog-v1-8568c9bddc-pztrm
catalog-v1-8568c9bddc-pztrm
catalog-v2-688ff89fdd-8mz9l
We see catalog-v1 weighted roughly 9:1, which we can repeat from any Namespace.
Note that under the hood, Kuma is creating MeshHTTPRoutes, kuma.io custom resources, that look very similar to HTTPRoutes but use Kuma’s concept of kuma.io/service and on their own can be used in both Kubernetes and Universal and across Kuma zones.
One of the greatest boons for implementers of Gateway API has been the presence of conformance tests. These can be run by any Gateway API implementation to ensure that they behave as specified. They’ve been especially useful to improve the language of the spec and resolve ambiguities.
Kuma already passes the conformance tests for Gateway+HTTPRoute but there’s a small but growing set of tests for HTTPRoute as a GAMMA resource that Kuma passes as well.
Kuma is committed to being a fully conformant Gateway API and GAMMA implementation and participating in the upstream project under the Network Kubernetes SIG project. At the moment, GAMMA is focused on standardizing routing between implementations but its focus isn’t limited to routing and it intends to explore all potential use cases of Gateway API for meshes.
If Gateway API or GAMMA is something that sounds interesting, we’d love for you to take a closer look at the roadmap and get involved with the upstream project!