Proxy-Wasm: It’s WebAssembly for Proxies
Picture this: You're building the next generation of microservices architecture in your organization. The orchestration is in place, containers are humming, and you've chosen Kong Gateway (naturally) to manage the APIs, ensuring smooth communication.
But then, you hit a snag.
You need a custom filter not part of the standard library of plugins, or you envision a unique way to manipulate, observe, or control the traffic. Traditional non-Lua PDKs can be cumbersome (because of their out-of-the-process nature), require intimate knowledge of the Kong request lifecycle, etc.
Enter WebAssembly and Proxy-Wasm. What if you could develop custom filters with the power, flexibility, and portability that WebAssembly offers and seamlessly integrate filters developed for Envoy in Kong Gateway?
In this post, we'll dive into WebAssembly, and how it's revolutionizing the way we think about and implement custom filters, offering a horizon of possibilities that once seemed like a distant dream.
What is Proxy-Wasm?
Proxy-Wasm is an open-source specification that allows engineers to create filters, similar to custom plugins in Kong, using a language they're comfortable with. These filters can then run on any proxy that supports the Proxy-Wasm standard, such as Kong Gateway, Envoy, and potentially Kuma in the future.
Imagine being able to write a filter in Go, which (we hope) you're familiar with, and have it work on both Kong Gateway and Kuma simultaneously. That’s pretty neat!
Let's start from the beginning and explore what Proxy-Wasm is, why it's useful, and how it works.
From the beginning
WebAssembly is relatively new, it was announced in 2015 but not released until 2017.
The name, WebAssembly, is intended to suggest that assembly-like language can be executed by web browsers on the user’s computer (i.e. client-side). The original intent was to provide near-native code execution speed for web browsers, but as we’ll discuss, the use cases have expanded beyond this. It was designed for high-performance computationally intensive tasks, such as games, multimedia applications, and simulations.
The developer experience in the context of web development is first to write some functionality in a high-level programming language that you are familiar with (Rust and Go are popular), and then using a WebAssembly-compatible compiler, compile the source code to a wasm module (these are those .wasm files). The output is a wasm module, low-level bytecode, that can now be run in a Wasm runtime.
What is the Wasm runtime?
Well, wasm code runs in a sandboxed environment to ensure security. It has limited access to system resources and can't perform arbitrary operations without going through the defined APIs. There are several runtimes available today: Wasmtime, Wasmer and V8 as some examples.
How does this work for web browsers?
Assuming you want to use this from the context of web browsers, the wasm modules can be loaded and executed within the web browser using a combination of Javascript and browser-specific runtime engines that provide Wasm support. Today, all major browsers support Wasm.
This is all great. What does this have to do with you, an engineer, using gateways every day to produce new business functionality?
Transition to Proxy-Wasm
There is a fundamental limitation in today’s proxies, in that many are extremely powerful and highly performant but challenging to extend.
Proxy-Wasm is an emerging specification that extends the capabilities of WebAssembly (Wasm) to the realm of networking. It allows developers to write and deploy WebAssembly modules (referred to as "filters") that can be inserted into the request path of proxies, such as HTTP servers, API gateways, or service meshes, to manipulate network traffic. Proxy-Wasm is particularly valuable for scenarios where you must implement custom logic, security, or observability features in your network infrastructure.
How does proxy-wasm work?
Proxy-Wasm is made of two complementary counterparts: a host ABI and an SDK library.
The proxy ABI exposes low-level bits of the proxy to Proxy-Wasm SDK library used to write the filters. All filters are built atop the SDK library's abstractions, themselves powered by the ABI and the underlying runtime. There are currently Proxy-Wasm SDKs available in AssemblyScript, C++, TinyGo, Rust.
For the full discussion, consider reading Kong’s What is Proxy-Wasm Documentation.
What is the developer's experience?
Continuing with the original example, the experience would be:
- A developer writes a “filter” (analogous to a Kong plugin) in a high-level programming language that you are familiar with and has the Proxy-Wasm SDK available
- Then, using the available WebAssembly-compatible compiler, compile the source code to a wasm module (again, the .wasm file)
- Load the wasm module into the respective proxy
- Configure the proxy to execute the filter for incoming requests
But, in order for all this to happen, the proxy must implement and expose the Proxy-Wasm ABI. So, how did Kong make this happen?
Hello, WasmX
Kong under the hood runs Nginx, and relies on OpenResty to provide the Kong capabilities we all love. OpenResty is highly performant, written in Lua, and uses LuaJIT, a Just-In-Time (JIT) compiler, and provides a rich set of libraries to develop all the key Kong features (highlighted in blue in the diagram below).
In order to also provide support for Proxy-Wasm, the Kong engineering team built an ngx_wasm module that implements the Proxy-Wasm specification and sits, metaphorically speaking, alongside the LuaJIT VM.
With this, Kong platform engineers can now write “filters” in the Proxy-Wasm ecosystem to manipulate and/or interact with the API requests that are proxied through the gateway.
Just like the custom Lua plugins, a Proxy-Wasm filter provides an HTTP context with entry points into the request/response lifecycle of nginx phases. In the diagram below, we’ve summarized the HTTP context for you to give you an idea of the correlation between a custom Lua plugin and a filter.
Can both Lua plugins and Proxy-Wasm filters run side-by-side?
The answer is yes. But, what happens is in each request phase described above, the Lua plugins are executed before the filter.
This is very important to understand because this will most likely be a very common occurrence in production. For example, you will want to use Kong’s OIDC plugin and a custom wasm filter simultaneously.
Why are we doing this again?
You might ask yourself, "Why should I waste my time right now to understand WebAssembly?" Here's why:
1. The Kong Gateway is more extensible, and with WebAssembly, the filters run in a sandboxed VM.
2. Write once, run everywhere. Theoretically, a Proxy-Wasm filter that works on Kong Gateway will work on Envoy, and vice versa with no code changes. In fact, this was proven during the Kong Builders episode A Practical Application of WebAssembly + Kong Gateway.
That is pretty cool. Can you see the possibilities start to unfold?
Questions the Community has asked so far
- What are the supported Wasm Runtimes Available in Kong Gateway? The default in Kong Gateway is Wasmtime. However, our ngx_wasm module also supports Wasmer and V8.
- Can I write a filter in Python? Today, it is not possible because there is no Python-based Proxy-Wasm SDK available. Look to the Proxy-Wasm Spec for more information on available SDKs.
- Does this affect how the custom plugins are compiled and deployed? No, in fact, both Lua Plugins and filters can run together, as discussed above.
- What is the expected overhead of using wasm instead of writing Lua plugins? In terms of performance, Lua-based plugins are the most efficient due to Kong Gateway's native Lua architecture. Next are the Proxy-Wasm filters operating in the nginx module, and the least efficient are Kong PDK-based plugins (written in Go, Javascript, Python), which involve external inter-process communication with the nginx engine.
Wrapping Up
In this article, we've touched upon the intricacies of building filters using WebAssembly and Proxy-Wasm, and witnessed firsthand how this combination elevates the potential of Envoy, Kong Gateway, and Kuma to unprecedented heights. But this is just the beginning. As the ecosystem continues to evolve, so too will the challenges we face, demanding even more versatile solutions.
As we wrap up, remember: in the ever-changing world of microservices, innovation doesn't wait. But with WebAssembly and Kong Gateway in your toolkit, we're not just keeping pace but leading the charge.
Onwards!
P.S. Stay tuned for the November episode of Kong Builders where we continue to explore Proxy-Wasm with Go and Kong Gateway.
In the meantime, let’s point you to some Kong quickstarts.
- Proxy-wasm-go-ratelimiting: This is an example of a go-based rate-limiting filter
- Proxy-wasm-go-filter-template: this is a template to start your own filter with the go SDK
And, for more examples of filters that you can test on Kong, look to tetratelab’s list of examples on GitHub: tetratelab/proxy-wasm-go-sdk.