As a Solutions Engineer here at Kong, one question that frequently comes across my desk is “how can I transform a Kong logging plugin message into a format that my insert-observability-stack-here understands, (e.g., ELK, Loki, Splunk, etc.)?”
In this blog, I’m going to show you how to easily accomplish converting a Kong logging payload to the Elastic Common Schema.
In order to accomplish this task, we’re going to be running Kong Gateway in Kubernetes and using two Kong plugins.
If you don’t already have an instance of Kong running in a Kubernetes cluster, connect to your cluster and run the following the commands to get one in seconds.
See how to install Kong in Kubernetes for more information. Once you have an available instance of the Kong Gateway, continue.
First, create an empty Kubernetes manifest file called, elastic-common-schema.yaml.
Next, let’s define our KongPlugin resources. The first plugin we will create is the serverless pre-function. From the Kong plugin docs, a serverless pre-function plugin:
Runs before other plugins run during each phase. The pre-function plugin can be applied to individual services, routes, or globally.
Since we’re logging, we’re concerned with the log phase or “context”. For more information on all available plugin contexts, read this doc.
Paste the below yaml in your manifest.
The above resource definition creates a KongPlugin that executes before the logging phase of each plugin defined in scope. The kong.ctx.shared.mystuff=kong.log.serialize() is a single line of Lua code that stores the logging payload into a shared context. From the Kong docs, a shared context is:
A [Lua] table that has the same lifetime as the current request. This table is shared between all plugins. It can be used to share data between several plugins in a given request.
For more info on shared contexts, see this doc.
The key to doing the transformation is the custom_fields_by_lua configuration. From the Kong docs, the custom_fields_by_lua is:
A list of key-value pairs, where the key is the name of a log field and the value is a chunk of Lua code, whose return value sets or replaces the log field value.
which gets transformed into:
Congratulations, you have transformed a Kong log payload into an Elastic Common Schema format ready for ingestion! This pattern can be used to easily transform Kong logging messages into any format for ingestion with any observability stack.
We're very excited to announce Kong Mesh 2.12 to the world! Kong Mesh 2.12 delivers two very important features: SPIFFE / SPIRE support, which provides enterprise-class workload identity and trust models for your mesh, as well as a consistent Kuma R
Meet Emily. She’s an API product manager at ACME, Inc., an ecommerce company that runs on dozens of APIs. One morning, her team lead asks a simple question: “Who’s our top API consumer, and which of your APIs are causing the most issues right now?”
In the last two parts of this series, we discussed How to Strengthen a ReAct AI Agent with Kong AI Gateway and How to Build a Single-LLM AI Agent with Kong AI Gateway and LangGraph . In this third and final part, we're going to evolve the AI Agen
In my previous post, we discussed how we can implement a basic AI Agent with Kong AI Gateway. In part two of this series, we're going to review LangGraph fundamentals, rewrite the AI Agent and explore how Kong AI Gateway can be used to protect an LLM
This is part one of a series exploring how Kong AI Gateway can be used in an AI Agent development with LangGraph. The series comprises three parts: Basic ReAct AI Agent with Kong AI Gateway Single LLM ReAct AI Agent with Kong AI Gateway and LangGr
What Is RAG, and Why Should You Use It? RAG (Retrieval-Augmented Generation) is not a new concept in AI, and unsurprisingly, when talking to companies, everyone seems to have their own interpretation of how to implement it. So, let’s start with a r
In February 2024, Kong became the first API platform to launch a dedicated AI gateway, designed to bring production-grade performance, observability, and policy enforcement to GenAI workloads. At its core, Kong’s AI Gateway provides a universal API