If you're running Kafka at any real scale, you've probably hit some version of these three problems. Compliance wants specific fields — SSNs, emails, account numbers — locked down, but your consumers still need the rest of the record. Your topic names are load-bearing in ways nobody planned for, so…
Running Apache Kafka on Amazon Managed Streaming for Apache Kafka (MSK) gives you a managed broker with no ZooKeeper to operate, automated patching, and multi-AZ replication handled by the service. What it doesn't give you is a safe, governed way to expose Kafka access beyond your VPC boundary.…
This is part of a three-part series. For the full story, see Your AI Agent Knows What, It Doesn't Know Why and The Case for Kafka as the Agent's Memory Layer . We made the case for why agentic AI systems need a durable commit log rather than static state snapshots. The argument is…
This is part of a three-part series. For the full story, see Your AI Agent Knows What, It Doesn't Know Why and Building the Agentic Commit Log . Once you accept that agentic AI systems need a durable commit log — an ordered, immutable record of every tool call, decision, and context shift —…
Most engineering teams adopt Apache Kafka for one simple reason: it works. It scales effortlessly, it is incredibly reliable, and it powers real-time systems across almost every industry. But as your Kafka usage expands across different teams, regions, and external consumers, success creates a…
Modern Kafka deployments struggle with a familiar tension. You want fine-grained access control per client, per team, and even per request. However, traditional ACLs force you into static, cluster-level configurations that are brittle, hard to scale, and painful to maintain. Administrators are…
Event-driven architectures (EDAs) have pretty quickly transformed from a niche engineering pattern to the all-pervasive central nervous system of the modern enterprise. With Apache Kafka at the helm, enterprises are rapidly shifting gears from synchronous REST APIs to asynchronous event streams to…
There's a problem that almost every platform team running Kafka at scale eventually hits, and it usually starts with a reasonable ask: "Can you give our partners access to this event stream?" What follows is rarely simple. You start scoping VPC peering. Then someone asks about firewall rules. Then…
The messaging and data streaming industries are constantly evolving, with major shifts such as large vendor purchases. If your company relies heavily on a platform such as Apache Kafka and you recently discovered that your vendor was acquired by a competitor, you are most likely dealing with a…
Apache Kafka powers real-time data at companies processing substantial event volumes daily. The new KRaft architecture makes getting started easier than ever. The platform handles massive data flows for more than 80% of Fortune 100 companies ( Apache Kafka ), according to the Apache Software…
Kafka delivers massive value for real-time businesses — but that value comes at a cost. As usage grows, so does complexity: more clusters, more topics, more partitions, more ACLs, more custom tooling. But it doesn’t have to be that way. If your team is managing Kafka at scale, here are five…
Managing Kafka as an island on its own got you this far. But scaling it securely and efficiently across your organization? That's another matter entirely. Apache Kafka is the number-one event streaming platform used by developers and data engineers worldwide to build reliable, scalable real-time…
Kong customers include some of the most forward-thinking, tech-savvy organizations in the world. And while we’re proud to help them innovate through traditional APIs, the reality is that their ambitions don’t stop there. Increasingly, our customers are investing heavily in real-time data and event…
For the past several years, there have been murmurs of an incoming convergence between API management and event management. This has been more than a vendor-led phenomenon; industry analysts — such as Gartner and Forrester — see the worlds colliding as well, which is why Gartner mentions event…
Today, we're pleased to announce Kong Gateway 3.10, our next Long-Term Support (LTS) version. This LTS version will be supported until March 2028, up to three years from the release date. Please check our documentation here for more details on Kong’s support policy. Highlights of this release…
We announced the Kong Premium Technology Partner Program at API Summit 2024, and Confluent was one of the first in the program. This initial development was all about ensuring that the relationship between Kong and Confluent — from a business and product perspective — fully represented our joint…