Cargill Uses Kong to Create Unified API Platform Across Environments

Creating an API-first architecture to enable innovation

As an industry leader, Cargill has always recognized that enabling rapid innovation is critical to maintaining a competitive advantage. However, Cargill’s legacy IT systems were slowing its ability to create new digital products and services to address the evolving needs of its customers and partners.

In response to these new digital imperatives, Cargill began the process of transforming its internal suite of APIs to be more dynamic, thereby enabling consumption across the entire company.

“We put Agile, DevOps, and CI/CD at the heart of everything we do, and we realized that we needed new tooling to match the new ways we wanted to build products and services,” said Jason Walker, senior enterprise architect at Cargill. “We have a wealth of data from 150 years in business, but we needed to create a simple way for folks to access it and use it to enhance our stakeholders’ experiences. By building an API-first microservices architecture, we knew we could simultaneously provide access to those data and resources and take deployment concerns out of our developers' workflows.”

Pragmatic Digital Transformation with Kong

With the goal of bringing more digital experiences to stakeholders, Walker and his team came up with a three-phase plan to increase agility and transition from siloed legacy IT architecture to a modern, microservices architecture.

To accomplish this, Walker’s team wanted to partner with a high-performance API platform that could function equally well in centralized and decentralized deployments.

“We wanted to first centralize the platform to make it consistent, then decentralize it to maximize scale, but the legacy API management vendors we looked at weren’t aligned with our vision for decentralization,” Walker said. “Kong is lightweight and fast enough to run at the edge, with Kubernetes or in a mesh, which fits our vision perfectly. Together with Kong, we achieve the low latency needed to deploy high-performance, cloud-native microservices and the flexibility to seamlessly integrate them across all our systems.”

The team at Cargill was also looking for not just a vendor, but a partner to help them build this new architectural vision.

In addition to superior performance and flexibility, Kong allowed Cargill to simplify management and monitoring across its highly complex architecture. Kong’s out-of-the-box management tools and open-source ecosystem provide Cargill with robust capabilities to simplify the management of APIs across their entire lifecycle.

“With the level of scale and complexity we deal with at Cargill, it’s critical that all of our technology partners provide the maximum leverage possible,” said Colin Schaub, lead API engineer at Cargill. “The tools provided by Kong make managing the platform easy, and the community ecosystem is great about adding new plugins. It’s allowed us to bring more services to market more quickly and helps us manage them better once they’re in production.”

Building a Unified Developer Experience with Kong

Creating a scalable, standardized developer experience was a foundational element of Cargill’s vision for an API-centric hybrid architecture. With thousands of internal and external developers consuming their services worldwide, ensuring consistent standards was paramount to ensuring governance. It was critical for Cargill that the API platform they selected provide a consistent experience to minimize friction and enable sharing of services.

“We wanted to make sure that we were building for scale and enabling the services we built to be re-used across the company, not doing things on a one-off basis,” Walker said. “Reusable capabilities need to have one single source of truth, and Kong ensures that all of our constituents are consistent in how they use our services. With Kong, we can just give them a platform-in-a-box that ensures standardization, security, and compliance.”

Delivering On-Demand Scalability with Kong and Kubernetes

The second phase of Walker’s team’s vision was to decentralize the platform to facilitate dynamic scalability. A major component of this was utilizing cloud-native design patterns, and Kong’s native support for Kubernetes was a critical factor in enabling the Cargill team to easily scale up and down with demand.

“With the number of new services we’re adding and creating from decoupling legacy applications into microservices, manually managing our infrastructure would be nearly impossible. Using Kong with Kubernetes allowed us to make decentralization a reality,” Walker said. “Kong and Kubernetes let us define how we want our services to behave and then automatically scale the services up or down depending on the situation. The combination of the two allows us to optimize for resource efficiency, resiliency, and service availability all in one automated solution.”

Cargill uses AWS EKS/Kubernetes for all cloud deployments. It deploys Kong through its

pipeline using the Kubernetes ClusterIP type, but allows ingress from the outside to Kong, and

then Kong brokers incoming requests to the upstream APIs. Upstreams are defined within Kong

as Kubernetes Services so Kong only needs to be aware of the single Service name of each

Upstream.

Everything in the stack is managed by AWS: EKS, RDS, Route 53. API teams are allowed to

use whichever AWS capability they wish including DynamoDB, MySQL, SQS, and the bulk of

Kong activity is managed via Kubernetes.

Building the Future of Cargill’s Architecture

A key component of Cargill’s decision to select Kong stemmed from confidence in the vision Kong is building towards.

“It was critical for us that the API platform we chose would allow us to move fast now, but also take us where we want to go,” Walker said. “Kong’s view of where the world of software architecture is headed aligns well with ours at Cargill.”