In this episode of Kongcast, I had the pleasure of speaking with Aaron Weikle, the founder and CEO at MS3, about supporting legacy-based applications as companies add the next generation of microservices.
Check out the transcript and video from our conversation below, and be sure to subscribe to get email alerts for the latest new episodes.
Kaitlyn: Could you start by setting the stage for our viewers on the problems organizations run into and how this relates to common connectivity challenges?
Aaron: Our organization has been in the integration space for over a decade, and so we’ve seen the evolution of technology as it goes from a legacy-based type of B2B interconnectivity with pipe-delimited data, fixed fieldsets through SOAP integrations, and now we’re into our next generation of RESTful APIs and gRPC.
There’s a lot of technical debt that organizations have strung along for decades across multiple industries. All of this needs to be supported as they're going through digital transformation. This problem is organizations want the latest and greatest, but they still need to support legacy-based protocols and interconnectivity. So we’ve come up with some solutions to help address those needs.
Kaitlyn: So one of the things we’ve talked a little bit about is how organizations in the early 2000s adopted these SOAP-based protocols, and they’re now having to figure out what to do with those services. How are you seeing organizations handling this?
Aaron: Organizations have done a variety of different things. They’ve implemented technology tooling like ESBs and provide both message and protocol-based abstractions. They’ve implemented various technologies to implement a way to hide the SOAP behind the scenes.
A gentleman named Martin Fowler wrote a document around the Strangler Pattern or Strangler Facade. We like to take that pattern and implement it in the Mullet Pattern, where you have a nice clean API in the front, and you got the messy SOAP in the back.
What that does is provide a layer of abstraction between the consumers and the legacy systems. Because you have that abstraction point, it allows a period of disconnection between the services. As you begin to modernize the backend systems, those are not impacting your consumers.
This is important because no one wants the consumers to have to make modifications every time a change goes into place. That is a very costly approach. Taking this Mullet Pattern helps to alleviate the type of cost initiatives that organizations face on a daily basis.
Kaitlyn: So one of the ways that you’ve addressed this, I believe, is through leveraging some custom plugins in Kong Gateway. Can you walk us through that approach?
Aaron: You know, Kong Gateway is pushing the boundary of the next-generation capabilities, supporting RESTful-based protocols, gRPC, protobufing and GraphQL. And you intertwine that with service mesh for really a connected platform and not so much just an API gateway.
Sadly, organizations, when they look at Kong, they think it’s just an API gateway and don’t understand that it’s that next step to interconnectivity. And it’s much more than just that.
We’re providing a way that organizations can leverage Kong as a product for their next-generation modernizations and give that same platform the ability to support legacy-based communications.
And so we’ve built a series of plugins that support everything from exposing SOAP services and having that abstracted through the Kong Gateway to being able to consume and have your automation tooling and be able to ingest a WSDL and generate those stubs from a consumer perspective.
We’ve provided capabilities for data validation and SOAP action validation. And that’s key because the best way to address system performance is to identify problem calls at the source or as early as possible. Suppose we can do data validation and SOAP action validation at the API gateway level. In that case, that’s alleviating processing time that a very busy backend system that may not have the same capabilities of modern technology won’t have to worry about.
In addition, one of the big problems that we see is the poor implementation of fault handling within SOAP. SOAP doesn’t give you a nice, clean status code associated with a problem. Like when you have 401, 403 or 404 errors in RESTful APIs, that is pretty self-telling just in the status code. But everything that comes back from SOAP is just a 500. Inside of that, they have what’s called a SOAP fault body.
Now, historically with SOAP faults, they dump a lot of information. There could be stack traces or sensitive information—many really ugly things that you don’t want to go out to your end consumers. If you’re in the health or financial industry, there could be sensitive information coming back across the wire that you don’t want, and you can't control that.
One of the components that we’ve built is a SOAP fault extractor. When the SOAP fault comes back, we clean it up and say, “contact your administrator,” and take that heavy lift of being able to process that off of the systems and having to make sure that everything is coming back clean. Because at the end of the day, we don’t want our developers or our clients’ developers working on building enhancements to SOAP services, which should be slowly going away. Instead, they should be focusing on writing new, RESTful or gRPC-based services that can go through the API gateway. By providing that, whatever technical debt they historically have had, we can now clean it up with some of these plugins.
As I discussed earlier, we can expose the RESTful-based APIs and the gRPC-based protocols through the frontend API gateway. Or we’ll give our clients the ability to expose legacy SOAP services without having to rewrite them right now. They can expose that through Kong with a contract, whether with OpenAPI or through a protobuf doc, and your consumers can start consuming that. The plugin will map from standard HTTP protocols or the objects within the gRPC into the SOAP actions.
SOAP has SOAP actions that say, “What is it that we’re going to do with this inbound request?” And so it could be processing data, retrieving data and updating data, but it all comes through the same HTTP request, which is an HTTP POST. So what we do is we provide the capability to map a GET operation on the frontend to an HTTP POST with a say, “get customer” SOAP action on the backside. Likewise, if we’re doing a POST operation for the customer, we do a POST operation to “create customer” on the SOAP action.
So this gives us the ability to blend the RESTful frontend capability into the SOAP backend. And that’s going to alleviate the rewrite of consumers down the line. Especially as you’re going from, let’s say you have an old ERP system, maybe it’s an SAP system, or it’s a Microsoft Dynamics and your move to a Salesforce, right? Those legacy systems are using SOAP.
As you’re migrating your ERP components into Salesforce, now, your consumers are completely oblivious that any of this work is going on on the backside because they’re still getting the data models. They're still using the operations the way that they did before. So that’s really powerful.
Next, we created an additional plugin kit for Kong. We took a Java-based component, and we built it to utilize the communication bridge within Kong. We decided to go in this direction because a lot of the powerful data transformation tooling is written in Scala or Java or things used in a JVM.
One of the open source platforms that we love is called DataSonnet. DataSonnet is a template-based transformation language that was written at Scala, and Google created it. Shortly thereafter, Databricks adopted it, and then we added a bunch of capabilities and functionality to enable complex data transformation.
This is important, especially if you want to do things like orchestration in the API gateway. Maybe, from a mobile application perspective, you don’t want them to send in these huge requests and these huge responses back and iterate through the data to make the next set of requests. That becomes erroneous to the mobile applications where their CPUs and memory just can’t handle that amount of work.
So this capability allows Kong to have a lightweight call come in from a mobile application, and it can use the data on it to call this a product catalog. Then that product catalog will return information, and we can use another callout to a system to provide additional information. This starts to shrink the data model to only what the mobile application wants and then returns it. That pulls the workload off the mobile application and into backend systems with substantially more resources. And so that’s another really powerful way that you can master Kong with these plugins to support a legacy on the backend, modern and mobile things on the frontend.
Kaitlyn: Yeah, that was a fantastic overview.
I know you have a demo prepared, so I’m super excited to see how this is done. Check out Aaron's demo in the below video:
I hope you'll join us again on November 29 for our next Kongcast episode with Chinmay Gaikwad from Epsagon called "Metrics and Logs Are Out, Distributed Tracing Is In."
Until then, be sure to subscribe to Kongcast to get episodes sent to your inbox (and a chance to win cool SWAG)!
Ever feel like you're fighting an uphill battle with your API strategy? You're building APIs faster than ever, but somehow everything feels harder. Wasn’t API-first supposed to make all this easier? Well, you're not alone. And now industry analys
In this article, which is based on my talk at VueConf Toronto 2023, we'll explore how to harness the power of Vue.js and micro frontends to create scalable, modular architectures that prioritize the developer experience. We'll unveil practical strate
Survey Says: Google LLMs See Usage Surge, Most OK with DeepSeek in the Workplace Enterprise adoption of large language models (LLMs) is surging. According to Gartner , more than 80% of enterprises will have deployed generative AI (GenAI) applicatio
Kafka delivers massive value for real-time businesses — but that value comes at a cost. As usage grows, so does complexity: more clusters, more topics, more partitions, more ACLs, more custom tooling. But it doesn’t have to be that way. If your tea
A Practical Look at When (and When Not) to Use Ambient Mesh The word on the street is that ambient mesh is the obvious evolution of service mesh technology — leaner, simpler, and less resource-intensive. But while ambient mesh is an exciting develop
I'm commonly asked by customers for advice on how they can build a good platform cross-charging model for their organization. And my gut reaction is nearly always "don't." We'll come back to why I think that later, but first let's look at what cross
Unify the API and Eventing Developer Experience with the Kong Event Gateway and API Platform Introduction: The EDA and API worlds are converging . . . finally For the past several years, there have been murmurs of an incoming convergence between API