Brad x Ahmed: Elements for Building Cloud Native Applications
From time to time, Kong’s Brad Drysdale and Ahmed Koshok exchange emails about the latest trends in tech. In this exchange, Brad and Ahmed discuss the continued quest for more innovation faster, the role of automation in API strategy, and more.
from: Bradto: Ahmeddate: Oct. 29, 2022, 6:58 PMsubject: Elements for Building Cloud Native Applications
Greetings from Australia, Ahmed!
I wanted to pick your brain about a trend I’m seeing more of here in Australia — and across the broader Asia-Pacific region.
It seems every organisation's cloud journey is accelerating unabated. Whilst I suspect the current macroeconomic pressures are causing companies to want to reduce their costs as much as they can (and what better way to do that then embracing cloud?), it also seems to be equally motivated by this desire to innovate faster than they ever have before.
It's almost like what used to be bucketed as "Digital Transformation" is now more of a race to produce and ship innovative digital products and experiences for customers as quickly as possible.
For some perhaps the pandemic kickstarted their cloud motion, purely because they lacked other options to stay solvent during extraordinary times. More than that, it seems the new competitive battleground for customers is being fought out in the digital space, and therefore, in the cloud. I was wondering if it's similar where you are?
So it's made me wonder, what are the key ingredients to being successful in this new reality?
The future of software is distributed — that much we know. This trend started the very first day one of these organisations adopted a SaaS offering of something.
From there it's been a race to getting as much workload into the cloud in some shape or fashion. Now we're clearly at the point where mainstream adoption of cloud platforms is driving most organisations to completely rebuild (modernise) existing applications, or construct new, innovative applications in a cloud-first, cloud native fashion.
That's not without its challenges.
The first thing that comes to mind is the role which APIs and microservices play when it comes to breaking down traditional monolithic applications into smaller, finer-grained, easy to discover and even easier to assemble APIs and application components. Once that's understood, how then do you find these things and make use of them when they are scattered across the cloud and rapidly growing in number?
APIs are absolutely the lingua franca of this cloud world. (Yes, I just used “lingua franca” in a sentence. Deal with it.) So it's no surprise at all really, to see more and more of them in use.
Yet an API strategy is more than just "oh, APIs exist."
There's a whole lifecycle that they come burdened with, and a completely new mindset and culture inside of these organisations needed to make these APIs successful. From design, testing, deployment, discovery, adoption, management, security, observability, versioning, end-of-life events . . . the list goes on! Kinda tough to deal with when the APIs we’re talking about here are deployed on infrastructure you rent rather than own, don't you think?
So my question to you is: where to begin?
If an organisation is looking to take advantage of the cost benefits and innovation velocity that cloud platforms afford them (and it's a given that they want to modernise existing applications and build new ones in a cloud native fashion), where do they begin?
APIs clearly play a role here, as do microservices. So what would you suggest as the right place to start? Do you have any thoughts on what elements of strategy need to be considered to make this journey successful?
From: Ahmedto: Braddate: Oct 30, 2022, 9:41 AMsubject: Re: Elements for Building Cloud Native Applications
Hi Brad,
All’s well here in Virginia, USA! Autumn weather is in full effect and the beautiful outdoors are here for adventures.
As for technology fun, I am seeing similar patterns. Sure, going to the cloud is a big one, but certainly not the only one.
No two organizations are identical; yet the motivations for big shifts are usually a result of an improvement needed in at least one of three areas: quality, time, or cost. A golden triangle, per se.
Time and cost are easy to appreciate, and measuring them is direct. Quality can be subjective, and it covers concerns that are not as readily apparent.
I noticed that cost can go up, if quality goes up significantly more, or time is reduced to offset the increased cost via increased revenue, or improved experience, for example.
These motivations can be overlayed into other major trends we’re observing. CI/CD, containerization, AI, security . . . you name it. Each one of them contributes to this golden triangle in some way.
One thing is certain, the pace of change shows no signs of slowing. Unforeseen events cause shockwaves that present serious challenges. The agile, and adaptable survive and thrive. The static and slow stagnate and decay.
APIs and microservices are helping breakdown the capabilities, functions — ingredients, shall we say — into reusable and independently scalable components. Done effectively, any organization, can build, rebuild, adapt, transform, and scale. No wonder we’re seeing these shifts!
Indeed, you make a great point about the challenge of managing those “scattered” capabilities.
It’s a significant concern that Kong Konnect is tackling directly. The idea of a scalable "Service Hub" can help with this. Beyond a mere directory, the objective is to also govern, monitor, and enforce usage patterns. While chaos has its utility, especially in innovation, it cannot be the default, or only way to get work done predictably or reliably.
As with any tool, APIs need appropriate attention as you indicated. A virtuous lifecycle that takes them from inception to retirement. At some point, an API will be withdrawn to make way for new APIs that can do the job more effectively.
Automation is key for getting a virtuous lifecycle going. Automation reduces errors, increases responsiveness, increase quality — we could go on.
Your last question is an excellent one. How can organizations change so much so fast?
From on-premises to cloud or hybrid. From VMs to cContainers. From monoliths to APIs and microservices. From manual process to CI/CD. From waterfall to agile. More so, how to do these together?
Change is difficult. (Sorry/not sorry for the cliché.) But there’s hope. Getting an organization to make the transition requires a few things.
- First, an objective, a goal, a purpose, a target.
- Second, a team to make it happen. As the saying goes, if you have a dream, get a team.
- Third, tools and processes.
- Fourth, a plan.
- And finally, a change agent — the force to make it happen, and remove obstacles, that will inevitably surface.
Every organization will have a different plan, team, priorities, definition of success, and obstacles. These are wonderful problems to solve. And there’s no shortage of them.
From: Bradto: Ahmeddate: Nov 1, 2022, 9:38 PMsubject: Re: Elements for Building Cloud Native Applications
You raise some good points. And some familiar ones, based on recent conversations with IT leaders as well as what I read in the industry more broadly.
The one which really stands out is automation.
It's sounding like automation is the key to being able to innovate fast, keep quality high, keep compliance in check, AND maintain all of this across a very distributed and heterogeneous (from technical perspective as well as a team perspective) IT landscape!
Automation can mean lots of things to lots of people, but in this context it's really about ensuring every aspect of the API lifecycle is automated.
From design through testing, deployment, compliance, and enforcement, as well as automated aspects around monitoring and feedback. I would imagine the more you automate, the faster you move (and with less errors along the way). And the more you can shift-left the cross-cutting (but important) concerns, like security and compliance, the more you can automate early in the lifecycle, saving so much manual (and error prone) work down the track.
Broadly speaking, if an organisation forging their path to API success mastered API automation (APIOps is a great term for it!) then surely that's putting them in good stead.
In my mind any API program then distills down to:
- Being API-led as an organisation
- Being API-first in mindset
- Being design-first in creation
- Being discovery-led in adoption (think of the re-use!)
- Shifting-left where you can
- Embracing the modern, distributed IT landscape
- Being cloud-first as your deployment substrate
- Then: automate, automate, automate!
from: Ahmedto: Braddate: Nov 2, 2022, 2:07 PMsubject: Re: Elements for Building Cloud Native Applications
Automation, shifting . . .
These days around one of eight car models in the U.S. are offered with a manual transmission. And one of 25 five cars sold have a manual transmission. Not only are automatic transmissions easier to operate, they’re also more efficient as they are smarter. They shift faster and are able to match the optimal engine speed more often with the needed torque a vehicle requires.
The benefits of automation are hard to overstate. On the other hand, manual transmissions are so rare that cars without them have an inherit anti-theft mechanism. But I digress.
Back to automation, and in the context of APIs, I must agree. Once a lifecycle is established, automating it is a must.
There’s a good xkcd comic that illustrates this. Given the frequency of developing and revising APIs, the time savings alone per developer can be substantial.
Less obvious are the improvement in quality given the lack of manual errors (mis-shifts?) and the eventual morphing of the lifecycle being applied at a wide scale. Our colleague Melissa VDH agrees with us. She proves that speed and quality shouldn’t compete. It shouldn’t be a trade-off. And I agree.
I like the distilled API program you suggest. I wonder how it may overlay onto a maturity model? Here’s one example I found on this idea.
But can we do better? Perhaps we can think of a simplified version from Novice, Beginner, Intermediate, Advanced, and Expert. We then have a matrix which shows what activities and metrics are anticipated at each stage, and what transitional steps can be taken. The API program therefore guides a user from novice to Expert.
Things to consider. Until next time.
WATCH: Implementing an API-First Operating Model
Learn the critical steps to accelerating digital transformation through an API-first operating model. In this webinar, we discuss how the benefits of this approach and how organizations can equip developers with tools to deliver faster into production.