Argo Use Cases for GitOps & ML

In this Kongcast episode, Henrik Blixt, Product Manager for Argo at Intuit, gives an introduction to Argo, an open source tool for Kubernetes and incubating project of CNCF.

Check out the transcript and video from our conversation below, and be sure to subscribe to get email alerts for the latest new episodes.

Kaitlyn Barnard: Today, I’m super excited to welcome Henrik Blixt, Product Manager for Argo at Intuit. Henrik’s here today to talk to us all about Argo, which is an open source tool for Kubernetes and an incubating project of CNCF. Henrik, do you want to introduce yourself?

Henrik Blixt: Thanks for having me. I'm really excited to be here. My name is Henrik Blixt. I’m a Product Manager at Intuit and I’m responsible for Argo, both internally within Intuit and I’m also a maintainer of the Argo project. I do a lot of product management in the community. I work with the community partners and CNCF and things like that.

Kaitlyn Barnard: I would love to hear a little bit about how you ended up in this space and what your background and role in Argo looks like.

Henrik Blixt: I’ve been in and out of the open source community for a long time, ever since I first laid my hands on Linux back in the mid-nineties. That’s been a thread that’s gone through my whole career in and out with Linux operating systems. I worked with OpenStack for quite a few years, and now lately we’re doing a lot of work around Kubernetes and Argo, of course.

I’ve always been a vendor throughout my professional life. Vendors make the open source community tick. A lot of the people that contribute to open source are from vendor companies and they do a lot of really good work for the community. But when you’re working for a vendor, there’s always, especially as a product manager, there’s always that kind of selling thing in the back of your head. When I saw this opportunity to be a product manager for an open source project from an end user instead, that was a really interesting twist to me and something that was a really interesting challenge. Coming at it from building a better project, building a better community. I don’t have to worry about selling. It’s all about making Argo great.

Kaitlyn Barnard: Absolutely. I think that’s one of the most interesting things about this CNCF ecosystem in particular. There’s this common knowledge around the business value of this open source ecosystem and the projects that come into play. I love seeing all of the vendors now that are really dedicated to the upstream efforts and to maintain this community, because we’ve all agreed how important it is and what the value is there.

Henrik Blixt: It’s great to see. The community is growing. We have a large number of vendors that are involved in Argo as well as in the community. The community keeps growing, the projects keep growing so it’s really exciting to be part of it.

What is Argo?

Kaitlyn Barnard: How about we start off from the beginning here. Can you give us a high level overview of what exactly the Argo project is?

Henrik Blixt: Argo is a little bit of an oddball when it comes to CNCF projects because generally you have a project that is solving a specific use case or specific problem. Whereas, Argo is actually a collection of four projects that solve different problems. Those are Argo CD, which is more on the continuous deployment side. We have Argo Rollouts, which is used for progressive delivery. Argo Workflows, which is a Swiss Army knife of a workflow orchestration engine. And Argo Events, which is an event framework. Those come together under the Argo umbrella. Argo itself was created about five years ago at a startup that was later acquired by Intuit. We’ve been an incubating project with CNCF now for almost two years to the day. Wow.

History of Argo

Kaitlyn Barnard: What is the history behind Argo? What was the need and reason for its creation in the first place?

Henrik Blixt: It was started about five years ago at a startup called Applatix. The first project in Argo is actually Argo Workflows, which was the first project that Applatix was working on. Intuit was going through a massive modernization and cloud native journey. We realized we needed to change a lot of how we did our software development to be able to innovate faster and be in lockstep with technology. Applatix was acquired by Intuit and basically tasked with building a whole new development platform built on cloud native technologies. During the course of building that platform, additional needs popped up. Argo Workflows was an initial part of the platform and then we realized when we started looking at GitOps that there weren’t really any good tools out there so Argo CD was born. Then as we started looking into how we roll out applications, especially at a FinTech company like Intuit where we have high demands of uptime, especially during tax season. We started looking at how to build on and improve how Kubernetes does rollouts of new applications and new versions. Hence Rollouts was born. During this whole phase, Argo Events was actually developed by a company called BlackRock and donated to the project. It’s really a process of Intuit realizing these were the needs we had and then building projects that fulfilled those needs to build this internal platform that we now have 5,000 developers using daily. Everything we run is now run on Argo, with Argo, and on Kubernetes.

Argo Use in Production Increased 115% in 2021

Kaitlyn Barnard: I love that. That’s such a nice example of all those vendors coming together within this project. That’s really neat. I feel like I have been seeing Argo everywhere. What has been that recent catalyst of Argo’s popularity?

Henrik Blixt: CNCF released their annual survey a few weeks ago and I saw that the production use of Argo increased 115%. Looking at the responses, you can see that over half of the companies that responded are either using Argo or evaluating it, which is just mind blowing coming from the vendor side. If I had those numbers back in my vendor days, I’d be really, really excited. I think there are a few things that are coming together to drive that. Argo Workflows has been around for a long time but ML and AI is becoming very commonplace now and people are facing the same automation challenges that we faced in the whole drive of DevOps on the software development side a few years ago. There’s a lot of drive to get to the ML Ops side of things. I don’t think that anyone that’s involved in cloud native has missed GitOps or hasn’t heard about GitOps. There’s even a separate conference at Kubecon now specifically for GitOps. I think that’s been another big catalyst on the CD side, especially where a lot of companies are looking into and adopting GitOps.

And it’s a great project. We have a very diverse community. If you look at the maintainers, for example, coming from something that was all Intuit, 100% dominated by Intuit. Now only a third of the maintainers are from Intuit and the rest of the maintainers are spread across a large number of companies. The fact that we have a very strong and very diverse community certainly helps as well. And then just the fact that Argo is really good at what it does.

How to Use Argo for GitOps and ML

Kaitlyn Barnard: It really speaks to that value piece of the project itself, but also how powerful the community coming together is to help build that. I think one of the most interesting things about Argo is really all of the facets of the project. And as we’ve been talking, you’ve kind of mentioned there’s two ways to look at it, right? The GitOps piece and then the Machine Learning piece. Could you talk through those two different use cases and the current state of Argo as a project that fits into that?

Henrik Blixt: We generally divide up the four projects, even though they’re slightly different. Argo CD and Argo Rollouts are the ones that fit into the GitOps category. Then we have Argo Workflows and Argo Events that are more on the ML Ops side because it’s generally how people use it.

Argo CD was basically built for GitOps. It’s a continuous delivery tool for Kubernetes that adheres to the GitOps principles and lets you manage your applications with Git as the source of truth. Of course, all being cloud native, all being built on Kubernetes, which makes all four projects easy to deploy anywhere. That's one of the core pieces of the GitOps piece.

Argo Rollouts builds on top of that because there are some deficiencies in how Kubernetes does deployments. Argo Rollouts plugs into Argo CD and your application deployments, allowing you to do, as the name implies, progressive delivery and a gradual rollout of your application. You can do some really neat stuff in terms of slowly progressing a deployment of a new version. You do 10% of your new version and then you can run some automated checks, make sure everything is healthy, and then you progress to the next level. Every time you increase the number of pods or the amount of traffic, you do some checks to make sure it’s healthy. That enhances what’s already there in Kubernetes and Argo CD so those two go really nicely together.

Argo Workstreams on the other hand is a container native workflow orchestration engine. Even though we say Argo CD and Argo Rollouts are the GitOps side of things and Workflows and Events are ML Ops, they’re not really end-to-end solutions. If you want to do GitOps, you need something more than Argo CD because we don’t do CI, for example. You generally would hook some CI pipeline into that. And the same thing goes for Argo Workflows and Events. They’re not the end-all-be-all for ML Ops, but we’ve designed the project in such a way that they’re building blocks so you can take them and build a GitOps platform like we did at Intuit. The same thing goes for the ML Ops side of things. We have Argo Workflows, which is also cloud native and is used for a lot of other things than ML Ops as well. There are a fair number of users that are reporting using it for infrastructure automation or actually using it as the CI piece on the GitOps side. So you can combine them too. That can also show some of the strength of the project, how they’re really building blocks that you can switch in and out as you build your larger platform.

Kaitlyn Barnard: It’s really helpful to understand those two pieces, GitOps and ML, and where the individual projects fit in. But I like that additional explanation of how flexible it is and really the building blocks at the end of the day.

Henrik Blixt: One of the key things is automation and that’s GitOps, DevOps, DevSecOps, ML Ops, it’s one of the key value points. It’s the same thing with Argo Rollouts, it’s automation. How do we become faster? How do we become more agile without losing traceability and some control? Because it’s easy to move fast, but moving fast in a repeatable fashion or making sure that you have control over how you move fast – that’s the hard part. As you’re moving faster on the CD side and deploying applications faster, you still need to keep track of a lot of things. The same thing goes on the ML side to start automating your pipelines and doing that to build repeatability and adding checks into that as well. And that’s where all this comes together.

Kaitlyn Barnard: Today, we’re going to focus on that GitOps use case in particular in the demo. Do you want to give us a brief introduction to what you'll be showing us today?

Henrik Blixt: I'm going to show a couple of demos. I'll start with Argo CD and show from a UI standpoint how you can deploy an application, what it looks like, and how you can roll back if something goes wrong. Generally, you would do this in an automated fashion through a CI pipeline or something like that, but to make it a bit more visible today, we’ll do it in the UI. Then on the Argo Workflows side, I’ll show some fairly simple ways you can use Argo Workflows but give a good understanding of the strength and the power of the orchestration engine.

Demo: Argo CD and Workflows

Thanks for Joining Us!

I hope you'll join us again on April 18 for our next Kongcast episode with Jessica Deen from Microsoft.

Until then, be sure to subscribe to Kongcast to get episodes sent to your inbox (and a chance to win cool swag)!