On Connectivity and Conflict

The Two Generals' Problem is a well-known thought experiment about how asynchronous - and potentially unreliable - communications can cause, shall we say, issues.

These great generals from history are facing a cunning and capable enemy. Think of someone like Khalid ibn al-Walid, Napoleon, Erwin Rommel or Sun Tzu.

If the generals attack together, they will win. If they attack separately, they will lose. The problem is that they are communicating over an unreliable and slow medium. The generals can never be sure that they won't make a mistake and attack separately. Here is what the communication might look like.

The generals continue to repeat this indefinitely because they are using unreliable and asynchronous communications. They can never be sure their last message arrived. There is always a risk they will attack alone and thereby be guaranteed to lose. (See it illustrated here.)

Now let's switch to the business of running organizations where leaders, who quote military generals just a little, have to wrestle with problems like this.

A non-optional component

Ask business and IT executives about their priorities today, and you may get answers along these lines:

• Scale
• Analyze
• Automate
• Increase speed
• Increase security
• Get closer to customers, suppliers and partners

Returning to our analogy, they are asking their cavalry (IT teams) to be better at acquiring, using, analyzing and actioning data. I wonder where they got this idea?

Let's have a look at the world's most valuable companies. There is one legitimate oil company in the top six. The other five are mining a new type of oil, also known as digital gold…in other words: data. Yes, data is not fungible, and it is static. The key point is that it can unlock legitimate value and power.

The writing is on the wall. All organizations know if they do not get their data game on, then they will slowly, and maybe painfully, decay. Maybe even a new well-funded disruptor will eat their lunch.

Data in motion

How do we get data? How do we get it reliably, securely and quickly? How do we get it with a large variety, volume and velocity? This happens to be an important question given that APIs constitute 83% of internet traffic.

I regularly have this conversation with organizations, and I see them not only looking for data but for a foundation on which their data acquisition, storage, analysis and usage is agile, scalable and portable.

Here is what we mean by that: They want to release fast and often, so they prefer Kubernetes to spinning rust - erm, bare metal. They prefer multi-cloud to on-premise only. They prefer open source and extensible technologies to proprietary and slow technologies. They prefer microservices to monoliths. They prefer DevOps to manual cycles.

Well, this makes sense. We have had data integration technologies for a rather long time, have we not? Let's have a look down memory lane.

• • EDI
  • CORBA
  • DCOM
  • ESB
  • EAI
  • SOAP
  • MOM (think *MQ*)

I am sure those familiar with these terms recall the promises made when using them. And to a good degree, they delivered; I worked for a vendor that had a handful of solutions based on these technologies.

But are they up to the task in today's reality of scale, speed and scope in the cloud native world? I would not be so sure. The conversations I am having on a regular basis with my to-be customers confirm this.

Change is not easy, and these technologies and the classic platforms, protocols and environments they run on are not going away tomorrow. A few organizations are using gRPC and GraphQL, while the mainstream uses REST, and some holdouts may still be on DCOM, CORBA, or gasp, something proprietary.

A smooth transition is in order.

Connectivity and conflict

In part 2 of this blog series, I will share the details of a specific battle from more modern history that exemplifies this principle.