Turning DevOps Into A Real-Time Multiplayer Game: System Initiative Rewrites The ‘Laws of DevOps’
Adam Jacob, founder of Chef, is pretty high up the list when it comes to personalities that have the credibility to fundamentally redefine DevOps. Jacob’s new venture, System Initiative (SI), certainly does not disappoint when it comes to throwing out all of the traditional seaweed on the DevOps anchor in favor of a fresh new start.
Do We Need A Fresh Start?
The idea of throwing out current DevOps structures, processes, and toolchains seems like an odd proposition. Why don’t we just fix what’s broken and minimize disruption? Maybe we can double down on DevOps Dojos, Centers of Excellence, and similar techniques to propagate successful DevOps across the enterprise. Jacob argues that we need a fresh start to completely eradicate the issues that have been plaguing DevOps for one and a half decades now. He proposes to replace today’s sequential approach toward DevOps, where individual players complete their own tasks and push a button that alerts the next player to start on her or his own set of tasks, and so on. Instead, SI seeks to bring all players together within a single collaborative workspace, similar to tools like Figma or Miro, to jointly and in real-time define new application environments.
Isn’t This Just IaC On Steroids?
While this may sound like a more collaborative approach toward Infrastructure-as-Code (IaC), the System Initiative platform is fundamentally different. Instead of writing YAML, product teams receive a workspace that allows users to define application stacks by combining assets from a palette including AWS services, Kubernetes clusters, Linux operating systems, Docker container images, and other components into an app stack. SI enables the creation of complete models of application stacks within their overall context of private and public cloud environments. But what does that mean and why should you care?
The Not So Evil (Digital) Twin
System Initiative introduces the concept of creating a digital twin of the company’s current application environments. You can think of a digital twin as a data model that continuously keeps track of all relevant configuration of each individual building block within an application stack. For example, if you misspell the name of a Docker image, a Helm template, a security group, an environment variable, or a security group the System Initiative engine immediately alerts you of your mistake, based on the digital twin that includes all relevant context of your overall environment. System Initiative goes beyond alerting as it can leverage the digital twin of an application environment to automate the creation and configuration of the individual components of an app stack. E.g. the platform can automatically configure the Linux OS including the container runtime required by a Docker specific Docker image. While this example shows the basic value of System Initiative, the actual excitement comes when creating the AWS cloud environment needed to run our containers.
Correctly configuring EC2 machine instances, operating system images, security groups, load balancers, and all the other components needed for creating a complete cloud environment requires a lot of attention to detail. And this is where it gets really interesting. When I expose a specific port in my container image, System Initiative will automatically update all affected areas of the environment, including the Podman container runtime, the underlying Linux OS, the AWS instance including its own operating system image, and of course, the ingress controller. This eliminates all of the little but impactful mistakes you could make when opening ports by hand.
But could I not achieve the same degree of automation through traditional IaC? No, because traditional IaC platforms do not track all of the different relationships between application components. Therefore, users need to know where to make changes, which opens up a significant risk of overlooking something or making a simple but impactful configuration mistake. Once the model is complete, SI can deploy the required resources on AWS, all in the correct order to ensure flawless connectivity.
Is It Too Good To Be True?
In the past, we have seen numerous AI ventures drop out of the quest for the automatic deployment and continuous optimization of complex app environments because of the enormous complexity of creating a sufficiently comprehensive digital twin. Therefore the question becomes: Can SI pull off what others couldn’t?
Quick Take
Completely redefining how we think about DevOps is a daunting task that will require buy-in from developers, platform engineers, and DevOps engineers. For these personas to stick out their collective necks and demand the adoption of this new real-time paradigm to DevOps, there needs to be a clear payout in the form of getting their work done more easily, quickly, and at higher quality and lower risk. This is the exact promise of SI, as this platform aims to automatically take care of the error prone configuration tasks that come with the adoption of distributed microservices environments in hybrid multi-cloud environments. Tracking all of the dependencies within the app stack of each individual microservice requires significant, often manual effort, on the side of developers and DevOps roles, slows down the organization’s ability to quickly respond to new developer requirements. SI’s simulation-based approach automatically tracks these dependencies, enabling a high degree of automation and validation of changes to the application.
But Is It Feasible?
When evaluating the feasibility of using the SI platform to replace today’s DevOps processes and toolchains it is key to be realistic. While Adam’s demo looks amazing, we need to remember that the SI platform is still at the very beginning of its lifecycle and does not yet have the ambition to absorb each and every use case. In the contrary, SI provides the users of its private beta with a core construction kit that provides a glimpse of the ultimate value of this platform. For SI to cover a large range of use cases it will be crucial to get enterprises excited about the implication of nearly complete infrastructure automation and validation to make them want to contribute staff hours to the SI open source project that aims to create the scripts that implement the policy rules, validations, and automations needed for broad production use of the new platform.
