Continuous Security for App Modernization— Why Red Hat Snapped up StackRox

Torsten Volk
6 min readJan 11, 2021

Red Hat acquired StackRox to address “compliance and security” as the #1 bottleneck slowing down the application release lifecycle, application modernization, and digital transformation as a whole.

“We cannot go faster than we already are, as compromising compliance and security could and probably would sink our company,” said the CIO of a large global telecommunications firm. His customers, internal and external, continue to ask for more software capabilities, but of course without lowering quality, increasing cost, or compromises in the security area. “Documentation is critical for us. When the next audit rolls around we need to prove compliance without scrambling for the details,” says the head of application platforms of a major SaaS provider. These are common stories that all boil down to forcing enterprises into the unfortunate choice between speed, cost, quality, and innovation. This is exactly the vicious triangle digital transformation (am I still allowed to dig out this much overused term) aims to shatter and it is also why Red Hat acquired StackRox. StackRox “injects” a set of cloud native services directly into Kubernetes (OpenShift). These services enable organizations to define, enforce, and report on application centric compliance controls across DevOps pipelines and Kubernetes clusters running in the data center and on public clouds. The following chart demonstrates the dire need for this type of protection, by mapping out all 921 Kubernetes-related security vulnerabilities recorded in the CVE database in 2020.

Google, Jenkins, and Git-related security vulnerabilities were the most frequent cloud-native categories in the 2020 CVE database.

“While we lost no data we did lose a lot of trust of our management when we had to report that the downtime was the result of one of our developers ‘tricking’ the Kubernetes scheduler into ‘believing’ that it had more resources available than it actually did. Changing that setting seemed harmless to the developer, but massively affected our business.” (DevOps team lead, professional services firm)

The Need for Kubernetes End-to-End Security

The chart shows the exploitability (x-axis), impact (y-axis), and overall criticality rating (bubble size) of all Kubernetes related security vulnerabilities recorded in the CVE database in 2020.

3 Key Observations

  1. Vulnerabilities can be everywhere.
  2. AWS, Google, and Azure managed Kubernetes offerings are not free from vulnerabilities.
  3. CI/CD and GitOps-related categories add more security vulnerabilities to the mix.

For anyone wanting to dig deeper into the security vulnerabilities illustrated in the above chart, I have created a categorize full-text list of all of the 981 corresponding CVE records in one large, but easy-to-read, table.

Why Red Hat Acquired StackRox — From #ShiftLeft to #StartLeft

The StackRox platform consists of a set of Kubernetes services that provide a policy engine and API to protect Kubernetes applications throughout their lifecycle (image source: StackRox.com)

The more Red Hat can show that its cloud native platforms can protect software development, build and deployment processes, runtimes, and operations and management platforms from the security nightmares derived from the above chart, the more comfortable these customers will feel to push on with their app modernization projects, as they no longer live in fear of “risking the farm”.

This is exactly where the StackRox core value proposition comes in. The StackRox platform offers a core policy engine and API, consisting of a set of Kubernetes-native services that play together to protect all corporate Kubernetes clusters, their container runtimes, and the entire DevOps toolchain. In short, StackRox protects all of the components required to release, run and operate applications safer by introducing a set of security guardrails (Kubernetes services) that prevent bad things from happening. I like to call this approach “Starting Left”, as it focuses on proactively protecting the overall DevOps process and infrastructure without the involved personas (DevOps engineers, developers, cloud platform ops, sys admins, security pros, etc.) having to scramble to “shift left”.

Why Starting Left Is Key

By definition, adopting distributed application architectures and running them in the data center and on multiple public clouds, delivers a massively increased attack surface for the outside to take advantage of. There are two major points of entry for such attacks:

Human Error

All it takes is one simple configuration mistake buried in the YAML code for the deployment, configuration, ingress control, monitoring, or any other aspect of the application lifecycle that is defined in a few thousand lines of YAML. Mislabelling application components is another frequent mistake with potentially dire consequences, as Kubernetes workflows rely on these labels to enforce policy compliance. Mixing up numbers in IP tables that define traffic routing, not testing control plane backups, and mistakes in the setup of role based access control (RBAC) are more examples of popular mistakes. Each one of these seemingly minor mistakes has the potential to become the entry point for a future attack or the cause of performance inconsistencies, reliability challenges, or scalability issues down the road.

“Had we known that one of our development teams was running old container images on our clusters to avoid having to upgrade a number of APIs in their application code, we would immediately have jumped on patching the platform when the CVE was issues. Instead we decided to apply the patch during the next update window, which turned out to be too late.”

Inconsistent Management and Governance

Humans make errors and blaming them for a small configuration error within increasingly complex environments is the wrong approach. Instead, we need a consistent management and governance platform that spreads its tentacles across the entire organization and provides us with the ability to observe and enforce all critical compliance and security rules. We need to automatically catch it when our test automation platform tries to spin up a Kubernetes cluster that allows outside network connections to our private cloud and we need to automatically prevent developers from including non-certified container images into their builds.

The Kubernetes universe is large and offers many options for developing, provisioning, running, orchestrating, and managing cloud native apps. The chart shows all CNCNF products based on the number of code commits over the previous 12 months

“As we were running on EKS, we thought the issue was on the AWS side,” says the application lead of a well-known SaaS provider. However, the AWS help desk told us that EKS was up and running and that the issue must be on our end. Ultimately, this turned out to be the case, as the performance inconsistencies originated from our development team using an outdated version of a library that controlled the integration between EKS and other AWS services. Pinning down this issues took us 3 days, while continuously receiving inquisitive phone calls from our management.” (Development Engineer, professional services firm)

The Start Left paradigm addresses both of these pain points by providing the required intelligence, automation workflows, compliance templates, and collaboration platform to ensure continuous security without compromising on speed, cost, or innovation.

Templates for NIST, PCI, HIPAA

StackRox compliance dashboard showing levels of compliance with individual controls (Source: StackRox.com)

Audits are not fun at the best of times and tend to become exponentially more painful the more fragmented and hard to obtain and consolidate the required reports are. StackRox provides standardized but customizable compliance checks and the corresponding dashboards for CIS benchmarks, NIST, PCI , and HIPAA regulations. From these compliance dashboards, authorized users receive recommendations in terms of how to fix current issues and prevent future ones. After receiving a preview of the potential impact of the recommended policy changes, users can make them live to immediately enforce a better level of compliance.

“All it took was one tiny mistake in one routing table for my department’s customer transaction data to be exposed to the outside. But worst of all, we were only made aware of this issue much later when one of our core business applications started ‘stuttering’ from the resource contention caused by an outside attack. As there was a spelling mistakes in one of the labels we never ”

Bottom Line: Let’s Start Left

I feel passionate about replacing today’s “shift left” paradigm with a new “start left” worldview. This new worldview relies on automatically preventing all of the terrible things developers, DevOps guys, system admins, or any other user role can accidentally (or on purpose) do to Kubernetes apps. StackRox leverages onboard Kubernetes capabilities to provide the security and compliance guardrails needed to “pull out the stops” and release software faster and at lower risk.

--

--

Torsten Volk

Artificial Intelligence, Cognitive Computing, Automatic Machine Learning in DevOps, IT, and Business are at the center of my industry analyst practice at EMA.