KubeCon 2024 Insights: Spotlight on Platform Engineering, Large Language Models, and WebAssembly
Kubernetes was ‘born’ on June 6, 2014 and it has been on a wild ride ever since. KubeCon 2024 in Paris, almost, marks the platform’s 10ths anniversary, a great time to look back and connect the dots between then and now.
The Origins: Accelerate Dev and Simplify Ops
As a look at the Kubernetes website from 2014 (waybackmachine.org) shows, the platform was created and the code donated by Google to allow enterprises to “manage a cluster of Linux containers as a single system to accelerate Dev and simplify Ops.”
Kubernetes was described at the time as a platform that absorbs ‘an ocean of user containers’, and puts it on centrally ‘scheduled and packed’ nodes. The original ambition already was to manage workloads based on the declared user intentions. At Google I/O 2014, Brendan Burns and Craig McLuckie publicly introduce and demo Kubernetes for the first time.
The rise of Kubernetes coincided with the rapid decline of OpenStack as a mainstream infrastructure cloud platform, due to its inability to handle the hairball of legacy environments that can be found within enterprise environments.
The First KubeCon in 2015
KubeCon happened for the first time at the Palace Hotel in San Francisco, CA between November 9th and 11th. It was focused on educating ‘early production users and contributors’ in building and running ‘highly scalable distributed microservices apps’ on Docker containers and the Kubernetes scheduler.
There were exactly 16 vendors willing to sponsor KubeCon, a trivial sounding number compared to the over 300 sponsors of the modern day event. You will notice the absence of AWS, Google, and Azure on the sponsor list, and the early presence of RedHat and VMware.
It is interesting to note that at the time there was no talk about managing multiple Kubernetes clusters, full stack observability or running Large Language Models (LLM). In 2014 there were not even a handful of production clusters running enterprise workloads of any significance on Kubernetes.
Fast Forward to 2024: Platform Engineering Takes Center-Stage
10 years later over 90% of enterprises have adopted Kubernetes in some shape or form and organizations are working on making the management and operations of Kubernetes platforms an ‘outsourced commodity’ for cloud providers to handle. 2024 is all about creating application platforms to maximize developer productivity. The key goal of creating such platforms is to eliminate overhead tasks that take away developer time from writing business code.
The chart shows the most frequent and most important topics of all 12,320 Tweets tagged with #kubecon between March 3 and March 25 in blue. The same topics appear in orange to indicate the frequency and importance of these same topics within the 493 sessions from the official KubeCon 2024 (Paris) schedule.
WebAssembly: The Rising Star
WebAssembly is the rising star of cloud native technologies in 2024, as enterprises have started to comprehend the potential impact of running applications and microservices in the form of small binaries that consistently run on any device and operating system. Everyone involved in figuring out why a containerized app works great in one environment, but terribly in another very similar environment, will find this value proposition appealing. Using the component model to move all dependencies, such as code libraries, frameworks, and boilerplate code out of the application and into the runtime environment leads to apps that are much smaller in their storage footprint and can therefore be moved to where they are needed. This ability to move apps without changing any code between environments takes a massive load off developers as they no longer have to learn how to code for specific VM, container, and public cloud infrastructure. Now platform engineers can safely scale, move, and upgrade applications without the constant risk of them breaking, for seemingly no reason.
Large Language Models: The Up and Coming Workload
The KubeCon 2024 keynote focused showing that Kubernetes is ready to take on any workload. The original CNCF chart ends with AI Inference in 2023 but we could easily add 2024 as the year of WebAssembly as a new type of Kubernetes app workload. Point being that CNCF is laser-focused on promoting Kubernetes as the universal platform that is ready for business-critical applications today and also can be used for anything the future may throw at it, such as WASM and LLMs. Thinking this a little further, running LLMs on WASM, as seen with Llama Edge, and running WASM on Kubernetes could bring the flexibility we need to vastly expand LLM use cases. For example, running LLMs on WebAssembly, exemplified by initiatives like Llama Edge, along with deploying WebAssembly on Kubernetes, e.g. via Fermyon’s SpinKube, illustrates a transformative leap towards enhancing application flexibility and scalability, potentially revolutionizing the utilization and efficiency of LLMs across diverse computing environments.
OpenTelemetry, eBPF and Service Mesh: Observability Remains Key
OpenTelemetry, eBPF, Service Mesh, and Prometheus were big topics at KubeCon 2024 in Paris as they jointly aim to address infrastructure and app instrumentation as the ‘big bad wolf’ in may cloud native ‘rooms’. Jointly these technologies generate the telemetry data needed for optimal visibility, observability, and monitoring of complete cloud native app stacks.
OpenTelemetry
OpenTelemetry stands at the forefront of simplifying cloud-native application observability by providing a unified, vendor-neutral framework for collecting traces, metrics, and logs. Its comprehensive auto-instrumentation capabilities allow developers to focus on their core product rather than the intricacies of monitoring. By integrating directly into application code, OpenTelemetry automatically captures a wealth of telemetry data across various languages and frameworks, making it significantly easier to trace requests from end to end — even in highly distributed systems. This breadth of coverage ensures that every microservice, regardless of its underlying technology, contributes to a holistic view of the application’s performance and behavior. Moreover, OpenTelemetry’s commitment to open standards fosters interoperability and prevents vendor lock-in, empowering organizations to choose the tools that best fit their monitoring and observability needs.
eBPF
eBPF changes the observability and monitoring landscape by providing deep visibility into system and network performance without the traditional overhead of kernel-level instrumentation. Operating at the kernel level, eBPF enables the dynamic insertion of powerful monitoring and security capabilities directly into Linux kernels, allowing for an unprecedented level of insight into the behavior of applications and the system. This mechanism is especially valuable in cloud-native environments where understanding the interaction between applications and the underlying infrastructure is crucial for optimizing performance and security. eBPF’s non-intrusive nature means that it can gather detailed metrics and logs without modifying applications or the operating system, making it an indispensable tool for real-time, high-fidelity observability that scales with the complexity of modern cloud-native applications.
Service Mesh
Service Mesh technology, exemplified by solutions like Istio and Linkerd, abstracts communication and observability concerns from the application level to the infrastructure layer, offering a decoupled, transparent approach to managing service-to-service communication. With its built-in observability features, a Service Mesh provides fine-grained insights into the health, performance, and security of inter-service interactions, enabling automatic tracing, metrics collection, and logging of all traffic that traverses the mesh. This level of control and visibility is instrumental in identifying and diagnosing issues within microservices architectures, where the complexity and dynamic nature of service interactions can otherwise obscure the root causes of performance bottlenecks or failures. By centralizing telemetry data collection and empowering developers with tools for traffic management and policy enforcement, Service Meshes play a pivotal role in enhancing the observability and reliability of cloud-native applications.
Final Words
In times of publicly listed organizations watering down their commitment to open-source software, the success of KubeCon 2024 in Paris was crucial. Enterprises need open source software to collectively work on ‘digesting’ the massive hairball of legacy technologies they currently need to manage alongside freshly created cloud native apps.
Decisively positioning Kubernetes as the natural platform for running LLMs is a great move by CNCF, as the number of these models within and outside of enterprises will grow exponentially over the next few years. Kubernetes, potentially paired with WASM, brings the scalability and flexibility to efficiently handle these new workloads.
