How to Trim the Fat from Your Container Infrastructure

How to Trim the Fat from Your Container Infrastructure

When factoring the total cost of ownership(TCO) for a piece of software, one must consider the short-term and long-term direct and indirect costs. There are, of course, obvious costs like purchasing licenses, paying the salaries of the people who manage it, and the spin-up costs of the infrastructure to run it. Then there are the less-obvious, indirect costs like maintaining the software, applying patches, remediating vulnerabilities, addressing alerts, scaling the infrastructure, and being license compliant. And all those painful meetings in doing such!

These non-trivial, indirect costs are a second-order consequence that we affectionately call “software weight.” It’s a concept related to TCO and describes how these indirect, long-term costs slow down an organization. Software weight is an invisible burden that must be pushed forward through constant growth, not unlike Sisyphus pushing a rock up a hill only for it to roll down again . It’s not quite the same as “tech debt,” either because that’s usually about known risks or costs that are kicked down the road to be dealt with later.

Software weight, although harder to identify is just as important as it’s a product of the way modern software is developed.Thankfully, it’s more soluble than these other more obvious costs.

Every major software workload today includes open source, and as much as 80% of that code is completely unnecessary in production workloads. But, a lot of time and attention needs to go into managing that unnecessary, unused code. Patches, vulnerability management, licensing, endless meetings, and software attack surface management are required for all deployed software, whether it’s used or not. So, developers end up spending 10-15% of their time patching and maintaining code and container security. Unnecessarily.

As you can see, there’s plenty of superfluous, bloated software weight that an organization carries with it every day.

Now that we know what software weight is, let’s talk about why it exists, how to identify and measure it, and what you can do to eliminate it. And we promise it’s a lot easier than going to the gym.

Why Software Weight Exists

At a salad bar, we can take a handful of spinach, a few cherry tomatoes, some ranch dressing, and put it on a plate for lunch. We take what we need (okay, sometimes a little more) and have a nice meal. At the OSS component buffet, however, if we want some spinach, we have to take the entire salad bar! And if we want a small bowl of soup, we have to take the entire soup cart—including that thick green stuff that’s congealed from lack of heat.

The reality is: it’s very easy to add software and it’s so difficult to remove it. Worse, it just piles up as a product evolves, gathering more weight to carry around. Sometimes the weight accumulates because a lead developer left and the backfill doesn’t understand how to carry the work forward. Other times, a junior member joins the team and  isn’t sure how to best extend existing functionality, so they make an educated guess as they bolt on new libraries—full of months-old soup.

Any experienced person who’s familiar with OSS components is aware of this problem. It’s tolerated because there’s never been an easy or effective way to reduce software weight. Noodling through code is very difficult and time-consuming, especially as the broader code base evolves and grows through new releases.

Besides, even if a developer wanted to remove software weight, it is often unclear whether it will interrupt core functionality. Developers rarely have total transparency into the final production environment in which the product actually runs. It gets even more complicated when the product is deployed into ephemeral, distributed, and orchestrated environments, like Kubernetes. A container might spin up or down, talk to other ephemeral containers, and be part of an interwoven web of interactivity only lasting for minutes. 

Dynamic environments make software weight difficult to observe, manage, and understand. Employee turnover increases software weight and product changes—including EOL’d technologies—increase the risk of removing the weight. If you can’t determine what software is used, that extra weight just sits there, and the vulnerability and patching backlog grows.

How to identify and measure software weight

Until recently, there was no way to “weigh”your software container. Unless an organization starts with minimal “distroless” images, there’s a high probability that a container will use a full-fat, bloated base OS image. Though minimal, low-weight container images are a great idea, maintaining them across every build, engineer, and deployment is very expensive and requires a lot of overhead. It’s hard to recruit expensive engineers to support that development path and it’s expensive to do, which is why only a handful of select companies like Google and Coinbase do it. 

To understand the software weight of a container, you need access to helpful visibility tools and augmented real software bills of materials (RBOMs). With these, you’ll get some important insights into component usage, vulnerability scans, and patching reports. We’ve developed and used these tools every day for some time, and we're seeing that the average container comes with about 120vulnerabilities to fix. Triple-digit vulnerability counts in a single container are intimidating and the stress of remediation can be overwhelming, especially for startups with a high risk of ruin in the event of a breach. But there is hope in sight; we’ve been using our tools and the load is truly lightened when we can take one of our own 1GB containers and reduce it to 70MB.Relief!

Scans of large enterprise environments can literally have more than 1.5M vulnerabilities to remediate. We’ve seen reports showing large infras with more than 300,000 components not running on the latest patches. These numbers are so extraordinarily high that teams simply learn to live with the risk. To fix numbers these high would be insurmountable.

The industry has responded to these numbers with a “shift left” mentality, forcing security and container management onto development teams. Though “shift left” is a movement with good intentions, it’s not realistic to expect software teams to be experts in software development, containerization and microservice security, and runtime environment configuration. And you cannot shift left 1.5M vulnerabilities regardless of what the twittersphere tells you. You need a different approach.

How to cut software weight without risking functionality or velocity

The good news is that we now have the ability to remove software weight without risking production functionality or dev team velocity. Our product, RapidFort, has scanning and profiling capabilities that shows you what’s in your containers and compares the result to what’s actually being used. RapidFort also comes with an estimation engine that uses a machine learning model to determine how much of the container can be optimized. You immediately see what weight can be lifted.

The estimation report is a statistical model that incorporates programming language, open source technology and operating system. It tells you the range of percentage of the container that can be safely eliminated. For example, after running the scan the model will give you an estimate that between 72% and 92% of the vulnerabilities and patches can be removed. We love it. If all diets could be so motivating!

A quick caveat: there’s a big difference between efficient software and optimized software. High optimization is not an indication of high efficiency. Efficient, well-designed software is measured in compute overhead and speed, whereas optimized software is measured by how many unnecessary files are used.Efficient or not, software optimization leads to less code, less risk, fewer problems, and a reduction in your software attack surface.

There is no downside to knowing what’s running in your production environment. When you have a profile from production that tells you everything you’re using, you can get a delta from your software test coverage and start the optimization journey. Focus on what you use. You’d be surprised at how well optimization works—we certainly were when we started optimizing our own infrastructure.

In fact, we’ve had such fantastic results, a friend of ours called the product “a diet pill for your infrastructure.” More enthusiastic users have described it as “like magic” and “incredible.” We feel we are at the start of an important trend in software development where it's as important to manage what you put into software as it is what you take out.

You would be amazed to learn that a popular technology like NGINX ships as 114MB but only takes 11.2MB to run as a web server, and 1.3MB to run as a load balancer. The most incredible statistic is that to support both of these use cases, it only requires 11.3MB. When you strip away these unnecessary, bloated dependency trees, there is a lean and mean core software base dying to get out!

‍

‍

Optimize your containers for free and see for yourself

At RapidFort, we’re developers and we’ve struggled with our own issues around software weight. We’ve felt as over whelmed and frustrated as any other developer who cares about software efficiency, security, and bloat. So, we developed the product to solve our own frustrations and challenges with unused code in our containers. We’re hoping a lot of people use our product, gain from its benefits, and give us feedback to make the product better.

If you’d like to give RapidFort a try, checkout our free trial. Or, if you’d like to give your infrastructure a scan, you can download our scanning engine for free. 

Have any questions? We’d love to talk to you.Just send us an email at info@rapidfort.com

‍