A while back, I eliminated almost 80% of my Redis container code and it ran just fine. Despite having just 20% of the original container’s components, Redis still did everything I needed. Deleting all that code also made the container more secure, eliminating 77% of the known vulnerabilities. These astonishing results are made possible by RapidFort, which surgically eliminates unused, vulnerable components with precision.
Hardening containers with RapidFort is super easy. Maybe it’s not “so easy your grandmother could do it,” but anyone building a containerized app can do it in a few minutes. I’ve personally hardened thousands of containers—not because I’m a genius, but because the process is so simple:
- Install RapidFort’s instrumentation tools in your container
- Run some coverage scripts to see what components your app uses
- Delete everything else
Steps 1 and 3 are self-explanatory, but Step 2 usually needs a little explanation. Most people ask: What’s a coverage script? Is that a QA thing? Do I need to be a programmer? What programming languages do I need to know?
I’m going to answer all these questions about coverage scripts, container hardening, and how you can use RapidFort to reduce your software attack surface.
What’s a coverage script?
Put simply, a coverage script puts your container to work by covering its intended use cases. A coverage script is not a test script, nor is it used to check for accurate functional software. You don’t need to prepare any data sets, you don’t need to record sessions, and you don’t need special tools to write or execute the script. All you need to do is know what your container is supposed to do.
Coverage scripts exercise your container by invoking executables, libraries, and scripts in the container. A good coverage script covers all of the functionality you need from the container. Back to my Redis example, there are dozens of ways to use Redis (e.g. database, cache, message broker, queuing system), but if I only need it for a reverse proxy, then my coverage script only needs to cover that use case.
RapidFort uses coverage scripts to see what components were used during the script’s execution. It tracks exactly which executables, libraries, frameworks, and dependencies were invoked so it knows what components to keep and which to delete. RapidFort also knows which vulnerabilities are associated with which components, which is important for hardening decisions you need to make.
Once a coverage script is developed and maintained, it can automate your OSS vulnerability remediation in perpetuity. RapidFort is designed to plug into CI/CD pipelines, which means that coverage scripts become an important part of securing your production workflow.
How coverage scripts fit into your SDLC.
How to write and execute a coverage script
Writing a coverage script is quick and straightforward. We write most of them in under an hour. The most complex coverage script I’ve written took less than one day to write.
The best way to learn how to write a coverage script is to look at one. We have dozens of coverage scripts in our Community Images project, but for now, let’s review the script I wrote for the Redis example:
SET Key0 Value0
GET Key0
DEL Key0
PING
ACL WHOAMI
ACL SETUSER virginia on +GET allkeys
ROLE
INFO
SADD testmember "test set"
SETEX name 10 kyle
LPUSH test_member test_group
HSET test test test
ACL GENPASS
ACL LIST
MODULE LIST
BGREWRITEAOF
CONFIG RESETSTAT
To a non-Redis user, this might look like assembly language, but these are just native commands for Redis (e.g. SET, GET, DEL, ACL). The commands in this script don’t do anything particularly interesting or complex, but they cover the functionality I need with my Redis instance. I need to:
- Get, set, and delete key/value pairs (GET, SET, DEL)
- Ping other systems (PING)
- Use access control lists to create users and use rules to provision access (ACL)
- Determine the role of a Redis instance within a cluster (ROLE)
- Get basic information and statistics about the server (INFO)
- Add members and set stored keys (SADD)
As you can see, I’m not doing anything special in this coverage script. I’m using dummy values to ensure my container can perform these basic functions. This coverage script isn’t representative of a typical workload, but it is representative of the typical functions the container must execute in its day-to-day operation.
There’s some additional functionality we wanted to retain for this container, so I also execute these shell commands to invoke some Redis-specific applications for benchmarking, making the CLI available, and high availability management with Redis Sentinel:
redis-benchmark --help
redis-cli --version
redis-sentinel --version
Executing this coverage script is also straightforward. We are using Kubernetes, so here’s the command I’m using for kubectl:
# run script
kubectl -n "${NAMESPACE}" \
exec -i "${RELEASE_NAME}"-master-0 \
-- /bin/bash -c "cat /tmp/test.redis | REDISCLI_AUTH=\"${REDIS_PASSWORD}\" redis-cli -h localhost --pipe"
Anyone working with containers should be able to understand what’s happening here.
Other example coverage scripts
If you’re not familiar with Redis, that’s perfectly fine. We have many other coverage scripts you can see in our GitHub repo. Take a look at coverage scripts for these popular containers:
What happens after the coverage script?
In the introduction to this article, I gave an oversimplified view of how RapidFort works. Here’s the more formal explanation, which should illustrate more clearly how coverage scripts fit into the container hardening process:
Before we talk about what happens after you run your coverage script, I want to establish the context in which it runs.
Start with a stub
Container hardening starts with instrumentation. Using the rfstub command, we create a “stub image,” which is your original image with additional dependencies necessary for RapidFort to trace the runtime behavior.
Coverage scripts are then run on the instrumented container, creating a workload profile. At this point, you gain complete visibility into what components are used and which vulnerabilities are “hot,” corresponding to software components in the execution path. Even if you don’t deploy a hardened image, you can use this information to simplify interactions with your security teams for remediation requests.
Use the workload profile to harden your container
Once you’ve run the coverage script, you’ve created a “workload profile” or “runtime profile,” which is essentially a map of what files are used and which aren’t. This is what RapidFort uses to determine what can be deleted.
At this point, you can log into your RapidFort dashboard to see everything in the workload profile.
RapidFort is intelligent enough to automatically harden and optimize the container, but you can also click the buttons yourself, build optimization profiles, and dig through the SBOM. It’s really up to you. For your first few containers, I recommend doing it manually so you can understand what’s happening under the hood.
When you’re ready to harden, you can run the rfharden command. This is when the actual hardening happens and you get a new, smaller, and more secure container.
Conclusions
Coverage scripts serve two purposes: 1. Generate a workload for your container that reflects its expected functionality, and 2. Produce a workload profile that’s used in the hardening process. They’re simple to produce and they’re nowhere near as complex as a QA test script.
Here are the takeaways and a couple more tips to help you get started:
- Coverage scripts generate payloads that exercise the workload, covering its functionalities.
- Coverage scripts are NOT test scripts. They require no verification of results and are easy to produce.
- Developers of coverage scripts identify and invoke executables, shared libraries and scripts needed by the workload.
- Ensure that periodic and cron jobs are invoked. This includes screen sessions and shell scripts that might be running periodically and not as a cron job.
- If your workload has scripts to update SSL certificates periodically, make sure that they are invoked.
- If you are unsure about which data, configuration, and UI files (html, jpeg, etc) are needed, use hardening profiles and/or the --keep-data-files option to the rfharden command.
- Refer to the rfharden manual page for further control over the optimization process or use “rfharden –help”
Want to get started hardening containers for free? Our free tier includes hundreds of free optimizations, and includes SBOMs, container registry scanning, CI/CD templates, and more.