The pros and cons of modern application security testing

Enterprises and large organizations are starting to pay attention to the software supply chain—and rightly so. Open source software (OSS) may be “free as in beer,” but it’s certainly not free of risk. In fact, the majority of security concerns in modern containers are due to OSS software, which can occupy up to 90% of the container itself. A product team could write the most rock-solid, bug-free, secure app ever made, only to deploy it to a container with hundreds (if not thousands) of security vulnerabilities.

Realistically, organizations need to go well beyond “paying attention” to the supply chain. They need to start scrutinizing. Most modern tooling is insufficient from a security perspective despite the fact that developers are still developing code to production faster than ever.

Common strategies in today’s market: SAST, DAST, and IAST

Dev teams are pushed harder every day to deliver value to the market as quickly as possible. OSS has provided a springboard for reliably increasing developer velocity. However, there are several trade-offs and compromises:

• Security: frameworks and packages that accelerate development will likely contain vulnerabilities
• Quality: rapidly-developed code will likely include logical and/or functional errors
• Technical debt: biasing toward delivery usually means pushing bug-free code out to the future
• Bloat: dev teams may become overly dependent on large frameworks for small bits of functionality
• Visibility: a delivery-first mindset deprioritizes concerns for monitoring and performance insights

To accommodate, devs look to vendors for quick-fix tools that have the veneer of security, predictability, and trustworthiness. These quick-fix tools rarely pan out and the teams deploy bloated apps with literally thousands of vulnerabilities. Many companies do want to understand their security, quality, and performance metrics, but they don’t have the time, resources, or skills to deploy what they need to get a full picture.

The most popular tools for software vulnerability management are SAST, DAST, and IAST. While these tools and strategies are helpful, they are not comprehensive or thorough enough to adequately address an application’s software attack surface.

Static application security testing (SAST)

Developers can use SAST tools to analyze and identify vulnerabilities while in the process of writing code. They are typically programming-language specific and are useful for identifying important issues like buffer overflows and SQL injections. The applications of these tools can be limited, but the tools are getting better every year. They can also be integrated directly into a CI/CD pipeline, which is helpful for dev shops with high automation capabilities.

There are three main problems with SAST tools:

1. They can only identify a small scope of issues
2. They frequently generate false positives and false negatives
3. They help devs treat symptoms, not root causes

SAST tools can tell developers where the problems are, but it can’t tell them why the problems exist or how complete the recommended solution needs to be. Essentially, the tool can detect which lines of code are causing the problem, but not how to implement a completely hardened solution.

SAST tools typically require access to the uncompiled source code of an application, which is why they perform what’s called “white box testing.” There’s full transparency into the code that’s being tested, unlike “black box testing” which is entirely opaque.

Dynamic application security testing (DAST)

DAST is sometimes called a “cousin” of SAST and is known as “black box testing.” It’s typically used for identifying vulnerabilities after an application has been compiled and deployed. Developers hope their use of SAST tools has identified and remediated all known vulnerabilities in their code. They use DAST tools to verify the supposed lack of vulnerabilities once their app is compiled and deployed.

Unlike SAST tooling, DAST tools are not programming language-specific. They are largely agnostic to the technologies under the hood and use common protocols for finding vulnerabilities. Because SAST and DAST tools operate in a hand-in-hand manner, they are often bundled together by vendors in a single product or service.

Like SAST, there are some unfortunate limitations to DAST capabilities. For example, when a vulnerability is identified, it cannot tell the developer where the vulnerability exists. DAST can execute on a known vulnerability, but it cannot decompile an app or reverse engineer the pre-compiled code to find the problematic code. Also like SAST, there is a high degree of false positives and false negatives, leaving developers with the difficult task of reverse engineering issues.

Interactive Application Security Testing (IAST)

The shortcomings of SAST and DAST have led to another type of security testing: Interactive Application Security Testing (IAST). Developers use it as a form of “gray box testing,” which is a combination of the black and white box testing of SAST and DAST. As you might expect, it combines some of the dynamic capabilities of DAST with the static code analysis capabilities of SAST.

Gray box testing involves deploying and running the application while also having access to the source code. IAST can sometimes do a better job than SAST at identifying code locations for vulnerabilities, making remediation an easier task for developers. IAST tools can watch and replicate actions taken by testers while also assessing what is happening within the application itself.

IAST is dependent upon the programming language of the app being tested. As such, it depends on language-dependent sensor modules to detect what’s happening in the application and its context. This can make life especially complicated for product teams writing in multiple languages or using a variety of open source components in their workflow.

IAST is also very time-consuming. It doesn’t test areas of code that aren’t executed, so its success can be dependent upon the depth of QA test coverage. Organizations that are not completely satisfied with their QA test coverage will have a hard time implementing IAST because it will require even more QA rigor.

IAST does a better job overall than SAST and DAST, offering the best of both worlds. There are still shortcomings, but they’re a huge improvement over SAST and DAST. But regardless of how good it is, there’s a bigger issue at hand: actually fixing vulnerabilities.

SAST, DAST, and IAST aren’t helpful for OSS

The biggest problem with AST tools is not to do with anything we’ve discussed so far. Unfortunately, there’s a much bigger problem: AST tools cannot help dev teams when vulnerabilities exist in OSS code they didn’t write. And when it comes to the code dev teams actually did write, fixing bugs and remediating issues is expensive.

Many vendors suggest using AST tools along with an SCA scanner to identify vulnerabilities in customer code and OSS components. As you may now be starting to see: this is getting very complicated and expensive.

AST tools are meant to help developers find issues in their own proprietary software and source code. The tools aren’t used for OSS because it’s not realistic to expect developers to solve problems that exist nested deep in a third-order software dependency. These tools can’t even identify which components are the cause of a vulnerability.

If 50-90% of code in a container is OSS, what is a dev team supposed to do with AST results? Even if someone is willing to wade through OSS code they haven’t written, there’s no guarantee they’ll know how to fix it. Worse, if they somehow build a patch that fixes the issue, what happens when there’s an update to that OSS that’s automatically pulled by a software package manager in the automated build process? It becomes a nightmare.

The best way to remove OSS vulnerabilities

There’s one verifiable way to remove OSS vulnerabilities and it’s relatively simple: remove all the unused software components and focus on what’s left. In our experience with container slimming and optimization, anywhere from 50-80% of software components in a container go unused. However, if they are present in a running container, they offer an opportunity for a bad actor to exploit a software supply chain attack.

The relationship between the number of vulnerabilities and the number of software components is nearly linear. We typically see a 1:1 relationship between the percentage of removed components and the percentage of removed vulnerabilities. In other words, if we remove 60% of the components because they’re unused, we typically see a 60% reduction in known vulnerabilities.

Obviously, removing components does not fix all the vulnerabilities, but it reduces the scope of the wild goose chasing. RapidFort relieves engineers from the need to fix so many vulnerabilities.

Application security testing has diminishing returns

No amount of AST tooling is going to resolve the overwhelming number of security vulnerabilities that come with OSS in software containers. It’s not feasible. Regardless of how many vulnerabilities AST can identify and help a dev remediate, there will still be countless OSS vulnerabilities that are much more likely to be hacked, especially when there’s a POC available.

The Linux Foundation has reported that a majority of companies do not scan their containerized applications for vulnerabilities, but many are planning to within the next year. For the time being, they are using patches, but patches can only go so far. They are expensive and many needed patches are unavailable. It’s not as much a wild goose chase as it is whack-a-mole.

RapidFort can help you today to generate a software bill of materials (SBOM) for free. We have a software composition analysis (SCA) scanner that’s forever free, too.

‍