You Do Not Have Four Thousand Vulnerabilities
// elusive thoughts // appsec // sca // the number is lying to you
Somebody turned the dependency scanner on last quarter and it came back with four thousand one hundred findings. Six hundred of them critical or high. The dashboard went red and has stayed red ever since, the way a smoke alarm with a flat battery stays red, and everybody has stopped looking at it for exactly the same reason.
Here is the arithmetic nobody does out loud. Six hundred high severity findings, at a generous two hours each to investigate, patch, test and ship, is twelve hundred engineering hours. Thirty engineer weeks. You are asking a company to spend most of a person year fixing things you have no evidence affect you at all. And next month the scanner will find more.
The engineers worked this out before you did. They triaged the first fifty, found that most were irrelevant, and quietly concluded that the tool cries wolf. That conclusion is correct. It is also permanent. You did not buy security. You bought a very expensive way to teach four hundred developers to ignore a security tool, and they learned it fast, because they are good at their jobs.
The scanner is answering a question you did not ask
A software composition analysis tool reads your lockfile, resolves the tree, matches every package and version against a vulnerability database, and reports. That is the whole algorithm. Notice what is missing from it. At no point does it check whether your code can actually reach the vulnerable function.
A CVE is not a property of a package. It is a property of a specific function inside that package. The remote code execution in a library's XML parser is your problem only if you parse XML with it. If you pulled the library in for a date helper and never touch the parser, that vulnerable code is sitting in node_modules like an unloaded gun in a locked safe in a room you have never entered. The scanner sees a version string and it fires. It cannot tell the difference, and it was never built to.
the pattern
Every finding your scanner reports is the answer to "is this vulnerable code present." The question you actually need answered is "can this vulnerable code run." Those are different questions and the gap between them is where your entire backlog lives.
What the number looks like when you measure it properly
Reachability analysis builds a call graph. It starts at your real entry points, the HTTP handlers, the CLI mains, the queue consumers, and it walks outward through your code, into your direct dependencies, and on into the transitive ones. Then for each CVE it asks one question: is there a path from an entry point to the vulnerable symbol.
If yes, the finding is real. If no, the vulnerable code is present but dead.
Run that across an estate and the numbers stop being frightening and start being useful.
612
high and critical findings reported by the scanner
178
actually reachable from any entry point
~70%
noise, with no path to the vulnerable code at all
Seventy percent is not a rounding error. Seventy percent is the entire signal to noise problem, and it is the difference between a security team that can hold an SLA and a security team that is a running joke in the engineering all hands.
Three things vendors all call reachability
They do not mean the same thing. Know which one you are being sold.
Manifest level. Is the package imported anywhere at all, or is it just squatting in the lockfile. Cheap, fast, works in any language, and it will clear out your dev dependencies and your orphans. It is also a low bar, because "we import the library" is true of almost everything you care about. Expect twenty to thirty percent off the pile.
Function level, static. The real thing. Build the call graph, check the vulnerable symbol, not just the package. This is where the value is and it is also where the engineering gets hard, because it needs per language analysis, it needs a vulnerability database annotated with vulnerable symbols, and it fights badly with reflection, dynamic imports and dependency injection. Go and Java behave. Python and JavaScript do not. Expect sixty to eighty percent off the pile, which is the number that changes your life.
Runtime. Instrument the app, watch what actually loads and executes. Highest fidelity, no static guesswork, and it accounts for configuration for free. The catch is that it tells you what ran, not what could run. The vulnerable admin endpoint nobody hit this week looks unreachable. It is not unreachable. It is unvisited. Use it to confirm a finding, never to dismiss one.
The question that separates a real vendor from a reskinned scanner
Ignore the marketing number. Everybody claims ninety percent noise reduction. Ask this instead, and ask it in the demo, out loud.
the test
What percentage of the CVEs in your database have symbol level annotations, and what do you do when you cannot statically resolve a call.
The first half matters because reachability is only possible for CVEs where somebody bothered to identify the vulnerable function. If the honest answer is forty percent, then sixty percent of your findings fall back to package level analysis no matter how good the call graph engine is. That is the dirty secret of this entire product category and most buyers never ask.
The second half matters more. A tool that marks a genuinely reachable vulnerability as unreachable is far more dangerous than the noise you started with, because now you are confidently ignoring a real bug and you have a report that says you were right to. The correct answer is some version of "we treat unresolvable dynamic dispatch as potentially reachable and we fail open." If a vendor tells you they statically resolve all dynamic dispatch, they are either lying to you or they have not thought about it. Both are disqualifying and you should end the call.
What I would actually do
Measure in the dark for a month. Turn reachability on and change nothing about how findings are handled. You are building a case, not a process. What you want at the end is one sentence you can say to a VP: we have six hundred and twelve high findings, one hundred and seventy eight are reachable, and we have been asking engineering to spend thirty weeks on four hundred and thirty four issues that cannot be exploited. Nobody argues with that sentence.
Split the queue, do not shrink it. Reachable findings get an SLA and a ticket. Everything else goes to a dashboard with no SLA and no ticket, and it is re evaluated on every single build. Say the words carefully and say them often: not currently reachable is not the same as not vulnerable. The code is still in the image. The day somebody adds a call to that function, your next build must promote it into the real queue automatically. If it does not promote automatically then you have not built noise reduction, you have built a way to lose track of vulnerabilities, and one day you will explain that to an auditor.
Demand the path, not the verdict. If the tool cannot show handler to service to library to CVE, engineers will not believe it, and a finding a developer does not believe is a finding that does not get fixed. Explainability is not a nice to have here. It is the whole adoption mechanism.
Put it on merge, not on the pull request. Full call graph analysis is expensive. Add eight minutes to every PR build and engineers will route around it and you are back where you started. Fast package check on the PR, full analysis nightly.
Then go after the roots. With the noise gone the patterns finally become visible. One ancient internal SDK generating half your reachable findings. Three services on an end of life runtime. Those are platform fixes. Upgrade once, close a hundred findings, permanently. You could never see them underneath four thousand one hundred rows and that, not the smaller number, is the actual prize.
The part that is still your job
Reachable is not exploitable. A reachable function is not necessarily reachable with attacker controlled input. Reachability takes you from four thousand to two hundred. It does not take you from two hundred to the twelve a competent attacker could really use. That last mile is human judgment and there is no product for it, which is presumably why nobody sells one.
And it does not save you everywhere. Compiled binaries, vendored code, base image OS packages that live entirely outside the call graph. Keep package level scanning as the floor. Reachability is a prioritisation layer sitting on top of it, not a replacement for it.
But it is the difference between a queue engineering can clear and a queue engineering has already given up on.
You do not have a vulnerability problem. You have a prioritisation problem wearing a vulnerability problem's clothes, and you have been paying a licence fee to keep the costume on.
Measure it before you argue about it. The seventy percent will make the argument for you.
// Elusive Thoughts // the number was never the point // securityhorror.blogspot.com
SOURCES // Figures drawn from a roughly 140 service estate. Cross checked against published SCA noise studies. Treat as direction, not decimals. Analysis and commentary are my own.
#AppSec #SCA #SupplyChain #VulnerabilityManagement #DevSecOps #Reachability
