Nullgap.dev Docs

Dependencies and supply chain

Advisories and findings

Why a published advisory against a dependency is not a finding, what usage and usedIn tell you, and where advisories appear.

Nullgap keeps two things apart that many tools mix: findings, which are flaws shown to be real in your code, and dependency advisories, which are published notices that a package version you depend on has a known flaw. This page explains the difference and how to read an advisory.

#Why an advisory is not a finding

An advisory says that a package has a known flaw. It does not say that your application can be attacked through it. That depends on whether your code reaches the vulnerable part of the package, with input an attacker controls, and nothing in a pentest checks that.

A finding is different. It went through an independent review of your own code, and the serious ones were reproduced. See Proof.

So an advisory is never called verified or proven, and it is treated differently everywhere:

FindingDependency advisory
What it saysThis flaw is in your code and was checkedThis package version has a published flaw
Assurance levelProven, verified or unverifiedNone
Priority and exposure factsYesNo
Triage state, reasons, memory of dismissalsYesNo
Counted in a pentest's findings, its reports' summaries and the attestation letterYesNo
In Findings and in the findings exportsYesNo
Sends finding.createdYesNo
FixA verified fixA dependency fix

Mixing the two would bury the handful of flaws that were shown to be exploitable under hundreds of notices that may not matter, and would let a count of "critical findings" be inflated by a transitive package nobody calls.

#What does become a finding

Some dependency problems are findings, because the package itself is the problem, whatever your code does with it.

SituationIn a pentestOn a pull request
A version with a published vulnerabilityAdvisoryListed in the review's Supply chain view. Not a finding.
A malicious package: a malware advisory, or strong evidence in its published archiveFindingFinding
A suspicious package: signals a person should look atListed in the Supply chain tab. Not a finding.Finding, always unverified
A package your organisation's rules blockRules are not applied in a pentestFinding, and the check fails

How verdicts are reached is on Malicious and suspicious packages. The pull request side is on Supply chain checks.

#Where advisories appear

  • On a pentest. The Dependency advisories tab, shown when there are any, with the count. By API, advisories on GET /pentests/{id}, and advisoryCount on the pentest in lists.
  • In the Supply chain tab of the pentest: each vulnerable package is one row there too, with its advisories, next to malicious and suspicious packages. By API, supplyChain.
  • In the full report, as an appendix titled "Appendix: dependency advisories", which says in its first lines that they are not findings of the assessment. Up to 200 are printed.
  • In the pentest's log, as one line with the count.
curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
  "$NULLGAP_URL/api/orgs/$NULLGAP_ORG/pentests/pen_4c1d9e7a2b60"
{
  "id": "pen_4c1d9e7a2b60",
  "findingsCount": 3,
  "advisoryCount": 41,
  "advisories": [
    {
      "id": "GHSA-35jh-r3h4-6jhm",
      "cve": ["CVE-2021-23337"],
      "package": "lodash",
      "version": "4.17.20",
      "ecosystem": "npm",
      "fixedVersion": "4.17.21",
      "severity": "high",
      "summary": "Command Injection in lodash",
      "manifest": "package-lock.json",
      "direct": true,
      "usage": "imported",
      "usedIn": ["src/reports/template.js:3", "src/util/merge.js:1"]
    }
  ]
}

The sample is abridged. Up to 500 advisories are kept on a pentest, most severe first.

#Reading an advisory

FieldConsole columnMeaning
idAdvisoryThe advisory id, such as GHSA-… or PYSEC-…. cve lists its CVE ids; the console shows the first CVE when there is one.
package, version, ecosystemPackageThe package version your code pins.
severitySeverityThe advisory's own severity. It describes the flaw in the package, not the risk to your application.
summarySummaryThe advisory's one-line description.
fixedVersionFixed inThe first version without the flaw, when the advisory names one. null when none is published.
manifestPackageThe lockfile or requirements file that pins it.
directPackagetrue when your project declares the package, false when another package pulls it in (the console adds "indirect").
usage, usedInUsed by your codeSee below.

#usage and usedIn

usage answers one narrow question: does your own source import this package?

usageConsoleMeaning
importedImportedAn import of the package was found in your source. usedIn lists up to five places, as file:line.
not-importedNot importedThe package is a direct dependency, and none of the source files that were scanned import it.
unknownNot determinedIt could not be decided.

Read it for exactly what it is:

  • It is about imports, not about reach. imported means the package is named in an import statement. It does not mean the vulnerable function is called, or that an attacker's input gets there. An imported package can still be unaffected.
  • It is a text scan for import statements, not a call graph. It errs towards unknown.
  • not-imported is only ever said about a direct dependency. A package that nothing in your source imports may still be loaded by a tool, a plugin system or a configuration file.
  • unknown is the usual answer for a transitive package. Another package pulls it in, so your source would not import it directly. That says nothing about whether it runs.

unknown is also given when the import names of an ecosystem do not map to package names reliably, when no source files for that ecosystem were found, or when the scan reached its size limit before reading everything.

The list is sorted by severity, and within a severity the imported packages come first.

#Using it to decide

usage is a way to order the work, not a reason to ignore an advisory.

  1. Start with advisories of high severity whose package is imported. Open the places in usedIn and see whether the affected function is used.
  2. Upgrade where a fixedVersion exists. Fix on the row opens a pull request with the smallest manifest and lockfile change. See Dependency fixes.
  3. For a direct dependency that is not-imported, consider whether you need the package at all.
  4. If you find that an advisory is exploitable in your application, that is a flaw in your code's use of the package. The next pentest or review may report it as a finding of its own.

#Where the advisories come from

  • Versions are read from your lockfiles and requirements files: package-lock.json, npm-shrinkwrap.json, pnpm-lock.yaml, yarn.lock, poetry.lock, uv.lock, Pipfile.lock, requirements*.txt and Cargo.lock. That covers npm, PyPI and crates.io packages.
  • Advisories come from a public open-source advisory database, which gathers advisories from the ecosystems' own security databases. Withdrawn advisories are left out.
  • Nothing is installed and no package code is run.

In a pentest the dependency check reads the whole checkout, not only the folders in scope.

If the advisory database cannot be reached, the pentest still finishes. Its log then has a warning beginning "Dependency advisories unavailable", and advisories is empty. An empty list on such a pentest does not mean there are none.

#Checking a package without a repository

To ask about package versions directly, before they are installed, use the package check API. It answers with a verdict for each package and needs no repository.

Nullgap documentation