Nullgap.dev Docs

Developers

The nullgap CLI

Scan a checkout on your own machine, or sign in with an API key and start pentests, triage findings, start fixes and gate a pipeline on your Nullgap server.

nullgap is a command-line program with two kinds of command:

  • Local commands run on a checkout of your code, on your machine: they scan source code, audit dependencies against published advisories, and can prove and patch what they find. They use a model endpoint that you provide, do not sign in to the Nullgap console and do not send their results there; runs are saved in a .nullgap folder inside the directory you run them from.
  • Hosted commands run on your Nullgap server with an API key of your organisation: start and follow pentests, read and triage findings, start fixes, download reports, review a pull request, run skills, and gate a pipeline. The work happens under your organisation's plan, limits, permissions and audit log, and what the commands show are the same records the console shows.

#Local or hosted: the rule

The command's name decides, and nothing else. Whether a key is set in the environment, or whether you are signed in, never changes where a command runs.

Runs on your machineRuns on your Nullgap server
scan, repair, deps, runs, scope, doctorlogin, logout, whoami, repos, pentest, findings, fix, report, review, gate
skills pullskills run, skills run-status, skills list --hosted
  • A hosted command must come first on the line, before any flag.
  • --local on a local command and --hosted on a hosted command are accepted and change nothing, so a script can state what it expects. --hosted on a local command, or --local on a hosted one, is an error that names the matching command (scan --hosted points to pentest start, fix --local to repair).
  • skills list, skills enable and skills disable keep their own rule, described under skills.
  • Hosted commands never read a .env file. Local commands do, as before.

The two modes use different keys. Local commands read NULLGAP_API_KEY, the key of your model endpoint. Hosted commands read NULLGAP_ORG_API_KEY, a Nullgap API key (those start with ng_). Neither is ever sent to the other's address.

For the REST API itself see the API overview, and for coding agents the MCP server. Pipeline examples are on CI examples.

#Local commands

CommandWhat it does
nullgap scan [path]Scans source code under path (the current directory when omitted).
nullgap repair [path]Takes findings from a saved code scan, proves each with a probe, writes a patch and verifies it on a fresh checkout.
nullgap deps scan [path]Audits npm, pnpm, yarn, PyPI and Cargo dependencies against the public advisory database.
nullgap deps fix [path]Writes the smallest upgrades that clear the advisories.
nullgap runsLists saved runs, newest first.
nullgap scope list, nullgap scope add <host>Shows or extends the list of hosts you are authorised to test live.
nullgap skills list, enable <name>, disable <name>Shows your organisation's skill catalogue, and switches a skill on or off.
nullgap skills pull [name ...]Copies Nullgap's client skills for a coding agent into the project.
nullgap doctorChecks the model endpoint, the tools on the machine and the sandbox.

nullgap --help prints the usage and nullgap --version the version. --quiet (-q) silences progress messages and --verbose adds detail; progress goes to standard error, results to standard output. Set NO_COLOR to switch colours off.

#The model endpoint

Verifying a finding, the deeper review and repair need a model endpoint that you provide and configure. The CLI reads four environment variables:

VariableMeaning
NULLGAP_BASE_URLThe address of the model endpoint.
NULLGAP_API_KEYThe key for that endpoint. Not needed when the address is on localhost.
NULLGAP_MODELThe model to use.
NULLGAP_PROVIDERThe name of a built-in preset that supplies an address and a default model, as an alternative to setting them yourself. nullgap --help lists the names.

The same can be given per command with --base-url, --model and --provider, which win over the environment. Variables may also be put in a .env file in the directory you run from; values already in the environment are not overridden.

export NULLGAP_BASE_URL="https://models.internal.acme.example/v1"
export NULLGAP_API_KEY="..."        # the key of your model endpoint
export NULLGAP_MODEL="your-model-id"
nullgap doctor

The code being analysed is sent to the endpoint you configure. Choose one you are allowed to send that code to.

#nullgap doctor

nullgap doctor shows what is configured and whether it works. Each line starts with ok or miss:

  • which model endpoint and model are configured, or why none is;
  • whether the endpoint answers, and whether the model supports tool calling, which the analysis needs;
  • git (needed for repair and --changed) and semgrep (an optional extra engine; the built-in rules run without it);
  • the sandbox that probes and tests run in: a container when a container engine is available;
  • the Node.js version.

Run it first when a command behaves unexpectedly.

#Scan source code

nullgap scan .

The scan collects candidates with its rule engines and then has the model check each one. Findings are printed most severe first, each with its status in square brackets, its location, the rule and its id:

CRITICAL Command injection: shell command built from dynamic input  [candidate]
         src/app.js:4  builtin/js-command-injection · f_99c68249da1f
FlagEffect
--deepAdds a model-driven review for logic and authorisation bugs that rules do not catch.
--no-verifySkips the model check and runs the engines only. Needs no model endpoint.
--changed <base-ref>Scans only the files changed since a git ref, for example --changed origin/main on a pull request. Needs git and enough history to reach the ref.

Statuses you will see: candidate (raised by an engine, not checked), verified (the check agreed it is real), proven (a probe reproduced it) and fixed (a patch passed verification). Candidates the check rejects as false positives are counted but not listed.

If no model endpoint is configured and you did not pass --no-verify, the scan warns, carries on with the engines only, and reports everything as candidate.

#Repair

nullgap repair --test "npm test"

repair needs a model endpoint. It takes the latest saved code scan and, for the three most severe findings, tries to reproduce the flaw with a probe, writes a patch, and checks on a fresh checkout that the probe no longer reproduces and, when you give a test command, that the tests pass. It prints each check by name as passed, failed or not run, and the diff.

FlagEffect
--run <id>Use this saved run instead of the latest code scan. nullgap runs lists the ids.
--finding <id>Repair this one finding.
--allRepair every active finding instead of the top three.
--test "<cmd>"A test command that must pass on the patched checkout.
--applyWrite verified patches to the working tree. Without it nothing in your tree changes.

--apply never commits, and a patch that was not verified is never applied. Review the diff like any other change.

Probes, tests and commands run in a container when a container engine is available. NULLGAP_SANDBOX selects the mode (auto, container or host); nullgap doctor shows which one is in effect.

#Dependencies

nullgap deps scan . --fail-on high
nullgap deps fix . --dry-run

deps scan reads the lockfiles and manifests it finds (npm, pnpm, yarn; pip requirements, Poetry, uv and Pipenv; Cargo) and looks every pinned version up in the public advisory database. It needs no model endpoint. It needs network access to the advisory database.

deps fix writes the smallest upgrades that clear the advisories into the manifests and prints each change as an upgrade, an override or one you have to do by hand, with the advisories it clears. --dry-run prints the changes and writes nothing. Afterwards run your package manager's install to refresh the lockfile.

#Output files and failing a build

These flags apply to scan and deps scan:

FlagEffect
--json <file>Writes the full run report as JSON.
--sarif <file>Writes SARIF 2.1.0, for code-scanning views that read it.
--md <file>Writes a Markdown summary.
--fail-on <severity>Exits with code 2 when any finding at or above the severity exists: critical, high, medium, low or info. Findings rejected as false positives do not count.

Any combination can be given; missing folders are created.

nullgap scan . --changed origin/main \
  --json nullgap-results.json --sarif nullgap-results.sarif --md nullgap-results.md \
  --fail-on high

#Exit codes

CodeMeaning
0The command finished. With --fail-on, nothing reached the threshold.
1The command could not run: a bad flag or path, a missing model endpoint where one is required, or another error. The message is on standard error, prefixed nullgap:.
2--fail-on was given and a finding reached the threshold. Report files are written before the command exits.

Without --fail-on, findings do not change the exit code.

#Saved runs and project settings

Everything the CLI keeps is in .nullgap under the directory you run from:

PathContent
.nullgap/runs/One JSON report per run. nullgap runs lists them with their id, kind, counts by severity and start time.
.nullgap/scope.jsonThe hosts added with nullgap scope add.
.nullgap/config.jsonOptional project settings, written by you.

.nullgap/config.json accepts:

{
  "failOn": "high",
  "testCommand": "npm test",
  "ignore": ["vendor", "*.min.js", "**/legacy/**"]
}
  • failOn is the default for --fail-on.
  • testCommand is the default for repair --test.
  • ignore lists paths or globs, relative to the scanned directory, that scan skips.

Run reports contain code excerpts. Add .nullgap/ to your .gitignore unless you want them in the repository.

#Skills

nullgap skills list
nullgap skills disable secret-detection
nullgap skills enable secret-detection
nullgap skills pull

list, enable and disable work on the skills of your organisation on a Nullgap server. They read three settings:

VariableValue
NULLGAP_URLThe address of your Nullgap console.
NULLGAP_ORGYour organisation's slug.
NULLGAP_ORG_API_KEYAn API key of that organisation. read is enough to list; switching a skill needs admin.

The key is NULLGAP_ORG_API_KEY, not NULLGAP_API_KEY: in the CLI that second name is the key of the model endpoint, and it is never sent to a Nullgap server. Without the three settings, nullgap skills list prints the built-in catalogue shipped with the program.

These three commands do not use the login stored by nullgap login; they read the three variables only. nullgap skills list --hosted lists the organisation's skills with the stored login or the variables of hosted commands, and fails when you are not signed in. Running a skill is a hosted command: see skills on the server.

pull copies the product skills of the client skill pack into the project, all of them or the ones you name. --path <dir> says where to write them. Each skill is checked against the Agent Skills format before anything is written, and nothing that is pulled is run. NULLGAP_SKILLS_SOURCE sets where they are read from: a folder or an https:// address.

#The scope list

nullgap scope add staging.shop.acme.example
nullgap scope list

The scope list names the hosts you are authorised to test live. scope add takes a host name, optionally with a port, or a wildcard such as *.staging.acme.example. Loopback addresses are always in scope. Only list hosts you own or have permission to test.

This list is local to the CLI. It is separate from the scope list of your organisation in the console, described in live testing.

#Hosted commands

CommandWhat it doesKey scope
nullgap login, logout, whoamiStores a key for your user, forgets it, shows the organisation, key name and scope.any
nullgap reposLists the repositories connected to the organisation.read
nullgap pentest start <repo>Shows the size and the price, asks, and starts a pentest.admin
nullgap pentest start --upload <dir>Uploads a local folder and starts a pentest of it.admin
nullgap pentest status <id>, logs <id>, list, cancel <id>Follows a pentest, reads its log, lists pentests, stops one.read (admin to cancel)
nullgap pentest resume <id>Shows how far a cancelled or failed pentest got and the price, asks, and continues it from where it stopped.admin
nullgap findings list, show <id>Lists findings with filters, shows one in full.read
nullgap findings triage <id>Sets a finding's state with a reason.write
nullgap fix start <finding-id>, fix status <run-id>Starts a fix and shows its outcome and pull request.write, read
nullgap report list, report downloadLists a pentest's reports and saves one as an HTML file.read
nullgap review start <repo> <pr-number>, review status <id>Requests a review of a pull request and shows its result.write, read
nullgap skills run <skill>, skills run-status <id>Runs a skill on demand and shows its output.write, read
nullgap gate --repo <repo>Exits with code 2 when open findings at or above a severity exist.read

<repo> is a repository's full name as nullgap repos shows it (acme/shop, in any case) or its id.

Every hosted command takes:

FlagEffect
--jsonPrints the result as JSON on standard output: the record the API returns, unchanged. Progress and warnings stay on standard error.
--server <url>The address of the Nullgap console to talk to, for this command.
--quiet (-q)No progress messages.

Note that --json is a switch here. On the local scan it takes a file name.

#Sign in

Create an API key in the console under Settings, Access, with the smallest scope that covers what you will do. Then:

nullgap login --server https://nullgap.acme.example

You are asked for the key; what you type is not shown. The key is checked with the server before it is stored, and the answer names the organisation, the key and its scope:

Signed in to Acme (acme) at nullgap.acme.example.
Key "Laptop" (…x7Qd), scope read.
Stored in /home/dana/.config/nullgap/credentials.json, readable by your user only.

To pass the key without a terminal, pipe it in. A key is never accepted as an argument: it would stay in your shell history and show in the process list.

printf '%s' "$KEY" | nullgap login --server https://nullgap.acme.example --key-stdin
WhereWhat
~/.config/nullgap/credentials.jsonThe stored key and server address, with file mode 0600. XDG_CONFIG_HOME is honoured; on Windows the folder is %APPDATA%\nullgap. Set NULLGAP_CONFIG_DIR to use another folder.
NULLGAP_ORG_API_KEYA key from the environment, for pipelines. It wins over the stored login.
NULLGAP_URLThe server address that goes with it.

Rules that keep the key where it belongs:

  • A key is only printed masked, as its last four characters.
  • A stored file that other users of the machine can read is refused. Run chmod 600 on it or sign in again.
  • A key is sent only to the server it was given for. A key from NULLGAP_ORG_API_KEY needs NULLGAP_URL or --server; it is never sent to the address of a stored login. A stored key is not sent to a different server because of --server or NULLGAP_URL: sign in to that server instead.
  • The address must be https://. http:// is accepted for localhost only.
  • A redirect from the server is refused, not followed.

nullgap whoami shows what is in use and where it was read from. nullgap logout deletes the stored file; the key itself keeps working until an owner revokes it in the console.

#Waiting for a run

Commands that start something (pentest start, fix start, review start, skills run) return as soon as the run is queued and print its id. Add --wait to follow it to the end; the status commands take --wait too. gate is the exception: it waits unless you pass --no-wait.

FlagEffect
--wait, --no-waitFollow the run to its end, or do not.
--timeout <minutes>How long to wait. Default: 360 for a pentest, 60 for a review, a fix and a skill run, 30 for gate.
--poll <seconds>How often to ask. Default 10.
--followWith pentest start --wait, pentest status --wait and pentest logs: print the run's log lines as they arrive.

When the time runs out the command exits with code 6 and the run carries on at the server.

#Pentests

nullgap pentest start acme/shop --ref v1.4.0

The command measures the code, prints the size and the price, and asks before it starts anything. Without a terminal it never starts a paid run on its own: pass one of

FlagMeaning
--yesAccept the price that is shown.
--max-price <amount>Accept any price up to this amount, in US dollars. Above it the command exits with code 4 and nothing starts.

Either way the server is given the accepted amount as a ceiling, and refuses to start above it, so a run cannot be charged more than what was agreed to. Other flags: --ref <branch, tag or commit> (the default branch when omitted), --path <folder> (repeatable) to limit the scope, --context <text> or --context-file <file> with notes for the testers.

A start is safe to repeat. The command sends an idempotency key with it, so when the answer is lost on the network and the command tries again, the server returns the pentest the first attempt started. To make re-running a whole pipeline job safe as well, give the key yourself: --idempotency-key release-1.4.0 starts one pentest however often the job runs within 24 hours.

A pentest started here reads the code only. Live testing is set up in the console or through the API: see live testing.

pentest status <id> prints the state, the open findings by severity and the link to the console. pentest logs <id> prints the log; a failed pentest's log says why it failed. pentest cancel <id> stops a queued or running pentest, which is then refunded.

pentest resume <id> continues a cancelled or failed pentest from where it stopped, for as long as its progress is kept (see stopping and resuming a pentest). It prints how many files had been examined, until when the pentest can be resumed, and the price: the pentest was refunded when it stopped, so resuming charges it again. It is confirmed the way a start is: answer the question at a terminal, or pass --yes to accept the price shown or --max-price <usd> to accept any price up to a limit. In a pipeline without either, nothing is resumed and nothing is charged. --wait follows the pentest to its end.

#Uploading a local folder

nullgap pentest start --upload . --name "shop snapshot"

This packs the folder, shows how many files and how much data would be sent and what is left out, asks, uploads it as an archive, and then prices and starts the pentest as above. In a pipeline --yes confirms both the upload and the price.

What is never sent, whatever an ignore file says:

  • version-control folders (.git and the like) and the .nullgap folder;
  • .env files in every spelling (.env, .env.local, prod.env, .envrc);
  • private keys and certificate stores (*.pem, *.key, *.p12, *.pfx, *.jks, *.keystore, id_rsa and its relatives);
  • credential files of common tools (.npmrc, .netrc, .pypirc, .aws, .ssh, Terraform state and variable files, credentials.json, service-account files).

Also left out: node_modules folders, links (a link could point outside the folder), and everything your .gitignore files exclude, in the root and in sub-folders. A .nullgapignore file with the same syntax excludes more, and is read after .gitignore.

The limits are the server's: 200 MB compressed, 1 GB and 100,000 files unpacked. The name-based rules above cannot recognise a secret written into an ordinary source file; look at what you upload.

#Findings

nullgap findings list --repo acme/shop --severity high
nullgap findings show fnd_8f3a2c9d1b4e
nullgap findings triage fnd_8f3a2c9d1b4e --state false_positive --reason "The id is cast to an integer before the query."

findings list shows open findings, most severe first. Filters: --repo, --state <open|patched|accepted_risk|false_positive|all>, --severity, --source <review|pentest>, --priority, --search <text>, --proven-only (only findings a probe reproduced) and --limit (50 by default). Each finding is printed with its state and how far it was checked: proven, verified or unverified.

findings triage sets the state. The rules are the console's: which states need a reason, and what an accepted risk needs, are described under triage; --review-at <date> sets the date an accepted risk is looked at again. The change is recorded in the audit log under the key's name.

#Fixes

nullgap fix start fnd_8f3a2c9d1b4e --wait

--scope class fixes every open place of the same flaw with one change, and --as pull_request always opens a pull request. While a fix for the finding is queued or running, starting another returns that run.

The outcome is stated in words, and a fix that was not proven is never called fixed:

OutcomeWhat the command says
fixed, with verified: trueFixed and proven: the flaw was reproduced and no longer reproduces with the change. The pull request link follows.
unverified, with verified: falseProposed, not proven: the flaw could not be reproduced, so the change was not checked against it. It was opened as a draft pull request for a person to review.
no_fix_needed, false_positiveNo change was made, and why.
failedNo fix was made, with the detail.

With --wait the exit code is 5 when the run failed. Add --require-proven to get 5 for a fix that was proposed without proof as well. In --json output the field to read is verified. See verified fixes.

#Reports

nullgap report list --pentest pen_4b1d9e7a2c60
nullgap report download --pentest pen_4b1d9e7a2c60 --kind executive --out report.html

A report is a self-contained HTML page, the one the console shows; print it to PDF from a browser. report download <report-id> takes a report's id directly, --kind is full, executive or attestation, and --out - writes to standard output.

#Reviews

nullgap review start acme/shop 128 --wait

Requests a review of the pull request's current commit. A commit that already has a review returns that review. Reviews of connected repositories start on their own when a pull request is opened or updated; this command is for asking again or for repositories where that is switched off.

#Skills on the server

nullgap skills list --hosted
nullgap skills run threat-modeling --repo acme/shop --text "New refund endpoint." --wait

skills run takes what the skill works on: --repo with --ref, --base-ref (to compare two versions) or --pr <number>, or --finding <id>, and --text or --text-file for skills that take text. Which of these a skill needs is described in running a skill. A skill run is sent once and never repeated automatically; if the answer is lost, the command says so and how to check.

#The gate

nullgap gate --repo acme/shop --pr 128 --fail-on high --sarif nullgap.sarif --md summary.md

gate exits with code 2 when open findings at or above --fail-on exist (default high), and 0 when none do. Open means not fixed, not accepted as a risk and not marked a false positive: the gate reads the same records the console shows, so triage there changes its result.

FlagWhat is looked at
--pr <number>The open findings of the newest review of that pull request. With --commit <sha> the review must be of that commit.
--ref <branch, tag or commit>The open findings of the newest pentest of that version.
neitherEvery open finding of the repository.
FlagEffect
--fail-on <severity>critical, high, medium, low or info.
--proven-onlyCount only findings a probe reproduced. How many others were left out is printed.
--sarif <file>Writes the open findings as SARIF 2.1.0.
--md <file>Writes a short Markdown summary. When the file is the one named by GITHUB_STEP_SUMMARY it is added to, not replaced.
--strictA review that was skipped fails the gate.
--no-waitDo not wait for a review or pentest that is still running.

The files are written whether the gate passes or fails.

The gate does not pass on something that was not checked:

  • With --pr, when no review exists yet it waits for one, because the review of a fresh push may not have been created when the pipeline starts. If none appears in time the exit code is 6. With --no-wait it is 1.
  • A review or pentest that failed or was cancelled gives exit code 5.
  • A review that was skipped (a filter, a size limit) looked at nothing. The gate says so and passes, unless you pass --strict.
  • With --ref, a pentest of that version must exist: pentests are started on purpose, so the gate does not wait for one to be created.

#Exit codes of hosted commands

CodeMeaning
0Done. For gate: passed.
1Wrong usage, something that was not found, a request the server rejected, or an unexpected error.
2gate: open findings at or above the severity.
3Not signed in, the key was refused, or the key's scope does not allow this.
4Not started: the price or the upload was not confirmed, the price is above --max-price, or the run cannot be paid for.
5A run that was waited for ended failed, cancelled or skipped.
6A run that was waited for was still going when the time ran out.
7The server could not be reached, kept failing, or kept limiting the rate.

Messages go to standard error, prefixed nullgap:, and say what to do next. They never contain the key.

#Network: retries, proxies and certificates

Each request waits up to 30 seconds for an answer. When a request fails on the network or the server answers 502, 503 or 504, the command tries again up to three times, waiting about 1, 2 and 4 seconds. On 429 it waits as long as the server's Retry-After header says, up to a minute.

Only requests that are safe to send twice are repeated after a network failure: reads, setting a finding's state, starting a pentest (it carries an idempotency key), starting a fix and requesting a review (the server answers both with the run that already exists). Starting a skill run and uploading a folder are sent once.

Behind a proxy, set the usual variables:

VariableMeaning
HTTPS_PROXY, HTTP_PROXY, ALL_PROXYThe proxy for https, for http, or for both. An http:// or https:// proxy, with a user name and password in the address if it needs them.
NO_PROXYHosts reached directly, separated by commas. * switches the proxy off.
SSL_CERT_FILEA PEM file of certificate authorities to trust in addition to the built-in ones, for a network that re-signs TLS.
NODE_EXTRA_CA_CERTSThe same, read by Node.js itself.

Lower-case names are read too. To the proxy, an https connection is an opaque tunnel: it does not see the key.

The server states the version of its API with every answer. When it is newer than the installed nullgap understands, the command prints a warning once and carries on; update the CLI.

Nullgap documentation