Start here
Your first pentest
Choose a repository and version, see the estimate, start a pentest, and read the results, in the console or with the API.
A pentest assesses one repository at one version. This page takes you from nothing to a finished pentest with its reports and the included retest.
#What you need first
- To be an admin or owner of the organisation, or an API key with the
adminscope. Members can read pentests and ask for an estimate, but cannot start one. - Code Nullgap can read: a repository from a connected code host, a public repository by address, or an uploaded archive.
- A way to pay for the run. A pentest is charged per run, as agreed for your workspace, and you see the price before anything starts. A retest is included.
- For live testing only: a staging deployment, a throwaway test account on it, and its host on your organisation's scope list.
#In the console
Open Pentests and press New pentest. You can also begin on New run, the screen at /<org>/start that a workspace with repositories opens on: paste the address of a public repository or pick a connected one, and it opens the same wizard with the repository filled in. There are six steps. Nothing runs and nothing is charged until you confirm on the last one.
- Repository. Choose a connected repository. On the same step you can add a public repository by its address or upload a
.zipor.tar.gzof the code. - Version. A branch, tag or commit. A branch means its latest commit at the moment the pentest starts.
- Scope. Folders to test, one per line. Leave it empty to test the whole repository. This is the pentest's path scope.
- Context. Optionally choose an application, whose threat model and project rules are given to the testers, and add anything else they should know: architecture notes, areas you are worried about, things that are out of scope.
- Live testing. Optional; see below.
- Review & start. The console counts the lines of code in scope and shows the price. Press Start pentest, then Confirm and start.
#What you will see while it runs
The pentest page updates on its own and shows six phases in order.
| Phase in the console | phase in the API |
|---|---|
| Create environment | creating-environment |
| Build call graph | building-call-graph |
| Map attack surface | mapping-attack-surface |
| Trace data flow | tracing-data-flow |
| Test candidates | testing |
| Write reports | reporting |
The status moves from queued to running to completed. An admin can press Cancel while it is queued or running. A cancelled pentest is charged for the share of its work done before it stopped, never more than its price, and the rest is given back; a pentest that fails on Nullgap's side is refunded. See what a cancelled pentest costs.
#Results
When the status is completed, the pentest page shows:
- Findings, each with its severity and assurance level. They also appear under Findings with everything else. Start with reading a finding.
- Dependency advisories, listed apart from the findings. They are published advisories against package versions the code pins, not findings. See advisories and findings.
- Not raised because of earlier decisions, if any: findings this run produced and did not raise because of an earlier triage decision, each with the decision it rests on.
- Reports: a penetration test report, an executive summary and an attestation letter. See reports.
#The included retest
Once you have fixed things, open the finished pentest and press Request retest. Give the branch, tag or commit that contains your fixes. The retest runs the same scope again and produces a new pentest and a new set of reports.
A finding is marked fixed only when its flagged code is gone and the retest did not find it again. One retest is included per pentest, on the same branch or tag, free while the code grew by no more than a quarter; it is used up once it starts, unless it fails on Nullgap's side. More on retest.
#Optional: live testing
Live testing tries the findings from the source code against a running deployment. It is read-only unless you allow more.
- Put the host on the scope list first. A target whose host is not on the list is refused with
out_of_scope. Setting the list is described on live testing; the API call is shown below. - In step 5, switch on Also test a running deployment and enter the Staging address.
- Describe the Sign-in steps in plain words and add your test Identities: a label, an access level, and a username and password (or a token) for each, up to five. Passwords go in the password fields only. Two or more identities with different rights let Nullgap prove one user can reach another's data.
- Tick I own this system or have written authorisation to test it, then press Check sign-in. You cannot continue until the check passes.
Tests that change data stay off unless you switch on Allow tests that change data and confirm that the target is a disposable staging environment.
#The same with the API
Every call below needs a key with the admin scope, except the estimate and the reads, which any key can make.
#Find the repository
curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/repositories"
Take the id of the repository you want. The samples below use repo_3f9a1c2b7d4e.
#Ask for the estimate
Nothing starts and nothing is charged.
curl -s -X POST -H "Authorization: Bearer $NULLGAP_API_KEY" \
-H "Content-Type: application/json" \
-d '{"repoId": "repo_3f9a1c2b7d4e", "ref": "main", "paths": ["services/api"]}' \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/pentests/estimate"
The answer has linesOfCode and the price. The price is null when the codebase is large enough to need a custom quote; such a pentest cannot be started until that is arranged.
#Start it
curl -s -X POST -H "Authorization: Bearer $NULLGAP_API_KEY" \
-H "Content-Type: application/json" \
-d '{"repoId": "repo_3f9a1c2b7d4e", "ref": "main", "paths": ["services/api"], "context": "Payments service. Focus on refunds."}' \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/pentests"
Abridged answer:
{
"id": "pen_8c1d2e3f4a5b",
"repoFullName": "acme/shop",
"ref": "main",
"paths": ["services/api"],
"status": "queued",
"phase": null,
"linesOfCode": 48210,
"retestOf": null
}
ref defaults to the repository's default branch, paths to the whole repository. applicationId is optional and must belong to the same repository.
#Follow it and read the results
curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/pentests/pen_8c1d2e3f4a5b"
The answer carries status, phase, and when finished findings, advisories, supplyChain, suppressed and reports. Each report has an id and a kind (full, executive or attestation). To be told when it finishes without polling, use the pentest.completed webhook.
Fetch a report as HTML:
curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/reports/rpt_5a6b7c8d9e0f/html"
#Start the retest
curl -s -X POST -H "Authorization: Bearer $NULLGAP_API_KEY" \
-H "Content-Type: application/json" \
-d '{"ref": "fix/refund-checks"}' \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/pentests/pen_8c1d2e3f4a5b/retest"
ref is optional; without it the retest uses the version of the original. The answer is a new pentest with retestOf set. When it has finished it carries retest, with the counts fixed, stillOpen and added.
#Set the scope list
The scope list is sent as scopeHosts to the settings route. The list you send replaces the whole list. Read it back with GET /api/orgs/{org}/scope-hosts, or see it in the console under Settings, Targets.
curl -s -X PATCH -H "Authorization: Bearer $NULLGAP_API_KEY" \
-H "Content-Type: application/json" \
-d '{"scopeHosts": ["staging.shop.acme.example"]}' \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/settings"
#Check the sign-in, then start with live testing
curl -s -X POST -H "Authorization: Bearer $NULLGAP_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://staging.shop.acme.example", "signInSteps": "Post the username and password to /login as a form.", "username": "tester@acme.example", "password": "the-test-account-password"}' \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/pentests/verify-access"
The answer is { "ok": true, "detail": "..." } or ok: false with the reason. Nothing is stored.
To start with live testing, add dynamicTargets to the start call: a list of up to five targets, each with url, signInSteps and identities (up to five, each {label, level, username, password} or {label, level, bearerToken}; the older username, password, secondUsername and secondPassword still work). The sign-in of every identity is checked again before the pentest starts. Passwords and tokens are stored encrypted for that pentest and its retest and are never returned; the pentest shows only each identity's label and level.
#What can go wrong when starting
| Status and code | Meaning | What to do |
|---|---|---|
400 out_of_scope | The live-testing host is not on the scope list, or is a private address. | Add the host to the scope list. Private addresses cannot be tested. |
400 access_check_failed | A live-testing sign-in did not work. Nothing started and nothing was charged. | Fix the address, the steps or the account, and check again. |
400 confirmation_required | liveStateChanging was sent without disposableConfirmed. | Confirm that every target is disposable, or leave state-changing tests off. |
400 custom_quote | The codebase needs a custom quote. | Narrow paths, or contact Nullgap. See getting help. |
402 payment_required | There is nothing to pay for the run with. | See usage. |
402 usage_limit | The workspace has reached its monthly usage limit. | Contact Nullgap to raise it. |
403 org_suspended | The workspace is suspended. | Contact Nullgap. |
#Next
- Scoping a pentest and coverage: what a run does and does not examine.
- Schedules: run a pentest on a timetable.
- Verified fixes: have a fix prepared for a finding.
The full field lists are in the pentests reference and the reports reference.
Nullgap documentation