Nullgap.dev Docs

Security and privacy

AI processing

What is sent to Nullgap AI and what is not, how identifiers are masked, how zero data retention is checked on every run, and where you can see the result.

Reviews, pentests and fixes are carried out by Nullgap AI. To do that work, parts of your code are sent for AI processing. This page says what is sent, what is held back, what Nullgap checks about how the AI side treats it, and where you can see that check for each run.

#What is sent

During a run, Nullgap AI reads the code the run needs:

  • for a pull request review, the changed files and the code around them that the change interacts with;
  • for a pentest, the code in the scope you chose;
  • for a fix, the finding and the code around it;
  • your threat model, project rules and custom checks for the application, so that the work reflects what you told Nullgap about your system.

Findings and explanations are produced from that processing. The prompts and the raw responses are not stored by Nullgap: not in the database, and not in the run log. See how your code is handled.

#What is not sent

Before anything is sent for AI processing, two things happen on Nullgap's side:

  • Credential files are not read. .env files (templates such as .env.example excepted), private key and keystore files, SSH private keys, .npmrc, .netrc and files named credentials are refused when the AI asks to read them and skipped when code is searched.
  • Credential-looking values are masked in every message that is sent. A masked value keeps its first four characters and its length. The same masking is applied to the excerpts stored with findings.

The masking is pattern-based: it covers well-known key formats, private key blocks and assignments whose name says they hold a credential. It cannot recognise an arbitrary secret with no telltale shape. The details and the limit are under credentials in your code.

Your stored integration credentials, code-host tokens and API keys are not part of any run's input.

#Identifier masking

Source code often contains personal data: names and e-mail addresses in fixtures, phone numbers in tests, card or ID numbers in sample records.

Before an AI model sees a request, the AI processing layer replaces such identifiers with realistic synthetic values, and puts the real values back in the answer. The model works on the substitute; your findings read as they should.

For every request, the AI processing layer reports how many identifiers it replaced. Nullgap adds these up per run and shows the total as identifiersMasked. A run whose code held no identifiers shows 0, which the console words as "No identifiers needed masking before AI processing".

This is a different mechanism from credential masking above. Credential masking happens inside Nullgap before a request leaves; identifier masking happens in the AI processing layer before a model sees it.

#Zero data retention

Zero data retention means that the AI side keeps nothing from a request once it has answered it: no prompt, no response.

Nullgap does not take this on trust per run. It checks:

  1. Every answer from AI processing carries a confirmation of whether that request was handled with zero data retention.
  2. Nullgap counts the AI requests a run made and how many of them were confirmed.
  3. Where the service requires zero retention, a request that comes back without the confirmation stops the run at once. No further data is sent.

#What a stopped run looks like

  • The review or pentest ends as failed, with the message "Zero data retention could not be confirmed for this request, so the run was stopped. No further data will be sent until it is confirmed again."
  • It is not retried automatically. Sending more data to find out whether the guarantee is back is exactly what must not happen.
  • Your audit log records security.zero_retention_unconfirmed with the actor nullgap and the run it concerns.

If you see this, start the run again later. If it repeats, contact Nullgap with the run id: see getting help.

#The per-run result

Every review and pentest that made at least one AI request carries a privacy object:

FieldMeaning
privacy.zeroRetentiontrue only if every AI request in the run was confirmed as zero retention. One unconfirmed request makes it false
privacy.identifiersMaskedHow many identifiers were replaced with synthetic values across the run

privacy is null for a run that made no AI request, such as a skipped review. It is also null while a run is still in progress: the figures are set when the run finishes.

Where you see it:

In the console, on a review's or a pentest's page, in the Privacy panel: "Zero data retention: confirmed", or a warning that it was not confirmed for every AI call in the run, and the number of identifiers masked.

In an evidence bundle, which records the privacy of the run that produced the finding. See evidence bundles.

In the API, as privacy on the review or pentest:

curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
  "$NULLGAP_URL/api/orgs/$NULLGAP_ORG/reviews?limit=1"

Abridged:

{
  "items": [
    {
      "id": "rev_3m8k2p",
      "repoFullName": "acme/shop",
      "status": "completed",
      "privacy": { "zeroRetention": true, "identifiersMasked": 14 }
    }
  ],
  "total": 214
}

#The organisation-level figure

zeroRetentionCoverage is the share of your organisation's AI requests in the last 30 days that were confirmed as zero retention, as a whole percentage rounded down. It is null when no run in that period made an AI request.

curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
  "$NULLGAP_URL/api/orgs/$NULLGAP_ORG/compliance"

Abridged:

{
  "zeroRetentionCoverage": 100,
  "guarantees": [
    { "id": "zero-retention", "title": "AI prompts and responses are never stored", "active": true },
    { "id": "identifier-masking", "title": "Identifiers are replaced before the AI sees them", "active": true }
  ]
}

In the console the same figure is on Settings, Data protection, under How your code is protected: "100% of AI calls in the last 30 days were confirmed zero-retention".

How to read guarantees:

Entryactive is true when
zero-retentionCoverage is exactly 100. With no runs in the last 30 days, it reflects whether the service requires zero retention
identifier-maskingAt least one identifier was masked in the last 30 days, or masking is declared on for the service
secrets, sourceAlways: credential masking and not keeping your source are part of the product and cannot be switched off

Anything below 100 means at least one AI request in the period was not confirmed. Find the run by looking for privacy.zeroRetention being false, or for security.zero_retention_unconfirmed in the audit log.

#The independent reviewer

A finding is not reported on the word of the AI pass that found it.

  • A first pass looks for candidate findings.
  • Each candidate then goes to a separate, independent pass whose job is to refute it. Only what could not be refuted is reported. No finder grades its own work.
  • The service can run the refuting pass on a different AI model from the one that found the candidate, so that the two do not share the same blind spots.
  • When a finding is proven with a probe, the probe that reproduced it is audited by a pass kept apart from the one that wrote it.

This is about the quality of findings rather than privacy, but it affects processing: the independent passes read the same code, under the same masking and the same zero-retention check, and their requests are counted in the same per-run figures.

How this maps to the assurance levels is explained under core ideas.

#Learning from your usage

Nullgap does not learn from one organisation's usage to change what it does for another, and there is no setting that switches such learning on. Your code is not used to train models.

An earlier version of the console had a switch called Help improve Nullgap (learnFromUsage in the settings API). Nothing ever acted on it, so it was removed: it is no longer shown, no longer returned by the API, and ignored if a client still sends it.

What Nullgap remembers inside your own organisation in order to work for you is a different thing, such as the pattern of a finding you dismissed. That memory is yours, is described under dismissal memory, and is removed by retention and erasure.

#Naming

The console, the API, logs and reports refer to the AI as Nullgap AI. Which model answered a request, and what processing a run consumed, are not part of what the product returns.

Nullgap documentation