Nullgap.dev Docs

Connect your code

Azure DevOpsNew

Connect an Azure DevOps organisation with a personal access token and service hooks, to review pull requests in Azure Repos.

An Azure DevOps organisation is connected with a personal access token you create and service hooks you add in each project. Pull requests in Azure Repos are then reviewed: a status on the pull request, a summary comment, inline comments and fix pull requests.

Only Azure DevOps Services (dev.azure.com) can be connected. Azure DevOps Server cannot; use an archive upload for code that lives there.

#What you need first

  • To be an owner of the Nullgap organisation, signed in to the console. If your account has multi-factor sign-in, you are asked for a fresh code. An API key cannot connect a code host.
  • In Azure DevOps, permission to create a personal access token and to create service hooks in each project you want reviewed.

#Step 1: create the access token

Create a personal access token for the organisation with these scopes:

ScopeAccessWhy
CodeRead and writeFetch the code, read pull requests, and push a fix branch.
CodeStatusSet the review's status on a pull request or commit.
Pull Request ThreadsRead and writePost the summary and inline comments and reply in their threads.

A personal access token acts as the person who created it, so comments and fix pull requests appear under that account. Nullgap ignores comments written by that account, so that it never answers itself. Use an account kept for this purpose, not the account of someone who will comment @nullgap review or reply to findings.

#Step 2: connect the organisation

Open Integrations. Under Source code, in the Azure DevOps row, press Connect and fill in:

FieldWhat to enter
OrganisationThe organisation name as it appears in its address: dev.azure.com/<organisation>.
Access tokenThe token from step 1. It is stored encrypted and never shown again.

Before anything is stored, Nullgap checks that the token works for that organisation. If not, you get code_host_rejected. An Azure DevOps organisation can be connected to one Nullgap organisation; a second gets already_connected.

#Step 3: add the service hooks

When the organisation is connected, the console shows a Webhook URL and a secret labelled Password (basic authentication).

In each project whose pull requests should be reviewed:

  1. Open Project settings, Service hooks, Create subscription, Web Hooks.
  2. Create four subscriptions, one per event: Pull request created, Pull request updated, Pull request merge attempted, Pull request commented on.
  3. For each, paste the URL into URL. Under basic authentication, enter any user name and paste the secret as the password.
  4. Leave Resource details to send on All, then finish.

A delivery with the wrong password, or to an address that does not exist, is answered 401.

#What you will see

The organisation appears in Integrations, and its repositories appear under Repositories as <project>/<repository> with review switched on. Disabled repositories are left out. Repositories and projects added later appear when you press Sync on Repositories; a new project also needs its own service hooks.

How Azure DevOps differs from GitHub:

  • The result is a pull request status named Nullgap Security Review, not a check with three outcomes. A failing review sets it to failed. A neutral result is written as not applicable, which is not shown as a pass. See checks and gating.
  • Suggested changes are shown as a patch. A verified fix for the lines under review is posted as a comment containing the change as a diff, together with a pull request to merge it from. See suggested changes.
  • Commands are for members of your Nullgap organisation. Nullgap cannot ask Azure DevOps whether a person has write access to a repository with the scopes above. A reply command (!fp, !accept, !valid, !reopen, @nullgap fix) or an @nullgap review comment is therefore acted on only when the commenter's Azure DevOps sign-in address is the e-mail address of a member of your Nullgap organisation. A collaborator with write access who is not a Nullgap member cannot use them; invite that person, or triage in the console.
  • People are identified by their sign-in name, which is what appears as the author of a review and on a review seat.

#Rotating the secret

In Integrations, press Rotate webhook secret on the organisation. A new secret is shown once, with the same URL. The old one stops working at that moment, so deliveries are refused until you have entered the new password in every service hook subscription you created.

Connecting the same organisation again, for example with a new access token, also issues a new secret.

#What can go wrong

Message or codeCauseWhat to do
code_host_rejectedThe token is wrong, expired, for another organisation, or lacks one of the three scopes.Create a new token with Code (read and write), Code (status) and Pull Request Threads (read and write).
already_connectedThe organisation is linked to another Nullgap organisation.Disconnect it there first.
step_up_requiredYour second-factor check is older than fifteen minutes.Confirm with your authenticator code, then try again.
Pull requests in one project are not reviewedThat project has no service hooks, or their password does not match.Create the four subscriptions in that project and check their history.
New commits are not reviewedThe Pull request updated subscription is missing.Add it.
Reviews and pentests fail after a whileThe personal access token expired.Connect again with a new token and update the password in each subscription.

#Removing

Press Disconnect on the organisation. The stored token and secret are deleted, reviews stop and the repositories leave the list. Past reviews and findings are kept. Delete the service hook subscriptions and revoke the token in Azure DevOps.

Nullgap documentation