Nullgap.dev Docs

Skills

Custom checks or skills

The difference between a custom check and a skill, what each can and cannot do, and which one to reach for when you want Nullgap to look for something of your own.

There are two ways to tell Nullgap to look for something specific to your organisation: a custom check and a custom skill. They are different things with different reach.

  • A custom check is a short brief that adds one thing to look for in every review and pentest.
  • A skill is a whole packaged capability: when it applies, a playbook, the tools it may call and what it returns. It can also be run by itself on demand.

If you can say what you want in a sentence or two, write a custom check. If you need a method with steps, a tool, a document as the result, or something you can run on its own, write a skill.

#Side by side

Custom checkCustom skill
What you writeA name and a brief of 20 to 1,500 characters, in plain wordsA SKILL.md in the Agent Skills format: frontmatter and a playbook of up to 48,000 characters
Where it livesSettings, Custom checks tabSkills, Custom skills tab
How many20 per organisation40 per organisation
Runs in pull request reviewsYes, when it applies to a changed fileWhen it lists review and applies to the changed files
Runs in pentestsYes, on the most exposed files it applies toWhen it lists pentest and applies to the repository
Runs by itself, on demandNoYes, when it lists on-demand
When it appliesFile globs and an optional regular expression on the file's textFile patterns, technologies and languages
ToolsNone: it reads the codeThe tools it names, where they exist and your organisation permits them
What it returnsFindingsFindings, a document, or a triage verdict on one finding
Can be switched off for one repositoryNo: a check is on or off for the organisationYes: overrides for one repository
How its findings are markedfinding.source starts with llm-custom:, the rule id with custom/finding.source is llm-skill: followed by the skill's id
Who manages itAdmins and ownersAdmins and owners
PortableNo, it is a Nullgap settingYes, the format is open and used by coding agents

#What they share

Neither can change how a finding is checked or reported.

  • Findings from both go through the same independent check as every other finding, and serious ones are reproduced in the sandbox. Neither a check nor a skill can mark anything verified or proven.
  • Text in either that tries to change the result format, the rules of evidence or the level of certainty is disregarded.
  • Neither can switch off a built-in check or make a review pass or fail by itself. Their findings follow the severity gate like any others.
  • Both are meant for flaws an attacker could use. A style rule or a preference will mostly produce nothing.

#Which one to use

You wantUse
"Every handler under /internal must call requireStaff() first"A custom check
A rule about one folder or one kind of file, stated in a sentenceA custom check
A method with several steps: what to look for, how to confirm it, what does not count, how to report itA skill
To use a scanner or the code index as part of the methodA skill
A document as the result: a threat model in your house format, a test plan, a control mappingA skill, run on demand
To run it only when asked, against one repository or one pull requestA skill, run on demand
To have it on for most repositories and off for a fewA skill
To reuse a skill your team already wrote for a coding agentA skill: upload the folder

Both can be used together. A check and a skill that find the same flaw produce one finding, not two: candidates are merged before they are reported. See duplicates.

#Where to read more

Nullgap documentation