Skills
Custom checks or skills
The difference between a custom check and a skill, what each can and cannot do, and which one to reach for when you want Nullgap to look for something of your own.
There are two ways to tell Nullgap to look for something specific to your organisation: a custom check and a custom skill. They are different things with different reach.
- A custom check is a short brief that adds one thing to look for in every review and pentest.
- A skill is a whole packaged capability: when it applies, a playbook, the tools it may call and what it returns. It can also be run by itself on demand.
If you can say what you want in a sentence or two, write a custom check. If you need a method with steps, a tool, a document as the result, or something you can run on its own, write a skill.
#Side by side
| Custom check | Custom skill | |
|---|---|---|
| What you write | A name and a brief of 20 to 1,500 characters, in plain words | A SKILL.md in the Agent Skills format: frontmatter and a playbook of up to 48,000 characters |
| Where it lives | Settings, Custom checks tab | Skills, Custom skills tab |
| How many | 20 per organisation | 40 per organisation |
| Runs in pull request reviews | Yes, when it applies to a changed file | When it lists review and applies to the changed files |
| Runs in pentests | Yes, on the most exposed files it applies to | When it lists pentest and applies to the repository |
| Runs by itself, on demand | No | Yes, when it lists on-demand |
| When it applies | File globs and an optional regular expression on the file's text | File patterns, technologies and languages |
| Tools | None: it reads the code | The tools it names, where they exist and your organisation permits them |
| What it returns | Findings | Findings, a document, or a triage verdict on one finding |
| Can be switched off for one repository | No: a check is on or off for the organisation | Yes: overrides for one repository |
| How its findings are marked | finding.source starts with llm-custom:, the rule id with custom/ | finding.source is llm-skill: followed by the skill's id |
| Who manages it | Admins and owners | Admins and owners |
| Portable | No, it is a Nullgap setting | Yes, the format is open and used by coding agents |
#What they share
Neither can change how a finding is checked or reported.
- Findings from both go through the same independent check as every other finding, and serious ones are reproduced in the sandbox. Neither a check nor a skill can mark anything verified or proven.
- Text in either that tries to change the result format, the rules of evidence or the level of certainty is disregarded.
- Neither can switch off a built-in check or make a review pass or fail by itself. Their findings follow the severity gate like any others.
- Both are meant for flaws an attacker could use. A style rule or a preference will mostly produce nothing.
#Which one to use
| You want | Use |
|---|---|
"Every handler under /internal must call requireStaff() first" | A custom check |
| A rule about one folder or one kind of file, stated in a sentence | A custom check |
| A method with several steps: what to look for, how to confirm it, what does not count, how to report it | A skill |
| To use a scanner or the code index as part of the method | A skill |
| A document as the result: a threat model in your house format, a test plan, a control mapping | A skill, run on demand |
| To run it only when asked, against one repository or one pull request | A skill, run on demand |
| To have it on for most repositories and off for a few | A skill |
| To reuse a skill your team already wrote for a coding agent | A skill: upload the folder |
Both can be used together. A check and a skill that find the same flaw produce one finding, not two: candidates are merged before they are reported. See duplicates.
#Where to read more
- Project rules and custom checks: fields, limits and how a check runs.
- Custom skills: the format, uploading, and what a custom skill cannot do.
- Running a skill on demand: inputs, outputs and how far to rely on them.
Nullgap documentation