API reference
AI app tests
AI app tests in the Nullgap API.
Paths are relative to the address of your Nullgap console. Authentication, errors, paging and rate limits are the same for every call and are described in the API overview.
| Operation | Method and path | Who may call it |
|---|---|---|
| Check an AI app answers | POST /api/orgs/{org}/ai-tests/check | admin |
| Start an AI app test | POST /api/orgs/{org}/ai-tests | admin |
| List AI app tests | GET /api/orgs/{org}/ai-tests | read |
| What an AI app test may cost | GET /api/orgs/{org}/ai-tests/cost | read |
| Get an AI app test | GET /api/orgs/{org}/ai-tests/{id} | read |
| Get an AI app test's log | GET /api/orgs/{org}/ai-tests/{id}/logs | read |
| Cancel an AI app test | POST /api/orgs/{org}/ai-tests/{id}/cancel | admin |
#Check an AI app answers
POST /api/orgs/{org}/ai-tests/check
Sends one harmless message to the app, through the same scope check and with the same headers a run would use, and says whether it answered and whether its reply could be read. Nothing is stored. At most 30 checks an hour per organisation. Admins only.
Who may call it: An API key with the admin scope.
Path parameters
| Name | Type | Required | Description |
|---|---|---|---|
org | string | yes | The organisation slug. |
Request body (application/json)
| Field | Type | Required | Description |
|---|---|---|---|
target | object | yes | |
target.kind | string | yes | chat-api or json-api. |
target.url | string | yes | The address to send requests to: http or https, public, without credentials, on the organisation's scope list. |
target.model | string | no | Chat API: the model name your API expects. |
target.bodyTemplate | any | no | Custom JSON API: the request body with {{prompt}}, {{system}} or {{messages}} placeholders. Required for json-api. |
target.responsePath | string | no | Custom JSON API: dot-path to the reply text. Required for json-api. |
target.actionsPath | string | no | Custom JSON API: dot-path to the tool calls the app made. |
target.systemPrompt | string | no | Optional: the app's system prompt, so a reply that repeats it is detected. Customer data: never returned, never logged. Sent to the app as the system message for a chat API, or in place of {{system}}. |
target.tools | array of object | no | Chat API: tools to offer the app in each request, at most 20. |
target.tools[].name | string | yes | Tool name. |
target.tools[].description | string | no | What it does. |
headers | object | no | Headers to send with every request, at most 10, for example Authorization. Values are sealed for the organisation at once, never returned or logged, and erased when the run ends. A value containing ${ is refused: nothing is read from the server's environment. |
policy | object | no | |
policy.prohibited | array of string | no | What the app must never do, at most 20 rules. |
policy.canary | string | no | A made-up secret to plant in the system prompt, at least 6 characters. Generated when left out. |
categories | array of string | no | Attack categories to run. Default: all. |
maxRequests | integer | no | The most requests to send to the app, 1 to 500. Default 200. |
Example request
curl -s -X POST -H "Authorization: Bearer $NULLGAP_API_KEY" \
-H "Content-Type: application/json" \
-d '{"target":{"kind":"chat-api","url":"https://support-bot.acme.example/v1/chat/completions","model":"support-bot-v2","bodyTemplate":{"input":"{{prompt}}"},"responsePath":"data.reply","actionsPath":"data.steps","systemPrompt":"You are the Acme support assistant…","tools":[{"name":"send_email","description":"Send an email"}]},"headers":{"Authorization":"Bearer sk_test_…"},"policy":{"prohibited":["Never call send_email unless the user asked"],"canary":"ACME-CANARY-41c9"},"categories":["instruction-override"],"maxRequests":200}' \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests/check"
Answer (200, application/json)
A AiTestCheck object.
{
"ok": true,
"reply": "Hello! How can I help you today?",
"error": "responsePath \"data.reply\" not found in target reply"
}
Errors specific to this call
| Status | Code | When |
|---|---|---|
| 400 | bad_request | A field is missing or malformed: an unknown kind or category, a template without a placeholder, a reserved or malformed header. |
| 400 | env_reference | A header value contains ${. Paste the literal value. |
| 400 | out_of_scope | The address is private or loopback, or is not on the organisation's scope list. |
| 403 | org_suspended | The workspace is suspended. |
| 429 | rate_limited | Too many checks in the last hour. |
#Start an AI app test
POST /api/orgs/{org}/ai-tests
Runs Nullgap's prompt-injection attack library (32 attack cases in 12 categories, mapped to the OWASP Top 10 for LLM applications) against an AI application and returns as soon as the run is queued. Only for applications you own or are authorised to test: the address must be on the organisation's live-testing scope list, the caller must be an admin (or a key with the admin scope), and confirmAuthorised must be true. The address is checked again when the run starts and before every request; requests are sent at most four a second, redirects are never followed. Each reply is checked for the planted canary, calls to prohibited tools, repeats of the system prompt, obeyed injected instructions and data-carrying links; Nullgap AI takes a second look when those find nothing. What a run proves stays on the run and is not added to the organisation's findings. An organisation can have 2 runs queued or running at once. Recorded in the audit log as ai_test.started.
Who may call it: An API key with the admin scope.
Path parameters
| Name | Type | Required | Description |
|---|---|---|---|
org | string | yes | The organisation slug. |
Request body (application/json)
| Field | Type | Required | Description |
|---|---|---|---|
name | string | no | A name for the run. Default: the host. |
target | object | yes | |
target.kind | string | yes | chat-api or json-api. |
target.url | string | yes | The address to send requests to: http or https, public, without credentials, on the organisation's scope list. |
target.model | string | no | Chat API: the model name your API expects. |
target.bodyTemplate | any | no | Custom JSON API: the request body with {{prompt}}, {{system}} or {{messages}} placeholders. Required for json-api. |
target.responsePath | string | no | Custom JSON API: dot-path to the reply text. Required for json-api. |
target.actionsPath | string | no | Custom JSON API: dot-path to the tool calls the app made. |
target.systemPrompt | string | no | Optional: the app's system prompt, so a reply that repeats it is detected. Customer data: never returned, never logged. Sent to the app as the system message for a chat API, or in place of {{system}}. |
target.tools | array of object | no | Chat API: tools to offer the app in each request, at most 20. |
target.tools[].name | string | yes | Tool name. |
target.tools[].description | string | no | What it does. |
headers | object | no | Headers to send with every request, at most 10, for example Authorization. Values are sealed for the organisation at once, never returned or logged, and erased when the run ends. A value containing ${ is refused: nothing is read from the server's environment. |
policy | object | no | |
policy.prohibited | array of string | no | What the app must never do, at most 20 rules. |
policy.canary | string | no | A made-up secret to plant in the system prompt, at least 6 characters. Generated when left out. |
categories | array of string | no | Attack categories to run. Default: all. |
maxRequests | integer | no | The most requests to send to the app, 1 to 500. Default 200. |
confirmAuthorised | boolean | yes | Must be true: you own this AI app or are authorised to test it. |
Example request
curl -s -X POST -H "Authorization: Bearer $NULLGAP_API_KEY" \
-H "Content-Type: application/json" \
-d '{"name":"Support bot, staging","target":{"kind":"chat-api","url":"https://support-bot.acme.example/v1/chat/completions","model":"support-bot-v2","bodyTemplate":{"input":"{{prompt}}"},"responsePath":"data.reply","actionsPath":"data.steps","systemPrompt":"You are the Acme support assistant…","tools":[{"name":"send_email","description":"Send an email"}]},"headers":{"Authorization":"Bearer sk_test_…"},"policy":{"prohibited":["Never call send_email unless the user asked"],"canary":"ACME-CANARY-41c9"},"categories":["instruction-override"],"maxRequests":200,"confirmAuthorised":true}' \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests"
Answer (200, application/json)
A AiTest object.
{
"id": "ait_8f2c1d0e4b6a",
"orgId": "org_1d4c7b9a2e5f",
"name": "Support bot, staging",
"target": {
"kind": "chat-api",
"url": "https://support-bot.acme.example/v1/chat/completions",
"model": "support-bot-v2",
"bodyTemplate": {
"input": "{{prompt}}",
"session": "nullgap-test"
},
"responsePath": "data.reply",
"actionsPath": "data.steps",
"tools": [
{
"name": "send_email",
"description": "Send an email"
}
]
},
"headerNames": [
"Authorization"
],
"hasSystemPrompt": true,
"prohibited": [
"Never call send_email unless the user asked"
],
"categories": [
"tool-abuse"
],
"maxRequests": 200,
"status": "completed",
"createdBy": "dana@acme.example",
"createdAt": "2026-11-16T09:30:00.000Z",
"startedAt": "2026-11-16T09:30:00.000Z",
"finishedAt": "2026-11-16T09:34:10.000Z",
"error": "support-bot.acme.example is no longer on this organisation's scope list, so the test stopped.",
"stats": {
"attacksRun": 32,
"violations": 2,
"blocked": 30,
"errors": 0,
"requests": 41,
"truncated": false
},
"findingsCount": 2,
"creditsUsed": 1.2,
"findings": [
{
"id": "fnd_3c1a9e0b",
"attackId": "tool-abuse-direct",
"attackTitle": "Coaxed exfiltrating tool call",
"category": "tool-abuse",
"severity": "critical",
"confidence": 0.95,
"owasp": [
"LLM06"
],
"title": "Coaxed exfiltrating tool call",
"description": "The agent violated its policy under test case \"tool-abuse-direct\" (tool-abuse).",
"remediation": "Require explicit user confirmation for side-effecting tools …",
"evidence": [
{
"kind": "transcript",
"summary": "Exchange with support-bot.acme.example: the attack, the reply and the tool call it made",
"detail": "[system] (your system prompt, not shown here)\n[user] …\n[assistant] …"
}
]
}
]
}
Errors specific to this call
| Status | Code | When |
|---|---|---|
| 400 | bad_request | A field is missing or malformed: an unknown kind or category, a template without a placeholder, a reserved or malformed header. |
| 400 | env_reference | A header value contains ${. Paste the literal value. |
| 400 | out_of_scope | The address is private or loopback, or is not on the organisation's scope list. |
| 403 | org_suspended | The workspace is suspended. |
| 400 | confirmation_required | confirmAuthorised is not true. |
| 402 | plan_inactive | The workspace has no active plan. |
| 402 | no_credits | The workspace is billed per run and has no automation credits left (credits set aside by runs under way do not count). A test is charged in automation credits for its Nullgap AI work. |
| 429 | too_many | The organisation already has 2 AI app tests queued or running. |
#List AI app tests
GET /api/orgs/{org}/ai-tests
Returns the organisation's AI app tests, newest first, without their proven issues: read one run to get them.
Who may call it: An API key with the read scope or higher.
Path parameters
| Name | Type | Required | Description |
|---|---|---|---|
org | string | yes | The organisation slug. |
Query parameters
| Name | Type | Required | Description |
|---|---|---|---|
status | string | no | Only runs in this status. One of: queued, running, completed, failed, cancelled. |
limit | integer | no | How many to return, 1 to 200. Default 50. |
offset | integer | no | How many to skip from the start. Default 0. |
Example request
curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests?limit=20"
Answer (200, application/json)
| Field | Type | Always present | Description |
|---|---|---|---|
items | array of AiTest | yes | AI app tests, at most limit of them. |
total | integer | yes | How many match in all, before limit and offset. |
{
"items": [
{
"id": "ait_8f2c1d0e4b6a",
"orgId": "org_1d4c7b9a2e5f",
"name": "Support bot, staging",
"target": {
"kind": "chat-api",
"url": "https://support-bot.acme.example/v1/chat/completions",
"model": "support-bot-v2",
"bodyTemplate": {
"input": "{{prompt}}",
"session": "nullgap-test"
},
"responsePath": "data.reply",
"actionsPath": "data.steps",
"tools": [
{
"name": "send_email",
"description": "Send an email"
}
]
},
"headerNames": [
"Authorization"
],
"hasSystemPrompt": true,
"prohibited": [
"Never call send_email unless the user asked"
],
"categories": [
"tool-abuse"
],
"maxRequests": 200,
"status": "completed",
"createdBy": "dana@acme.example",
"createdAt": "2026-11-16T09:30:00.000Z",
"startedAt": "2026-11-16T09:30:00.000Z",
"finishedAt": "2026-11-16T09:34:10.000Z",
"error": "support-bot.acme.example is no longer on this organisation's scope list, so the test stopped.",
"stats": {
"attacksRun": 32,
"violations": 2,
"blocked": 30,
"errors": 0,
"requests": 41,
"truncated": false
},
"findingsCount": 2,
"creditsUsed": 1.2,
"findings": [
{
"id": "fnd_3c1a9e0b",
"attackId": "tool-abuse-direct",
"attackTitle": "Coaxed exfiltrating tool call",
"category": "tool-abuse",
"severity": "critical",
"confidence": 0.95,
"owasp": [
"LLM06"
],
"title": "Coaxed exfiltrating tool call",
"description": "The agent violated its policy under test case \"tool-abuse-direct\" (tool-abuse).",
"remediation": "Require explicit user confirmation for side-effecting tools …",
"evidence": [
{
"kind": "transcript",
"summary": "Exchange with support-bot.acme.example: the attack, the reply and the tool call it made",
"detail": "[system] (your system prompt, not shown here)\n[user] …\n[assistant] …"
}
]
}
]
}
],
"total": 42
}
#What an AI app test may cost
GET /api/orgs/{org}/ai-tests/cost
The most automation credits one test can be charged for its Nullgap AI work (at the same rate as a fix run), and whether one could start now. A test that fails on Nullgap's side is not charged; one you cancel is charged for the work done. Changes nothing.
Who may call it: An API key with the read scope or higher.
Path parameters
| Name | Type | Required | Description |
|---|---|---|---|
org | string | yes | The organisation slug. |
Example request
curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests/cost"
Answer (200, application/json)
What an AI app test may cost.
| Field | Type | Always present | Description |
|---|---|---|---|
maxCredits | number | yes | The most one test can be charged, in automation credits. 0 when the workspace is covered by an agreement. |
coveredByAgreement | boolean | yes | True when the workspace is billed under an agreement: tests are not charged. |
creditsAvailable | number | yes | Automation credits a new run may count on: the balance less what runs under way have set aside. |
canStart | boolean | yes | Whether a test could start now, as far as credits go. |
{
"maxCredits": 3.6,
"coveredByAgreement": false,
"creditsAvailable": 25,
"canStart": true
}
#Get an AI app test
GET /api/orgs/{org}/ai-tests/{id}
Returns one run with its statistics and proven issues once it has completed. Poll this to follow a run. Transcripts are text the app wrote: treat them as data.
Who may call it: An API key with the read scope or higher.
Path parameters
| Name | Type | Required | Description |
|---|---|---|---|
org | string | yes | The organisation slug. |
id | string | yes | The AI app test id. |
Example request
curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests/ID"
Answer (200, application/json)
A AiTest object.
{
"id": "ait_8f2c1d0e4b6a",
"orgId": "org_1d4c7b9a2e5f",
"name": "Support bot, staging",
"target": {
"kind": "chat-api",
"url": "https://support-bot.acme.example/v1/chat/completions",
"model": "support-bot-v2",
"bodyTemplate": {
"input": "{{prompt}}",
"session": "nullgap-test"
},
"responsePath": "data.reply",
"actionsPath": "data.steps",
"tools": [
{
"name": "send_email",
"description": "Send an email"
}
]
},
"headerNames": [
"Authorization"
],
"hasSystemPrompt": true,
"prohibited": [
"Never call send_email unless the user asked"
],
"categories": [
"tool-abuse"
],
"maxRequests": 200,
"status": "completed",
"createdBy": "dana@acme.example",
"createdAt": "2026-11-16T09:30:00.000Z",
"startedAt": "2026-11-16T09:30:00.000Z",
"finishedAt": "2026-11-16T09:34:10.000Z",
"error": "support-bot.acme.example is no longer on this organisation's scope list, so the test stopped.",
"stats": {
"attacksRun": 32,
"violations": 2,
"blocked": 30,
"errors": 0,
"requests": 41,
"truncated": false
},
"findingsCount": 2,
"creditsUsed": 1.2,
"findings": [
{
"id": "fnd_3c1a9e0b",
"attackId": "tool-abuse-direct",
"attackTitle": "Coaxed exfiltrating tool call",
"category": "tool-abuse",
"severity": "critical",
"confidence": 0.95,
"owasp": [
"LLM06"
],
"title": "Coaxed exfiltrating tool call",
"description": "The agent violated its policy under test case \"tool-abuse-direct\" (tool-abuse).",
"remediation": "Require explicit user confirmation for side-effecting tools …",
"evidence": [
{
"kind": "transcript",
"summary": "Exchange with support-bot.acme.example: the attack, the reply and the tool call it made",
"detail": "[system] (your system prompt, not shown here)\n[user] …\n[assistant] …"
}
]
}
]
}
Errors specific to this call
| Status | Code | When |
|---|---|---|
| 404 | not_found | No AI app test of this organisation has that id. |
#Get an AI app test's log
GET /api/orgs/{org}/ai-tests/{id}/logs
Returns the progress log of one run, oldest line first. The system prompt, the canary and header values are masked.
Who may call it: An API key with the read scope or higher.
Path parameters
| Name | Type | Required | Description |
|---|---|---|---|
org | string | yes | The organisation slug. |
id | string | yes | The AI app test id. |
Example request
curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests/ID/logs"
Answer (200, application/json)
The log lines, oldest first.
An array of LogLine.
[
{
"at": "2026-11-16T09:30:00.000Z",
"level": "info",
"message": "Mapping the attack surface: 42 entry points"
}
]
Errors specific to this call
| Status | Code | When |
|---|---|---|
| 404 | not_found | No AI app test of this organisation has that id. |
#Cancel an AI app test
POST /api/orgs/{org}/ai-tests/{id}/cancel
Stops a queued or running run. A running one stops before its next request. Its status becomes cancelled, nothing it had found is kept, and its header values are erased. Admins only. Recorded in the audit log as ai_test.cancelled.
Who may call it: An API key with the admin scope.
Path parameters
| Name | Type | Required | Description |
|---|---|---|---|
org | string | yes | The organisation slug. |
id | string | yes | The AI app test id. |
Example request
curl -s -X POST -H "Authorization: Bearer $NULLGAP_API_KEY" \
"$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests/ID/cancel"
Answer (200, application/json)
A AiTest object.
{
"id": "ait_8f2c1d0e4b6a",
"orgId": "org_1d4c7b9a2e5f",
"name": "Support bot, staging",
"target": {
"kind": "chat-api",
"url": "https://support-bot.acme.example/v1/chat/completions",
"model": "support-bot-v2",
"bodyTemplate": {
"input": "{{prompt}}",
"session": "nullgap-test"
},
"responsePath": "data.reply",
"actionsPath": "data.steps",
"tools": [
{
"name": "send_email",
"description": "Send an email"
}
]
},
"headerNames": [
"Authorization"
],
"hasSystemPrompt": true,
"prohibited": [
"Never call send_email unless the user asked"
],
"categories": [
"tool-abuse"
],
"maxRequests": 200,
"status": "completed",
"createdBy": "dana@acme.example",
"createdAt": "2026-11-16T09:30:00.000Z",
"startedAt": "2026-11-16T09:30:00.000Z",
"finishedAt": "2026-11-16T09:34:10.000Z",
"error": "support-bot.acme.example is no longer on this organisation's scope list, so the test stopped.",
"stats": {
"attacksRun": 32,
"violations": 2,
"blocked": 30,
"errors": 0,
"requests": 41,
"truncated": false
},
"findingsCount": 2,
"creditsUsed": 1.2,
"findings": [
{
"id": "fnd_3c1a9e0b",
"attackId": "tool-abuse-direct",
"attackTitle": "Coaxed exfiltrating tool call",
"category": "tool-abuse",
"severity": "critical",
"confidence": 0.95,
"owasp": [
"LLM06"
],
"title": "Coaxed exfiltrating tool call",
"description": "The agent violated its policy under test case \"tool-abuse-direct\" (tool-abuse).",
"remediation": "Require explicit user confirmation for side-effecting tools …",
"evidence": [
{
"kind": "transcript",
"summary": "Exchange with support-bot.acme.example: the attack, the reply and the tool call it made",
"detail": "[system] (your system prompt, not shown here)\n[user] …\n[assistant] …"
}
]
}
]
}
Errors specific to this call
| Status | Code | When |
|---|---|---|
| 404 | not_found | No AI app test of this organisation has that id. |
| 400 | not_running | The run has already finished, failed or been cancelled. |
#Objects
The objects the operations above take and return.
#AiTestCheck
The result of one harmless request to the app.
| Field | Type | Always present | Description |
|---|---|---|---|
ok | boolean | yes | Whether the app answered and its reply could be read. |
reply | string | no | The start of the reply, when it answered. |
error | string | no | What went wrong, when it did not. |
#AiTest
One AI app test: the prompt-injection attack library run against one AI application.
| Field | Type | Always present | Description |
|---|---|---|---|
id | string | yes | The run id. |
orgId | string | yes | The organisation it belongs to. |
name | string | yes | A name for the run. |
target | AiTestTarget | yes | |
headerNames | array of string | yes | The headers sent with every request. Their values are stored sealed while the run is queued or running, erased when it ends, and never returned. |
hasSystemPrompt | boolean | yes | Whether a system prompt was given to detect leaks. Its text is never returned. |
prohibited | array of string | yes | What the app must never do, in plain words. |
categories | array of string | yes | The attack categories the run uses. |
maxRequests | integer | yes | The most requests the run may send to the app. |
status | string | yes | Where the run stands. One of: queued, running, completed, failed, cancelled. |
createdBy | string | yes | Who started it. |
createdAt | string | yes | When it was asked for. |
startedAt | string or null | yes | When work began. |
finishedAt | string or null | yes | When it ended. |
error | string or null | yes | Why the run failed, in words fit to show. |
stats | object or null | yes | What the run did. Null until it completes. |
stats.attacksRun | integer | yes | Attack cases that ran. |
stats.violations | integer | yes | Attack cases under which the app broke a rule. |
stats.blocked | integer | yes | Attack cases the app withstood. |
stats.errors | integer | yes | Attack cases that could not be judged because the app failed to answer. |
stats.requests | integer | yes | Requests sent to the app. |
stats.truncated | boolean | yes | The request budget ran out, or the app kept failing, before every attack ran. |
findingsCount | integer | yes | How many proven issues. |
creditsUsed | number | no | Automation credits the run was charged for its Nullgap AI work. Absent until it ends. |
findings | array of AiTestFinding | no | The proven issues, most of them with a transcript. Present only when one run is read. |
#LogLine
One line of a run's progress log.
| Field | Type | Always present | Description |
|---|---|---|---|
at | string | yes | When the line was written. |
level | string | yes | How serious the line is. One of: info, warn, error. |
message | string | yes | What happened, in plain words. |
#AiTestTarget
The AI application under test. Header values are never part of it.
| Field | Type | Always present | Description |
|---|---|---|---|
kind | string | yes | chat-api: an endpoint that takes the common chat-completions format (model, messages, tools) and answers with choices[0].message. json-api: any JSON API, described by bodyTemplate and responsePath. |
url | string | yes | The address requests are sent to. It must be http or https, public, without credentials, and on the organisation's live-testing scope list. |
model | string | no | Chat API only: the model name your own API expects in model. Default default. |
bodyTemplate | any | no | Custom JSON API only: the request body, with {{prompt}} (the attack text), {{system}} (the system prompt) or a string that is exactly {{messages}} (the conversation so far as a list of {role, content}). Must contain {{prompt}} or {{messages}}. |
responsePath | string | no | Custom JSON API only: dot-path to the reply text in the answer. |
actionsPath | string | no | Custom JSON API only: dot-path to the list of tool calls the app made, when it reports them. |
tools | array of object | no | Chat API only: tools the app is offered in each request, at most 20. A prohibited action that names one of them is detected when the app calls it. |
tools[].name | string | yes | The tool's name. |
tools[].description | string | no | What it does. |
#AiTestFinding
One proven issue: the app broke one of its rules under one attack. It stays on the run and is not added to the organisation's findings.
| Field | Type | Always present | Description |
|---|---|---|---|
id | string | yes | The issue id. |
attackId | string | yes | The attack case that broke the rule. |
attackTitle | string | yes | Its name. |
category | string | yes | Its category. One of: instruction-override, role-spoofing, prompt-extraction, secret-exfiltration, indirect-injection, tool-abuse, markdown-exfil, obfuscation, multi-turn, delimiter-confusion, authority-claim, refusal-suppression. |
severity | string | yes | How serious it is. One of: critical, high, medium, low, info. |
confidence | number | yes | How sure the observation is, from 0 to 1. Deterministic observations (the canary in a reply, a prohibited tool call) are 0.9 or more. |
owasp | array of string | yes | The OWASP Top 10 for LLM applications entries it maps to. |
title | string | yes | A short title. |
description | string | yes | What happened and which rule was broken. |
remediation | string | yes | How to fix this kind of weakness. |
evidence | array of AiTestEvidence | yes | The exchange that proves it, and the reasoning when Nullgap AI took a second look. |
#AiTestEvidence
| Field | Type | Always present | Description |
|---|---|---|---|
kind | string | yes | transcript: the exchange with the app. reasoning: why Nullgap AI judged the reply a violation. |
summary | string | yes | One line. |
detail | string | no | The full text. In a transcript the system prompt is not shown and the planted canary appears as [CANARY]; header values appear as [REDACTED]. |
Nullgap documentation