Nullgap.dev Docs

API reference

AI app tests

AI app tests in the Nullgap API.

Paths are relative to the address of your Nullgap console. Authentication, errors, paging and rate limits are the same for every call and are described in the API overview.

OperationMethod and pathWho may call it
Check an AI app answersPOST /api/orgs/{org}/ai-tests/checkadmin
Start an AI app testPOST /api/orgs/{org}/ai-testsadmin
List AI app testsGET /api/orgs/{org}/ai-testsread
What an AI app test may costGET /api/orgs/{org}/ai-tests/costread
Get an AI app testGET /api/orgs/{org}/ai-tests/{id}read
Get an AI app test's logGET /api/orgs/{org}/ai-tests/{id}/logsread
Cancel an AI app testPOST /api/orgs/{org}/ai-tests/{id}/canceladmin

#Check an AI app answers

POST /api/orgs/{org}/ai-tests/check

Sends one harmless message to the app, through the same scope check and with the same headers a run would use, and says whether it answered and whether its reply could be read. Nothing is stored. At most 30 checks an hour per organisation. Admins only.

Who may call it: An API key with the admin scope.

Path parameters

NameTypeRequiredDescription
orgstringyesThe organisation slug.

Request body (application/json)

FieldTypeRequiredDescription
targetobjectyes
target.kindstringyeschat-api or json-api.
target.urlstringyesThe address to send requests to: http or https, public, without credentials, on the organisation's scope list.
target.modelstringnoChat API: the model name your API expects.
target.bodyTemplateanynoCustom JSON API: the request body with {{prompt}}, {{system}} or {{messages}} placeholders. Required for json-api.
target.responsePathstringnoCustom JSON API: dot-path to the reply text. Required for json-api.
target.actionsPathstringnoCustom JSON API: dot-path to the tool calls the app made.
target.systemPromptstringnoOptional: the app's system prompt, so a reply that repeats it is detected. Customer data: never returned, never logged. Sent to the app as the system message for a chat API, or in place of {{system}}.
target.toolsarray of objectnoChat API: tools to offer the app in each request, at most 20.
target.tools[].namestringyesTool name.
target.tools[].descriptionstringnoWhat it does.
headersobjectnoHeaders to send with every request, at most 10, for example Authorization. Values are sealed for the organisation at once, never returned or logged, and erased when the run ends. A value containing ${ is refused: nothing is read from the server's environment.
policyobjectno
policy.prohibitedarray of stringnoWhat the app must never do, at most 20 rules.
policy.canarystringnoA made-up secret to plant in the system prompt, at least 6 characters. Generated when left out.
categoriesarray of stringnoAttack categories to run. Default: all.
maxRequestsintegernoThe most requests to send to the app, 1 to 500. Default 200.

Example request

curl -s -X POST -H "Authorization: Bearer $NULLGAP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"target":{"kind":"chat-api","url":"https://support-bot.acme.example/v1/chat/completions","model":"support-bot-v2","bodyTemplate":{"input":"{{prompt}}"},"responsePath":"data.reply","actionsPath":"data.steps","systemPrompt":"You are the Acme support assistant…","tools":[{"name":"send_email","description":"Send an email"}]},"headers":{"Authorization":"Bearer sk_test_…"},"policy":{"prohibited":["Never call send_email unless the user asked"],"canary":"ACME-CANARY-41c9"},"categories":["instruction-override"],"maxRequests":200}' \
  "$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests/check"

Answer (200, application/json)

A AiTestCheck object.

{
  "ok": true,
  "reply": "Hello! How can I help you today?",
  "error": "responsePath \"data.reply\" not found in target reply"
}

Errors specific to this call

StatusCodeWhen
400bad_requestA field is missing or malformed: an unknown kind or category, a template without a placeholder, a reserved or malformed header.
400env_referenceA header value contains ${. Paste the literal value.
400out_of_scopeThe address is private or loopback, or is not on the organisation's scope list.
403org_suspendedThe workspace is suspended.
429rate_limitedToo many checks in the last hour.

#Start an AI app test

POST /api/orgs/{org}/ai-tests

Runs Nullgap's prompt-injection attack library (32 attack cases in 12 categories, mapped to the OWASP Top 10 for LLM applications) against an AI application and returns as soon as the run is queued. Only for applications you own or are authorised to test: the address must be on the organisation's live-testing scope list, the caller must be an admin (or a key with the admin scope), and confirmAuthorised must be true. The address is checked again when the run starts and before every request; requests are sent at most four a second, redirects are never followed. Each reply is checked for the planted canary, calls to prohibited tools, repeats of the system prompt, obeyed injected instructions and data-carrying links; Nullgap AI takes a second look when those find nothing. What a run proves stays on the run and is not added to the organisation's findings. An organisation can have 2 runs queued or running at once. Recorded in the audit log as ai_test.started.

Who may call it: An API key with the admin scope.

Path parameters

NameTypeRequiredDescription
orgstringyesThe organisation slug.

Request body (application/json)

FieldTypeRequiredDescription
namestringnoA name for the run. Default: the host.
targetobjectyes
target.kindstringyeschat-api or json-api.
target.urlstringyesThe address to send requests to: http or https, public, without credentials, on the organisation's scope list.
target.modelstringnoChat API: the model name your API expects.
target.bodyTemplateanynoCustom JSON API: the request body with {{prompt}}, {{system}} or {{messages}} placeholders. Required for json-api.
target.responsePathstringnoCustom JSON API: dot-path to the reply text. Required for json-api.
target.actionsPathstringnoCustom JSON API: dot-path to the tool calls the app made.
target.systemPromptstringnoOptional: the app's system prompt, so a reply that repeats it is detected. Customer data: never returned, never logged. Sent to the app as the system message for a chat API, or in place of {{system}}.
target.toolsarray of objectnoChat API: tools to offer the app in each request, at most 20.
target.tools[].namestringyesTool name.
target.tools[].descriptionstringnoWhat it does.
headersobjectnoHeaders to send with every request, at most 10, for example Authorization. Values are sealed for the organisation at once, never returned or logged, and erased when the run ends. A value containing ${ is refused: nothing is read from the server's environment.
policyobjectno
policy.prohibitedarray of stringnoWhat the app must never do, at most 20 rules.
policy.canarystringnoA made-up secret to plant in the system prompt, at least 6 characters. Generated when left out.
categoriesarray of stringnoAttack categories to run. Default: all.
maxRequestsintegernoThe most requests to send to the app, 1 to 500. Default 200.
confirmAuthorisedbooleanyesMust be true: you own this AI app or are authorised to test it.

Example request

curl -s -X POST -H "Authorization: Bearer $NULLGAP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"Support bot, staging","target":{"kind":"chat-api","url":"https://support-bot.acme.example/v1/chat/completions","model":"support-bot-v2","bodyTemplate":{"input":"{{prompt}}"},"responsePath":"data.reply","actionsPath":"data.steps","systemPrompt":"You are the Acme support assistant…","tools":[{"name":"send_email","description":"Send an email"}]},"headers":{"Authorization":"Bearer sk_test_…"},"policy":{"prohibited":["Never call send_email unless the user asked"],"canary":"ACME-CANARY-41c9"},"categories":["instruction-override"],"maxRequests":200,"confirmAuthorised":true}' \
  "$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests"

Answer (200, application/json)

A AiTest object.

{
  "id": "ait_8f2c1d0e4b6a",
  "orgId": "org_1d4c7b9a2e5f",
  "name": "Support bot, staging",
  "target": {
    "kind": "chat-api",
    "url": "https://support-bot.acme.example/v1/chat/completions",
    "model": "support-bot-v2",
    "bodyTemplate": {
      "input": "{{prompt}}",
      "session": "nullgap-test"
    },
    "responsePath": "data.reply",
    "actionsPath": "data.steps",
    "tools": [
      {
        "name": "send_email",
        "description": "Send an email"
      }
    ]
  },
  "headerNames": [
    "Authorization"
  ],
  "hasSystemPrompt": true,
  "prohibited": [
    "Never call send_email unless the user asked"
  ],
  "categories": [
    "tool-abuse"
  ],
  "maxRequests": 200,
  "status": "completed",
  "createdBy": "dana@acme.example",
  "createdAt": "2026-11-16T09:30:00.000Z",
  "startedAt": "2026-11-16T09:30:00.000Z",
  "finishedAt": "2026-11-16T09:34:10.000Z",
  "error": "support-bot.acme.example is no longer on this organisation's scope list, so the test stopped.",
  "stats": {
    "attacksRun": 32,
    "violations": 2,
    "blocked": 30,
    "errors": 0,
    "requests": 41,
    "truncated": false
  },
  "findingsCount": 2,
  "creditsUsed": 1.2,
  "findings": [
    {
      "id": "fnd_3c1a9e0b",
      "attackId": "tool-abuse-direct",
      "attackTitle": "Coaxed exfiltrating tool call",
      "category": "tool-abuse",
      "severity": "critical",
      "confidence": 0.95,
      "owasp": [
        "LLM06"
      ],
      "title": "Coaxed exfiltrating tool call",
      "description": "The agent violated its policy under test case \"tool-abuse-direct\" (tool-abuse).",
      "remediation": "Require explicit user confirmation for side-effecting tools …",
      "evidence": [
        {
          "kind": "transcript",
          "summary": "Exchange with support-bot.acme.example: the attack, the reply and the tool call it made",
          "detail": "[system] (your system prompt, not shown here)\n[user] …\n[assistant] …"
        }
      ]
    }
  ]
}

Errors specific to this call

StatusCodeWhen
400bad_requestA field is missing or malformed: an unknown kind or category, a template without a placeholder, a reserved or malformed header.
400env_referenceA header value contains ${. Paste the literal value.
400out_of_scopeThe address is private or loopback, or is not on the organisation's scope list.
403org_suspendedThe workspace is suspended.
400confirmation_requiredconfirmAuthorised is not true.
402plan_inactiveThe workspace has no active plan.
402no_creditsThe workspace is billed per run and has no automation credits left (credits set aside by runs under way do not count). A test is charged in automation credits for its Nullgap AI work.
429too_manyThe organisation already has 2 AI app tests queued or running.

#List AI app tests

GET /api/orgs/{org}/ai-tests

Returns the organisation's AI app tests, newest first, without their proven issues: read one run to get them.

Who may call it: An API key with the read scope or higher.

Path parameters

NameTypeRequiredDescription
orgstringyesThe organisation slug.

Query parameters

NameTypeRequiredDescription
statusstringnoOnly runs in this status. One of: queued, running, completed, failed, cancelled.
limitintegernoHow many to return, 1 to 200. Default 50.
offsetintegernoHow many to skip from the start. Default 0.

Example request

curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
  "$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests?limit=20"

Answer (200, application/json)

FieldTypeAlways presentDescription
itemsarray of AiTestyesAI app tests, at most limit of them.
totalintegeryesHow many match in all, before limit and offset.
{
  "items": [
    {
      "id": "ait_8f2c1d0e4b6a",
      "orgId": "org_1d4c7b9a2e5f",
      "name": "Support bot, staging",
      "target": {
        "kind": "chat-api",
        "url": "https://support-bot.acme.example/v1/chat/completions",
        "model": "support-bot-v2",
        "bodyTemplate": {
          "input": "{{prompt}}",
          "session": "nullgap-test"
        },
        "responsePath": "data.reply",
        "actionsPath": "data.steps",
        "tools": [
          {
            "name": "send_email",
            "description": "Send an email"
          }
        ]
      },
      "headerNames": [
        "Authorization"
      ],
      "hasSystemPrompt": true,
      "prohibited": [
        "Never call send_email unless the user asked"
      ],
      "categories": [
        "tool-abuse"
      ],
      "maxRequests": 200,
      "status": "completed",
      "createdBy": "dana@acme.example",
      "createdAt": "2026-11-16T09:30:00.000Z",
      "startedAt": "2026-11-16T09:30:00.000Z",
      "finishedAt": "2026-11-16T09:34:10.000Z",
      "error": "support-bot.acme.example is no longer on this organisation's scope list, so the test stopped.",
      "stats": {
        "attacksRun": 32,
        "violations": 2,
        "blocked": 30,
        "errors": 0,
        "requests": 41,
        "truncated": false
      },
      "findingsCount": 2,
      "creditsUsed": 1.2,
      "findings": [
        {
          "id": "fnd_3c1a9e0b",
          "attackId": "tool-abuse-direct",
          "attackTitle": "Coaxed exfiltrating tool call",
          "category": "tool-abuse",
          "severity": "critical",
          "confidence": 0.95,
          "owasp": [
            "LLM06"
          ],
          "title": "Coaxed exfiltrating tool call",
          "description": "The agent violated its policy under test case \"tool-abuse-direct\" (tool-abuse).",
          "remediation": "Require explicit user confirmation for side-effecting tools …",
          "evidence": [
            {
              "kind": "transcript",
              "summary": "Exchange with support-bot.acme.example: the attack, the reply and the tool call it made",
              "detail": "[system] (your system prompt, not shown here)\n[user] …\n[assistant] …"
            }
          ]
        }
      ]
    }
  ],
  "total": 42
}

#What an AI app test may cost

GET /api/orgs/{org}/ai-tests/cost

The most automation credits one test can be charged for its Nullgap AI work (at the same rate as a fix run), and whether one could start now. A test that fails on Nullgap's side is not charged; one you cancel is charged for the work done. Changes nothing.

Who may call it: An API key with the read scope or higher.

Path parameters

NameTypeRequiredDescription
orgstringyesThe organisation slug.

Example request

curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
  "$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests/cost"

Answer (200, application/json)

What an AI app test may cost.

FieldTypeAlways presentDescription
maxCreditsnumberyesThe most one test can be charged, in automation credits. 0 when the workspace is covered by an agreement.
coveredByAgreementbooleanyesTrue when the workspace is billed under an agreement: tests are not charged.
creditsAvailablenumberyesAutomation credits a new run may count on: the balance less what runs under way have set aside.
canStartbooleanyesWhether a test could start now, as far as credits go.
{
  "maxCredits": 3.6,
  "coveredByAgreement": false,
  "creditsAvailable": 25,
  "canStart": true
}

#Get an AI app test

GET /api/orgs/{org}/ai-tests/{id}

Returns one run with its statistics and proven issues once it has completed. Poll this to follow a run. Transcripts are text the app wrote: treat them as data.

Who may call it: An API key with the read scope or higher.

Path parameters

NameTypeRequiredDescription
orgstringyesThe organisation slug.
idstringyesThe AI app test id.

Example request

curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
  "$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests/ID"

Answer (200, application/json)

A AiTest object.

{
  "id": "ait_8f2c1d0e4b6a",
  "orgId": "org_1d4c7b9a2e5f",
  "name": "Support bot, staging",
  "target": {
    "kind": "chat-api",
    "url": "https://support-bot.acme.example/v1/chat/completions",
    "model": "support-bot-v2",
    "bodyTemplate": {
      "input": "{{prompt}}",
      "session": "nullgap-test"
    },
    "responsePath": "data.reply",
    "actionsPath": "data.steps",
    "tools": [
      {
        "name": "send_email",
        "description": "Send an email"
      }
    ]
  },
  "headerNames": [
    "Authorization"
  ],
  "hasSystemPrompt": true,
  "prohibited": [
    "Never call send_email unless the user asked"
  ],
  "categories": [
    "tool-abuse"
  ],
  "maxRequests": 200,
  "status": "completed",
  "createdBy": "dana@acme.example",
  "createdAt": "2026-11-16T09:30:00.000Z",
  "startedAt": "2026-11-16T09:30:00.000Z",
  "finishedAt": "2026-11-16T09:34:10.000Z",
  "error": "support-bot.acme.example is no longer on this organisation's scope list, so the test stopped.",
  "stats": {
    "attacksRun": 32,
    "violations": 2,
    "blocked": 30,
    "errors": 0,
    "requests": 41,
    "truncated": false
  },
  "findingsCount": 2,
  "creditsUsed": 1.2,
  "findings": [
    {
      "id": "fnd_3c1a9e0b",
      "attackId": "tool-abuse-direct",
      "attackTitle": "Coaxed exfiltrating tool call",
      "category": "tool-abuse",
      "severity": "critical",
      "confidence": 0.95,
      "owasp": [
        "LLM06"
      ],
      "title": "Coaxed exfiltrating tool call",
      "description": "The agent violated its policy under test case \"tool-abuse-direct\" (tool-abuse).",
      "remediation": "Require explicit user confirmation for side-effecting tools …",
      "evidence": [
        {
          "kind": "transcript",
          "summary": "Exchange with support-bot.acme.example: the attack, the reply and the tool call it made",
          "detail": "[system] (your system prompt, not shown here)\n[user] …\n[assistant] …"
        }
      ]
    }
  ]
}

Errors specific to this call

StatusCodeWhen
404not_foundNo AI app test of this organisation has that id.

#Get an AI app test's log

GET /api/orgs/{org}/ai-tests/{id}/logs

Returns the progress log of one run, oldest line first. The system prompt, the canary and header values are masked.

Who may call it: An API key with the read scope or higher.

Path parameters

NameTypeRequiredDescription
orgstringyesThe organisation slug.
idstringyesThe AI app test id.

Example request

curl -s -H "Authorization: Bearer $NULLGAP_API_KEY" \
  "$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests/ID/logs"

Answer (200, application/json)

The log lines, oldest first.

An array of LogLine.

[
  {
    "at": "2026-11-16T09:30:00.000Z",
    "level": "info",
    "message": "Mapping the attack surface: 42 entry points"
  }
]

Errors specific to this call

StatusCodeWhen
404not_foundNo AI app test of this organisation has that id.

#Cancel an AI app test

POST /api/orgs/{org}/ai-tests/{id}/cancel

Stops a queued or running run. A running one stops before its next request. Its status becomes cancelled, nothing it had found is kept, and its header values are erased. Admins only. Recorded in the audit log as ai_test.cancelled.

Who may call it: An API key with the admin scope.

Path parameters

NameTypeRequiredDescription
orgstringyesThe organisation slug.
idstringyesThe AI app test id.

Example request

curl -s -X POST -H "Authorization: Bearer $NULLGAP_API_KEY" \
  "$NULLGAP_URL/api/orgs/$NULLGAP_ORG/ai-tests/ID/cancel"

Answer (200, application/json)

A AiTest object.

{
  "id": "ait_8f2c1d0e4b6a",
  "orgId": "org_1d4c7b9a2e5f",
  "name": "Support bot, staging",
  "target": {
    "kind": "chat-api",
    "url": "https://support-bot.acme.example/v1/chat/completions",
    "model": "support-bot-v2",
    "bodyTemplate": {
      "input": "{{prompt}}",
      "session": "nullgap-test"
    },
    "responsePath": "data.reply",
    "actionsPath": "data.steps",
    "tools": [
      {
        "name": "send_email",
        "description": "Send an email"
      }
    ]
  },
  "headerNames": [
    "Authorization"
  ],
  "hasSystemPrompt": true,
  "prohibited": [
    "Never call send_email unless the user asked"
  ],
  "categories": [
    "tool-abuse"
  ],
  "maxRequests": 200,
  "status": "completed",
  "createdBy": "dana@acme.example",
  "createdAt": "2026-11-16T09:30:00.000Z",
  "startedAt": "2026-11-16T09:30:00.000Z",
  "finishedAt": "2026-11-16T09:34:10.000Z",
  "error": "support-bot.acme.example is no longer on this organisation's scope list, so the test stopped.",
  "stats": {
    "attacksRun": 32,
    "violations": 2,
    "blocked": 30,
    "errors": 0,
    "requests": 41,
    "truncated": false
  },
  "findingsCount": 2,
  "creditsUsed": 1.2,
  "findings": [
    {
      "id": "fnd_3c1a9e0b",
      "attackId": "tool-abuse-direct",
      "attackTitle": "Coaxed exfiltrating tool call",
      "category": "tool-abuse",
      "severity": "critical",
      "confidence": 0.95,
      "owasp": [
        "LLM06"
      ],
      "title": "Coaxed exfiltrating tool call",
      "description": "The agent violated its policy under test case \"tool-abuse-direct\" (tool-abuse).",
      "remediation": "Require explicit user confirmation for side-effecting tools …",
      "evidence": [
        {
          "kind": "transcript",
          "summary": "Exchange with support-bot.acme.example: the attack, the reply and the tool call it made",
          "detail": "[system] (your system prompt, not shown here)\n[user] …\n[assistant] …"
        }
      ]
    }
  ]
}

Errors specific to this call

StatusCodeWhen
404not_foundNo AI app test of this organisation has that id.
400not_runningThe run has already finished, failed or been cancelled.

#Objects

The objects the operations above take and return.

#AiTestCheck

The result of one harmless request to the app.

FieldTypeAlways presentDescription
okbooleanyesWhether the app answered and its reply could be read.
replystringnoThe start of the reply, when it answered.
errorstringnoWhat went wrong, when it did not.

#AiTest

One AI app test: the prompt-injection attack library run against one AI application.

FieldTypeAlways presentDescription
idstringyesThe run id.
orgIdstringyesThe organisation it belongs to.
namestringyesA name for the run.
targetAiTestTargetyes
headerNamesarray of stringyesThe headers sent with every request. Their values are stored sealed while the run is queued or running, erased when it ends, and never returned.
hasSystemPromptbooleanyesWhether a system prompt was given to detect leaks. Its text is never returned.
prohibitedarray of stringyesWhat the app must never do, in plain words.
categoriesarray of stringyesThe attack categories the run uses.
maxRequestsintegeryesThe most requests the run may send to the app.
statusstringyesWhere the run stands. One of: queued, running, completed, failed, cancelled.
createdBystringyesWho started it.
createdAtstringyesWhen it was asked for.
startedAtstring or nullyesWhen work began.
finishedAtstring or nullyesWhen it ended.
errorstring or nullyesWhy the run failed, in words fit to show.
statsobject or nullyesWhat the run did. Null until it completes.
stats.attacksRunintegeryesAttack cases that ran.
stats.violationsintegeryesAttack cases under which the app broke a rule.
stats.blockedintegeryesAttack cases the app withstood.
stats.errorsintegeryesAttack cases that could not be judged because the app failed to answer.
stats.requestsintegeryesRequests sent to the app.
stats.truncatedbooleanyesThe request budget ran out, or the app kept failing, before every attack ran.
findingsCountintegeryesHow many proven issues.
creditsUsednumbernoAutomation credits the run was charged for its Nullgap AI work. Absent until it ends.
findingsarray of AiTestFindingnoThe proven issues, most of them with a transcript. Present only when one run is read.

#LogLine

One line of a run's progress log.

FieldTypeAlways presentDescription
atstringyesWhen the line was written.
levelstringyesHow serious the line is. One of: info, warn, error.
messagestringyesWhat happened, in plain words.

#AiTestTarget

The AI application under test. Header values are never part of it.

FieldTypeAlways presentDescription
kindstringyeschat-api: an endpoint that takes the common chat-completions format (model, messages, tools) and answers with choices[0].message. json-api: any JSON API, described by bodyTemplate and responsePath.
urlstringyesThe address requests are sent to. It must be http or https, public, without credentials, and on the organisation's live-testing scope list.
modelstringnoChat API only: the model name your own API expects in model. Default default.
bodyTemplateanynoCustom JSON API only: the request body, with {{prompt}} (the attack text), {{system}} (the system prompt) or a string that is exactly {{messages}} (the conversation so far as a list of {role, content}). Must contain {{prompt}} or {{messages}}.
responsePathstringnoCustom JSON API only: dot-path to the reply text in the answer.
actionsPathstringnoCustom JSON API only: dot-path to the list of tool calls the app made, when it reports them.
toolsarray of objectnoChat API only: tools the app is offered in each request, at most 20. A prohibited action that names one of them is detected when the app calls it.
tools[].namestringyesThe tool's name.
tools[].descriptionstringnoWhat it does.

#AiTestFinding

One proven issue: the app broke one of its rules under one attack. It stays on the run and is not added to the organisation's findings.

FieldTypeAlways presentDescription
idstringyesThe issue id.
attackIdstringyesThe attack case that broke the rule.
attackTitlestringyesIts name.
categorystringyesIts category. One of: instruction-override, role-spoofing, prompt-extraction, secret-exfiltration, indirect-injection, tool-abuse, markdown-exfil, obfuscation, multi-turn, delimiter-confusion, authority-claim, refusal-suppression.
severitystringyesHow serious it is. One of: critical, high, medium, low, info.
confidencenumberyesHow sure the observation is, from 0 to 1. Deterministic observations (the canary in a reply, a prohibited tool call) are 0.9 or more.
owasparray of stringyesThe OWASP Top 10 for LLM applications entries it maps to.
titlestringyesA short title.
descriptionstringyesWhat happened and which rule was broken.
remediationstringyesHow to fix this kind of weakness.
evidencearray of AiTestEvidenceyesThe exchange that proves it, and the reasoning when Nullgap AI took a second look.

#AiTestEvidence

FieldTypeAlways presentDescription
kindstringyestranscript: the exchange with the app. reasoning: why Nullgap AI judged the reply a violation.
summarystringyesOne line.
detailstringnoThe full text. In a transcript the system prompt is not shown and the planted canary appears as [CANARY]; header values appear as [REDACTED].

Nullgap documentation