Two ways to keep a human in the loop
A runbook-driven session asks the model for a coherent plan, then uses observations to update it. A step-by-step session asks for one next action at a time. The first can preserve structure; the second can adapt to surprises. Neither removes the need to recognize when a proposed action is invalid, destructive or outside scope.
The report found useful assistance in service enumeration, HTTP inspection and identifying plausible next steps. It also found that the human often had to repair the environment or interpret a subtle response distinction. The apparent simplicity of “paste output back into chat” can hide considerable judgment about which output matters.
Case: the environment is not what the model assumes
A minimal Kali container differs from a full Kali VM. Packages, wordlist paths and network capabilities can be absent. In the reported Podman session, network-tool behavior was repeatedly misdiagnosed because the assistant assumed privileges that the container did not possess.
A sound troubleshooting sequence separates process startup, tool syntax, privilege errors, routing, DNS and target behavior. Preserve the exact stderr and exit status. Do not respond to every failure by granting a container full privileges: establish which capability is necessary and use the least-privileged isolated runtime that meets the experiment’s needs.
| Symptom | First question | Evidence to retain |
|---|---|---|
| File not found | Does this image contain the assumed list? | Image digest, package version, actual path |
| Permission denied | Which operation and capability failed? | Exact stderr and runtime permissions |
| Hostname error | Is the redirect host resolvable in this runtime? | Location header, resolver result, working IP evidence |
| HTTP 200 on every path | Is this a wildcard route or generic response? | Known-valid and random nonexistent controls |
Case: HTTP 200 does not establish discovery
The report describes Gobuster encountering a uniform 200 response with a body length of 8753 bytes. That number belongs to that observation; it is not a reusable filter for other sites. A nonexistent path can return a branded landing page, a redirect target or an application shell with status 200.
Use several negative controls and compare the full response context: status, content type, redirect chain, body similarity and meaningful application content. A single byte length is brittle when dynamic tokens or personalization change the response. State your finding narrowly: “these routes share a generic response” rather than “all routes exist.”
Case: plausible code is not executable evidence
Long sessions in the classic track included malformed command structure, guessed Python functions and incorrect treatment of command-line tools as importable libraries. The small quantized Gemma run supplied unsupported vulnerability and API claims. These observations support validating generated details, not dismissing all local models.
Before execution, check whether the executable exists, whether the proposed flags match its actual version, and whether a generated import is documented. Prefer one bounded action with an explicit expected observation to a long shell pipeline. A local syntax check can reveal malformed code without touching a target at all.
Record the human contribution
If a tester recognizes a wildcard response, repairs VPN routing or chooses to stop a fruitless branch, that is meaningful expert intervention. Record what they knew, what they changed and how the run would have behaved without that correction. Do not credit the model with work supplied by an unlogged human.
The report describes an actual administrative-login outcome in a human-guided vibe-hacking session. That must not be conflated with agentic configurations that only reached an account-takeover stage. It also discusses an unreproduced provider-monitoring anomaly; a single anomaly should remain a tentative observation, not a general claim about provider controls.
Checkpoint
An assistant recommends a CVE after seeing a Microsoft HTTP API banner. What is missing? The vulnerable component, exact affected version, patch state, required configuration and an authorized validation method. A banner is a lead for investigation, not a vulnerability finding. Keep the CVE hypothesis separate until supporting evidence exists.
References & source notes
Source mapping: supplied report, pp. 20–26. Read the full bibliography and editorial notes, or find a tool in the upstream reference catalogue.
Adapted by OverWatch Labs from the OWASP GenAI Security Project’s LLM Exploit Generation: Phase 2 (October 2026). Reorganized, paraphrased and supplemented with teaching examples. No OWASP endorsement implied.