Elusive Thoughts // offensive research

I Gave the Machine the Whole Repo. Then I Stopped Letting It Pretend to Be the Pentester.

Notes from a few weekends wiring up a RAG-driven whitebox workflow. What happens when the model can actually see the codebase, and why the suites still don't get it.

by an AppSec engineer who still runs the exploit by hand // 05.08.2026

Everyone is selling you an AI that finds vulnerabilities. Almost none of it works, and the reason is stupidly simple. The model never sees your code. It sees the snippet you paste, reasons about that snippet in a vacuum, and hands you a confident paragraph about a function that has no idea what calls it, what sanitises its input three hops upstream, or which route exposes it with no auth at all. That is autocomplete with a security vocabulary, dressed up as pentesting.

I do this for a living. AppSec during the week, pentest consulting on the side. So over a few weekends I ran the obvious experiment. What happens if the model can see everything, and I stop asking it to be me?

> 01 // grep is not a methodology

The first thing you need is an index of the entire repo that a model can actually query. Not a snippet. The whole thing, structured so that retrieval means something.

Naive chunking wrecks code. Split a file every 512 tokens and you slice a function in half, strip the decorator that put authentication on it, and orphan a route from the handler that serves it. So you chunk along the AST. One node per function, per route, per class, with the file path and the line span carried along as metadata so you can always walk back to source.

# AST-aware chunking. one unit per function or route, not every 512 tokens.
import tree_sitter_languages as tsl

parser = tsl.get_parser("python")
tree   = parser.parse(open("app/routes.py", "rb").read())

for node in walk(tree.root_node):
    if node.type in ("function_definition", "decorated_definition"):
        store.add(embed(src[node.start_byte:node.end_byte]), meta={
            "path": "app/routes.py",
            "span": (node.start_point[0], node.end_point[0]),
        })

Embed those chunks with a code-aware model, drop them into a vector store, and retrieval finally does something useful. When a finding lands on line 812 of a file, you don't hand the model line 812 and pray. You hand it the sink, the functions that reach it, and the middleware that is meant to guard it.

# pull the code AROUND a finding, not just the flagged line
ctx = store.query(
    embed("authorization check for POST /api/v2/orders"),
    k=8,
    where={"repo": "checkout-svc"},
)
# -> route handler + auth decorator + the ORM call it reaches

> 02 // feed it the noise you already make

You already run scanners. They already scream. The problem was never a lack of signal. It was that every tool screams in its own dialect and nothing joins the noise back to the code. So you normalise everything into one stream the indexer can read.

# every scanner, one folder, SARIF where we can so the indexer joins to code
semgrep scan --config auto --sarif -o out/semgrep.sarif ./src
trivy   fs   --scanners vuln,misconfig,secret -f sarif -o out/trivy.sarif .
checkov -d . -o sarif > out/checkov.sarif
nuclei  -l urls.txt -jsonl -o out/nuclei.jsonl

Semgrep tells you where a taint flows. Trivy and Checkov tell you which piece of infrastructure-as-code left a door open. Nuclei tells you what actually answered on the wire. Sitting apart, they are four piles of JSON that a human reconciles by hand at 2am. Joined to the indexed code, a Semgrep taint path stops being a line number and becomes a sentence: this user-controlled parameter reaches a raw query inside a handler that Checkov also flags as sitting behind a public load balancer. That sentence is a finding. The four JSON files were not.

> 03 // one methodology, many lenses

I did not want forty one-off scripts rotting in a repo. I wanted a single methodology that changes scope depending on what I am pointing it at. So the workflow is one method that pulls in different playbooks based on the task in front of me.

# same brain. different scope. pick the lens, not a fresh script.
pentest mode=endpoint  target=/api/v2/orders     # pre-release gate, one route
pentest mode=full      scope=app.example.com     # the whole application
pentest mode=cloud     account=prod-aws          # posture + privesc paths
pentest mode=api       spec=openapi.json         # authz, IDOR, mass assignment

Endpoint mode is the one that earns its rent. A team ships a new route on a Friday afternoon, and before it goes live you point the workflow at that one endpoint. It pulls the handler, the authorisation around it, the request schema, and every disclosed report that ever looked like this shape of bug. Ten minutes of focused work, not a two week engagement booked for next quarter.

> 04 // ground it in reality, not textbooks

A model with no ground truth invents plausible nonsense and does it with total confidence. So I ground it in two sources that were written by people who actually break things.

HackerOne hacktivity first. Disclosed reports are the closest public record we have of how bugs really look in production, written by the researchers who found them. Retrieval over real reports means the model reasons from how this IDOR was actually chained into account takeover, rather than a tidy definition scraped off a wiki.

Rhino Security Labs for the cloud side. Their AWS methodology and their tooling is still the reference the rest of us borrow from. Pacu handles the enumeration and the privilege-escalation hunt once you have a foothold.

# rhino's pacu: enumerate what the creds can do, then hunt privesc
run iam__enum_permissions
run iam__privesc_scan
run cloudtrail__download_event_history   # know what they log before you move

> 05 // front end, back end, the API in between

The index does not care which layer it is reading. It reads the front end for sources: DOM sinks, postMessage handlers, the hardcoded key somebody swore was rotated last spring, and the API calls the client actually makes. It reads the back end for sinks: the query built by string concatenation, the deserialiser fed untrusted input, the authorisation check that is present on nine routes and quietly missing on the tenth. Then it reads the API surface against the spec and flags where the running implementation has drifted from the OpenAPI contract. Mass assignment lives in that gap. So does a good chunk of the IDOR on the internet.

The model points. You confirm. The dynamic checks on the routes it flags are still yours to drive.

# the model points at a route, you confirm it by hand
ffuf   -u https://target/FUZZ -w routes.txt -mc 200,301,401,403
sqlmap -u "https://target/api/orders?id=1" --batch --level 3 --risk 2
# Burp for the manual driving a model still can't do

> 06 // the model is not the pentester

Here is the part the platforms selling you a magic box will not print on the landing page. The model is not the pentester. It is the fastest and most confident junior you have ever hired, and it will lie to your face in perfect grammar.

Left to run on its own it does two things that get you owned. It hallucinates vulnerabilities that were never there and burns a day of triage proving they aren't real. And it walks straight past the chained logic bug, the one where three low-severity findings quietly compose into full account takeover, because that chain lives in business context that no scanner and no isolated snippet can see. A human sees it. The machine retrieves the parts, the operator builds the exploit and proves the impact. Take the operator out of the loop and you are left with a very fast generator of confident rubbish.

This is the thing the suites and the big vendors don't understand, or don't want to. They keep selling autonomy. A competent operator handed instant context, at a speed that used to be impossible, is the actual product. That is a very different thing from the autonomous box on the slide deck.

> 07 // where the edge actually is

Two groups win with this, and they win for the same reason.

Hackers win because your hours go to the interesting five percent. The recon, the mapping, the grind of reading 40k lines for the one call that matters, that is the work the machine does while you sleep. You turn up for the chaining and the impact, which was always the part that took real skill.

Companies win because whitebox coverage stops being an annual ritual. You gate every release. You feed the SAST and DAST you already pay for into something that finally correlates it instead of just filing it. You get a pentester's lens on every endpoint before it ships, rather than a PDF in December describing code you wrote in March.

None of this replaces the engineer. It arms one. That is the whole point, and it is the part the market keeps getting wrong. Feed the machine everything you have. Let it read the codebase the way no human has the hours to. Then sit down in the chair and do the job it still cannot do.

Tooling referenced: Semgrep, Trivy, Checkov, Nuclei, ffuf, sqlmap, Burp Suite, Pacu. Methodology grounded in HackerOne hacktivity and Rhino Security Labs. The model does the reading. You do the pentest.