Beyond the hackbot: how AI-assisted vulnerability research works

AI-assisted vulnerability research is often described through the idea of a “hackbot”: a nearly autonomous agent that receives a target, runs a few tools, and finds a critical flaw on its own.

That image is appealing, but it does not describe how the work usually happens.

Tools such as Codex and Claude Code can read more code, test more hypotheses, and repeat tasks at a pace that is difficult to match manually. This volume improves coverage. It does not guarantee depth, creativity, or quality.

What makes the research productive is the system around the model. Heuristics decide where to look. Steering controls how each hypothesis is investigated. Execution produces evidence. A persistent corpus preserves what was learned and improves the next round.

AI does not replace the research process. It makes that process easier to execute, record, and refine continuously.

Why “find vulnerabilities” is not enough

A prompt such as:

analyze this project and find vulnerabilities

contains almost no strategy. The agent still has to decide which surfaces deserve attention, which bug classes to prioritize, how long to spend on each hypothesis, and when an observation justifies building a lab or harness.

Without clear criteria, the likely result is a shallow audit. The model searches for familiar sinks, lists suspicious flows, and stops before the expensive parts. This is not necessarily a capability problem. Depth has a cost.

A complex hypothesis may require reading several layers, compiling a specific version, reconstructing a feature, creating rare states, and using negative controls to eliminate alternative explanations. If the goal appears to be “deliver any result,” ten weak suspicions can look more useful than one unresolved investigation.

The initial instructions must correct that incentive. A high number of suspicions should not count as success. A finding should exist only after validation, and obvious paths in mature projects should receive less attention than inconsistencies that remain poorly understood.

The agent also needs permission to do the required work. Compiling, instrumenting, writing minimal applications, creating temporary tools, and fuzzing are not distractions. They often separate a plausible reading from a demonstrated flaw.

Heuristics and steering

Heuristics are the criteria used to choose where to research. Steering is the correction applied during an investigation to keep the agent focused on the right question.

In mature projects, the most interesting flaws rarely appear as a single, obviously dangerous call. They emerge in the differences between layers:

• one component validates a representation while another consumes a transformed form of the same value;
• a control protects the primary path but misses an equivalent entrypoint;
• two features that are safe in isolation break a guarantee when combined;
• an API answers a question that is similar, but not identical, to the question the code intended to ask.

These patterns do not prove a vulnerability. They identify places where the security model may diverge from actual behavior.

Macro steering defines rules for the whole project, such as requiring concrete impact, prioritizing trust boundaries, and recording negative results. Local steering controls one hypothesis. It may tell the agent to stop searching for new sinks and build a minimal reproduction, or to close a branch whose impact still depends on an unlikely premise.

Persistence without this control only creates longer loops. A good system must know when to go deeper and when to stop.

Evidence before findings

A hypothesis should not advance only because it sounds plausible. It must survive evidence gates.

An initial observation deserves a cheap check. If a concrete hypothesis remains, the next step is to show that controllable data or states can reach the relevant point. Only then does a full reproduction become a sensible investment.

Even reproduced behavior is not automatically a vulnerability. It may be expected, depend on an artificial configuration, or fail to break any security property. Impact analysis must identify the attacker, the exploitation condition, the authority gained, and the effect on a real target.

Most ideas should not reach the final stage. Rejecting them early is not failure; it is correct allocation of time.

The negative result still needs to be useful. “Not vulnerable” says very little. The record should explain why the hypothesis looked promising, how it was tested, which mechanism prevented impact, and what future changes could invalidate the conclusion. This lets a later round distinguish repetition from reassessment.

The persistent corpus

The corpus is the knowledge accumulated about the target. It extends beyond the project files and the chat history.

A useful corpus preserves:

• the architecture and its main trust boundaries;
• tested hypotheses, evidence, and validity conditions;
• versions, commands, scripts, harnesses, and reproduction environments;
• suspicious behavior that remains unresolved;
• discarded paths and the reasons they were closed.

Without this record, every session relearns the project. With several agents, the waste grows: they can rebuild the same environments, repeat the same analysis, and rediscover hypotheses that were already eliminated.

Persistent memory also does not mean placing everything in one large file. Context must be retrieved selectively. When entering a component, the agent should receive the decisions related to that area, not the entire research history. When reviewing a change, it should find older conclusions that depended on the behavior that changed.

This is the goal of Proteus, the framework we built at Vyntra for continuous vulnerability research. It does not try to turn security into an automatic button. Its purpose is to maintain coherence across rounds, record new knowledge, and keep the investigation out of old loops.

Models and prompts can be copied. A mature corpus is specific to the target. After several rounds, it becomes a map of the architecture, its guarantees, saturated areas, and questions that still justify the cost.

How one round improves the next

An initial round may discover that a feature uses three internal representations for the same resource. That observation enters the corpus.

The next round does not need to “analyze the feature” again. It can compare how authentication, caching, and canonicalization handle each representation. If two use the same pipeline while the third is resolved by a later layer, the next steering step can focus on that divergence and require dynamic validation.

Depth does not come from one perfect prompt. It emerges as every round reduces uncertainty and makes a better question possible.

This cycle also prevents poor use of context. Instead of sending the complete history to the model, the system retrieves the decisions that can change the current investigation. The agent starts with what has already been proven, what failed, and which conditions remain open.

Creativity in the gaps between layers

Creativity in security research does not mean inventing random attacks. It means connecting real behaviors that are usually examined in isolation.

This reasoning led to flaws we found in mature projects. In the Next.js RSC segment artifact issue, one layer evaluated the protected pathname while another delivered content through a different entrypoint. In the Vercel AI SDK tool allowlist bypass, the check returned a positive result for inherited properties that had never been explicitly approved.

Neither case depended on a function labeled “unsafe.” The flaw lived in the difference between the question the code meant to answer and the question it actually answered.

Agents are useful for maintaining several relationships in parallel, navigating distant implementations, and building tests quickly. The corpus must still supply reliable facts, and steering must require a concrete security break before promoting the connection.

Labs, fuzzing, and execution

AI-assisted research is not limited to reading code. Agents can build minimal applications, mock servers, protocol clients, instrumentation scripts, and fuzzing harnesses at low cost.

This shortens the path from suspicion to evidence. However, “do some fuzzing” remains a weak instruction. A campaign needs to define:

• the behavior being explored;
• the states the harness must reach;
• the oracle that identifies a meaningful result;
• controls that prove the relevant part of the component was exercised.

Coverage, crashes, and results must return to the corpus. They should guide new seeds, harness changes, and the decision to continue or stop the campaign. Without memory, weak campaigns are repeated. Without steering, results accumulate without a rule for deciding which ones deserve investigation.

One well-directed agent can outperform many

Multi-agent architectures increase coverage and parallelism, but they do not solve the central problem by themselves.

Without shared memory, clear surface ownership, and consistent promotion criteria, several agents repeat work in parallel and produce more noise. Scale amplifies both good and bad processes.

One agent with strong heuristics, relevant context, execution capability, and persistent memory can be more effective than dozens of generic agents walking through the same repository without coordination. In our work, this model has supported investigations of complex flaws in heavily reviewed targets, with Codex acting as the main executor.

Additional agents become useful when the system can separate research fronts, share evidence, and prevent duplication. Coordination quality comes before quantity.

The researcher remains at the center

The researcher’s role changes, but it does not disappear. Instead of performing every read and test manually, the researcher controls the research system.

The researcher sets priorities, identifies trust boundaries, evaluates exploitability, corrects interpretations, and decides when a hypothesis deserves more cost. They also add context that is not present in the code.

A behavior may be reproducible but depend on an unrealistic condition. Another may look small in isolation but break a central product guarantee. The agent can demonstrate that something happens. Deciding whether it is a meaningful vulnerability still requires threat modeling and judgment.

This is why “assisted research” describes the process better than “autonomous research.” AI expands execution and part of the reasoning, but direction and quality criteria remain the researcher’s responsibility.

Conclusion

AI-assisted vulnerability research is not a button that produces a CVE. It is also not a matter of running as many agents as possible until one finds something.

The process is iterative. Heuristics choose where to look. Steering defines how to go deeper. Execution produces evidence. Evaluation removes weak hypotheses. The corpus preserves what each round taught.

Over time, the agent should repeat less work, operate on a smaller search space, and receive more precise questions. That is how a system moves from shallow coverage to depth.

AI increases the speed of research. Heuristics provide direction. The corpus provides continuity.