By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.
ToolAccuracy of FindingsDetects Non-Pattern-Based Issues?Coverage of SAST FindingsSpeed of ScanningUsability & Dev Experience
DryRun SecurityVery high – caught multiple critical issues missed by othersYes – context-based analysis, logic flaws & SSRFBroad coverage of standard vulns, logic flaws, and extendableNear real-time PR feedback
Snyk CodeHigh on well-known patterns (SQLi, XSS), but misses other categoriesLimited – AI-based, focuses on recognized vulnerabilitiesGood coverage of standard vulns; may miss SSRF or advanced auth logic issuesFast, often near PR speedDecent GitHub integration, but rules are a black box
GitHub Advanced Security (CodeQL)Very high precision for known queries, low false positivesPartial – strong dataflow for known issues, needs custom queriesGood for SQLi and XSS but logic flaws require advanced CodeQL experience.Moderate to slow (GitHub Action based)Requires CodeQL expertise for custom logic
SemgrepMedium, but there is a good community for adding rulesPrimarily pattern-based with limited dataflowDecent coverage with the right rules, can still miss advanced logic or SSRFFast scansHas custom rules, but dev teams must maintain them
SonarQubeLow – misses serious issues in our testingLimited – mostly pattern-based, code quality orientedBasic coverage for standard vulns, many hotspots require manual reviewModerate, usually in CIDashboard-based approach, can pass “quality gate” despite real vulns
Vulnerability ClassSnyk (partial)GitHub (CodeQL) (partial)SemgrepSonarQubeDryRun Security
SQL Injection
*
Cross-Site Scripting (XSS)
SSRF
Auth Flaw / IDOR
User Enumeration
Hardcoded Token
ToolAccuracy of FindingsDetects Non-Pattern-Based Issues?Coverage of C# VulnerabilitiesScan SpeedDeveloper Experience
DryRun Security
Very high – caught all critical flaws missed by others
Yes – context-based analysis finds logic errors, auth flaws, etc.
Broad coverage of OWASP Top 10 vulns plus business logic issuesNear real-time (PR comment within seconds)Clear single PR comment with detailed insights; no config or custom scripts needed
Snyk CodeHigh on known patterns (SQLi, XSS), but misses logic/flow bugsLimited – focuses on recognizable vulnerability patterns
Good for standard vulns; may miss SSRF or auth logic issues 
Fast (integrates into PR checks)Decent GitHub integration, but rules are a black box (no easy customization)
GitHub Advanced Security (CodeQL)Low - missed everything except SQL InjectionMostly pattern-basedLow – only discovered SQL InjectionSlowest of all but finished in 1 minuteConcise annotation with a suggested fix and optional auto-remedation
SemgrepMedium – finds common issues with community rules, some missesPrimarily pattern-based, limited data flow analysis
Decent coverage with the right rules; misses advanced logic flaws 
Very fast (runs as lightweight CI)Custom rules possible, but require maintenance and security expertise
SonarQube
Low – missed serious issues in our testing
Mostly pattern-based (code quality focus)Basic coverage for known vulns; many issues flagged as “hotspots” require manual review Moderate (runs in CI/CD pipeline)Results in dashboard; risk of false sense of security if quality gate passes despite vulnerabilities
Vulnerability ClassSnyk CodeGitHub Advanced Security (CodeQL)SemgrepSonarQubeDryRun Security
SQL Injection (SQLi)
Cross-Site Scripting (XSS)
Server-Side Request Forgery (SSRF)
Auth Logic/IDOR
User Enumeration
Hardcoded Credentials
VulnerabilityDryRun SecuritySemgrepGitHub CodeQLSonarQubeSnyk Code
1. Remote Code Execution via Unsafe Deserialization
2. Code Injection via eval() Usage
3. SQL Injection in a Raw Database Query
4. Weak Encryption (AES ECB Mode)
5. Broken Access Control / Logic Flaw in Authentication
Total Found5/53/51/51/50/5
VulnerabilityDryRun SecuritySnykCodeQLSonarQubeSemgrep
Server-Side Request Forgery (SSRF)
(Hotspot)
Cross-Site Scripting (XSS)
SQL Injection (SQLi)
IDOR / Broken Access Control
Invalid Token Validation Logic
Broken Email Verification Logic
DimensionWhy It Matters
Surface
Entry points & data sources highlight tainted flows early.
Language
Code idioms reveal hidden sinks and framework quirks.
Intent
What is the purpose of the code being changed/added?
Design
Robustness and resilience of changing code.
Environment
Libraries, build flags, and infra metadata flag, infrastructure (IaC) all give clues around the risks in changing code.
KPIPattern-Based SASTDryRun CSA
Mean Time to Regex
3–8 hrs per noisy finding set
Not required
Mean Time to Context
N/A
< 1 min
False-Positive Rate
50–85 %< 5 %
Logic-Flaw Detection
< 5 %
90%+
Severity
CriticalHigh
Location
utils/authorization.py :L118
utils/authorization.py :L49 & L82 & L164
Issue
JWT Algorithm Confusion Attack:
jwt.decode() selects the algorithm from unverified JWT headers.
Insecure OIDC Endpoint Communication:
urllib.request.urlopen called without explicit TLS/CA handling.
Impact
Complete auth bypass (switch RS256→HS256, forge tokens with public key as HMAC secret).
Susceptible to MITM if default SSL behavior is weakened or cert store compromised.
Remediation
Replace the dynamic algorithm selection with a fixed, expected algorithm list. Change line 118 from algorithms=[unverified_header.get('alg', 'RS256')] to algorithms=['RS256'] to only accept RS256 tokens. Add algorithm validation before token verification to ensure the header algorithm matches expected values.
Create a secure SSL context using ssl.create_default_context() with proper certificate verification. Configure explicit timeout values for all HTTP requests to prevent hanging connections. Add explicit SSL/TLS configuration by creating an HTTPSHandler with the secure SSL context. Implement proper error handling specifically for SSL certificate validation failures.
Key Insight
This vulnerability arises from trusting an unverified portion of the JWT to determine the verification method itself
This vulnerability stems from a lack of explicit secure communication practices, leaving the application reliant on potentially weak default behaviors.
AI in AppSec
September 9, 2026

Anthropic Reimagined the SDLC. AppSec Got Markdown Files

Anthropic recently published The AI-Native Software Development Lifecycle, a playbook for what happens when AI stops being a coding assistant and starts participating across the entire software development lifecycle. There is a lot in the playbook that I agree with, including its basic premise: AI is increasing the rate at which software can be produced, and that changes the economics of everything surrounding software development. Requirements, design, implementation, testing, review, deployment, operations, and security all have to keep up with a development process that increasingly operates at machine speed.

Anthropic takes that premise seriously throughout much of the playbook. Product management changes. Engineering changes. Testing changes. Operations changes. Agents don't merely help humans perform the old workflow faster; in many cases, the workflow itself is reconstructed around agents performing work, handing it to other agents, receiving feedback, correcting themselves, and escalating to humans when necessary.

Then they get to application security, and somehow, after imagining an entirely different way of building software, Anthropic imagines a remarkably familiar future for the people responsible for securing it.

Security teams encode their policies and guidance so Claude can consume them. Security concerns discovered during design can be routed to policy owners. Existing deterministic security controls remain in CI. Claude performs security review. Security teams connect repositories to Claude Security, schedule scans, triage the results, and feed larger problems back into the development lifecycle.

None of those are necessarily bad ideas. In fact, organizations should probably be doing many of them. But if that is our vision for AppSec in an AI-native SDLC, we have profoundly underestimated what this technology makes possible. We haven't reimagined AppSec; we've mostly given today's AppSec program some AI tools.

Everyone Gets an Agent Except the Security Team

This is particularly strange because Anthropic correctly identifies the problem. Security organizations are already sized around human software-development throughput, and if agents dramatically increase the amount of software being produced, security teams cannot perform their existing work a little faster and expect to keep up.

AppSec has always been outgunned. Most security teams can't threat model everything, aren't involved in every design conversation, can't deeply review every meaningful code change, and can't immediately investigate every external vulnerability report. They certainly can't continuously reevaluate every application as its architecture, dependencies, exposure, and business purpose change.

Instead, enormous amounts of AppSec capacity get consumed by what I think of as the finding economy: finding vulnerabilities, validating them, triaging them, assigning them, explaining them, chasing remediation, retesting them, and eventually closing them before starting the process over again. That work is necessary, but the backlog is gravitational. When enough of the organization is consumed servicing findings, it becomes incredibly difficult to spend time on the higher-order work that could reduce risk systemically.

AI complicates this problem because it can simultaneously make the backlog better and worse. AI-generated software means more code, more changes, more infrastructure, more dependencies, and more opportunities to introduce vulnerabilities. At the same time, agentic security gives us the ability to review more code, investigate more vulnerabilities, perform more threat modeling, provide more design consultation, automate remediation, correlate more signals, and operate with a level of coverage that wasn't practical when every meaningful security activity required human labor.

That shouldn't result in less security or an AppSec organization with nothing left to do. The opportunity is to automate enough of the mechanical and tedious work that has historically kept us underwater so we can dramatically expand security coverage while finally spending more time on the conditions that create systemic risk in the first place.

If Engineering Manages Fleets of Agents, So Will Security

Imagine the development environment Anthropic and others are describing actually arrives. A customer submits a feature request, a product agent helps interpret it and develops specifications, engineering agents implement it, other agents test and review it, deployment agents move it toward production, and operational agents monitor what happens afterward. Humans increasingly supervise systems performing work that humans previously performed themselves and these workflows are described in greater detail, here: “Nobody Makes Their Own Screws Anymore”

There is no compelling reason security should be different.

Threat modeling, secure design, guardrails, security training, code review, vulnerability management, external vulnerability-report management, bug bounty, consultation, incident response, detection, and organizational risk management can all be reimagined around the same capabilities. Importantly, that doesn't mean attaching an agent or a chatbot to every security process. Some of this work was already automatable, while other parts required forms of judgment that conventional automation couldn't provide.

Dependency remediation is a useful example. We've been comfortable with a primitive form of autonomous remediation for years through tools like Dependabot. What agents change is the breadth of work that can plausibly be automated. Traditional automation wasn't particularly good at reading a bug bounty submission, understanding the researcher's claim, investigating the relevant application behavior, determining whether the report was a duplicate, reasoning about exploitability and business impact, asking useful follow-up questions, finding similar patterns elsewhere, preparing remediation guidance, and routing the issue appropriately. Those activities require judgment, and semantic reasoning gives us a way to automate classes of security work that were previously difficult or impossible to express as deterministic software.

The same applies to secure-design consultation. The first interaction no longer has to be a developer waiting several days for a security engineer to join a meeting. An autonomous security capability could understand what someone intends to build, retrieve relevant organizational requirements, examine the existing application and architecture, identify likely trust and authorization boundaries, and provide first-level design guidance. Novel, ambiguous, or particularly consequential decisions can still escalate to a human security engineer, and what that engineer learns from resolving those situations can improve the system handling the next thousand consultations.

This pattern repeats throughout AppSec. Training doesn't disappear, but now we may need to train both humans and agents. Threat modeling doesn't disappear; instead, we gain the ability to perform it continuously while reserving scarce human attention for novel or unusually risky systems. PSIRT doesn't disappear; agents can help correlate signals, develop hypotheses, investigate affected systems, reconstruct events, recommend containment, and eventually perform bounded response actions as organizations become comfortable with their reliability. Code review certainly doesn't disappear either. We stop pretending that every security-relevant change must wait for a human being to manually inspect it.

The security practitioner becomes more important in this environment, not less. Their work begins moving up an abstraction layer.

The Security Engineer Starts Managing the Security Capability

Managing autonomous security systems has an interesting parallel to managing people. When we manage a team, we establish expectations, give people the information and tools they need, measure their work, notice when performance changes, investigate why, improve processes, constrain behavior where necessary, and determine which work can be delegated versus which work requires more experienced judgment.

Agentic systems require many of the same management functions, except we are also responsible for engineering pieces of the system performing the reasoning. Consider an autonomous threat-modeling workflow that performed well last month but suddenly begins missing authorization risks. The underlying threat-modeling agent may not be the problem at all. Perhaps the model changed, Git graph reconstruction is broken, framework detection is incorrect, repository context stopped identifying an important relationship between services, or the system is retrieving the wrong organizational authorization requirements. An upstream probabilistic operation may have shifted slightly and poisoned information consumed downstream. It is even possible that the entire system is behaving exactly as designed while the application environment has changed enough that the evaluation used to measure performance no longer represents what "good" looks like.

At that point, the security engineer's job is no longer merely to inspect the resulting threat model. They need to understand and operate the system capable of producing threat models. They need observability into the signals, context, reasoning, tools, and decisions that contributed to the result, along with evaluations capable of telling them when the quality of that capability is beginning to regress.

This leads to what I think will become one of the defining requirements of the next generation of application security:

The next generation of AppSec will require security practitioners to understand the systems performing security on their behalf.

We've been through versions of this transition before. When DevOps and infrastructure-as-code changed how applications were built and deployed, AppSec couldn't decide that infrastructure wasn't our problem. Security practitioners increasingly needed to understand cloud infrastructure, IAM, containers, CI/CD pipelines, Kubernetes, Terraform, secrets management, and the systems through which software reached production. The abstraction changed, so the competency had to change with it.

Agentic development creates another expansion. It is no longer sufficient only to understand the systems developers build; we increasingly need to understand the systems that build the systems. If AppSec is also going to build and operate autonomous security capabilities of its own, that requirement becomes even stronger.

Automation Does Not Mean Simplification

There is a tempting conclusion that organizations can draw from all of this: if AI performs more security work, humans perform less security work, and therefore organizations need fewer security resources, less training, and fewer tools. I think that badly misunderstands what is happening.

Automation can reduce the human execution required for individual security activities while simultaneously increasing the technical sophistication required of the humans responsible for those activities. We don't look at an engineer operating cloud infrastructure, Terraform, Kubernetes, distributed systems, CI/CD, observability platforms, and managed services and conclude that all this automation means they require less technical education. Automation raised the abstraction level at which they work.

The same will happen in AppSec. A security engineer will (and already are) manually review fewer pull requests while needing to understand models, harnesses, orchestration, evaluations, observability, context engineering, retrieval, tool use, permissions, deterministic and probabilistic system design, failure propagation, and agent governance. Underneath all of that, they still need to understand security well enough to recognize whether the system they're operating is producing good security outcomes.

If anything, organizations should expect the learning requirements for AppSec practitioners to increase during this transition. This technology does not eliminate expertise; it gives expertise leverage. One excellent threat modeler can help build and supervise a capability that threat models thousands of changes. One excellent secure-design engineer can build a consultation capability available across an entire engineering organization. One excellent vulnerability researcher can improve the machinery investigating every external vulnerability report. The practitioner does less of the repetitive execution personally, while their expertise influences dramatically more of the organization.

Autonomy Should Expand as Assurance Expands

None of this requires pretending agents are perfect, nor does it require establishing some permanent boundary where agents may only recommend actions while humans always make the consequential decisions. We've already accepted autonomous security operations in limited forms, and the appetite for significantly more autonomy already exists. On sales calls, I routinely hear security teams asking for exactly this with broader classes of vulnerabilities: give me the easy button, find these issues, and go fix them.

Agents expand the set of activities where meaningful automation is possible, but organizations don't have to jump immediately from manual operation to unrestricted autonomy. They can begin by observing what a system would have done, move toward recommendations, allow bounded operations with approval, measure outcomes, and expand the autonomous boundary as confidence increases. Operations that once seemed too consequential to automate may eventually become routine as the systems, harnesses, models, evaluations, and organizational experience around them improve.

That is largely how organizational trust in AI has already developed. The important distinction is that trust cannot become permanent because a system performed well during the last evaluation cycle.

Trust in autonomous security systems has to be continuously earned and continuously verified.

A workflow that performed acceptably yesterday isn't guaranteed to perform acceptably tomorrow. Models change, prompts change, applications change, organizational knowledge changes, retrieval changes, signals change, orchestration changes, and tooling changes. Providers themselves can change model behavior. This is why evaluations and observability cannot be something we bolt onto an agentic security system after we build it. They need to be foundational to the architecture.

Before delegating responsibility, we need some conception of what good performance means and instrumentation capable of telling us whether we're still getting it. When an evaluation begins declining, we also need enough observability to understand whether the problem came from a model, prompt, retrieval system, deterministic signal, organizational knowledge source, context-assembly process, orchestration change, evaluator, or something else entirely.

Don't just evaluate the outputs. Evaluate the conditions that produced them.

Probabilism Requires Discipline

There is another trap waiting for security teams adopting this technology: using AI for everything  because we can. An AI-native AppSec program should not be an enormous collection of LLMs asking other LLMs questions when reliable deterministic methods already exist.

Every probabilistic operation introduces another opportunity to get something wrong, and in an agentic system those errors can propagate. Imagine a simplified pipeline where repository understanding contributes to application modeling, application modeling informs a threat model, the threat model influences secure design, secure design informs implementation, and the resulting implementation is eventually reviewed and remediated. A subtle misunderstanding early in that chain can poison everything downstream. The threat model can confidently reason from an incorrect premise, the design agent can incorporate it, the implementation agent can faithfully build it, and a reviewing agent consuming similar incorrect context may even validate the resulting decision.

The result can be an extraordinarily sophisticated system that is consistently wrong.

The more consequential and autonomous these systems become, the more disciplined we need to become about where probabilism is introduced. If information can be reliably established through conventional software, establish it that way. Build and inspect the Git graph rather than asking a model to infer it. Parse dependency manifests and framework versions directly. Determine which files changed, query cloud APIs for actual state, read IAM policies, and retrieve the organization's actual policy rather than asking a probabilistic system to approximate facts that are already knowable.

Then use semantic reasoning where semantic reasoning provides meaningful leverage: understanding whether a change violates the intended business authorization model, interpreting a messy vulnerability report, reasoning about exploitability, or synthesizing architecture, business context, code behavior, and organizational policy into a threat model.

Probabilism has a budget, and that budget isn't measured only in tokens, compute, latency, and dollars. It is also measured in reliability. We should spend it where reasoning buys us something meaningful.

Security Has to Secure the Agents Too

Putting agents throughout the SDLC also introduces another responsibility for security. Historically, AppSec primarily asked whether the software developers were producing was safe. Now we also have to ask whether the autonomous systems producing and operating that software are safe.

An agent can produce perfectly secure code while behaving dangerously as an operator. That means we have to reason about what agents are allowed to access, which tools they can invoke, what information they can retrieve, where they can write, which operations require approval, and how their blast radius is constrained. We need to understand what happens when an agent interprets a legitimate objective more broadly than the organization intended, how abnormal behavior is detected, how actions are audited, and how one compromised or confused component can be prevented from poisoning the rest of a workflow.

AppSec increasingly has two related responsibilities: security performed by agents and security of the agents performing the work. Both require assurance and governance, and both require practitioners who understand the systems they're governing.

From Managing Findings to Managing Risk

This is where the opportunity becomes much larger than automated code review. What if AppSec finally stopped defining its operational reality primarily around findings? Not because findings stop mattering, but because we finally gain enough machine capacity to continuously understand something closer to the risk state of an application or service.

A service isn't risky because it has seventeen open findings. Its actual risk is informed by architecture, exposure, identity boundaries, business criticality, ownership, dependencies, code-change velocity, historical vulnerabilities, deployed controls, runtime behavior, incidents, organizational policy, and the relationships between all of them. Humans have always been capable of reasoning about those factors. What we haven't had is enough human capacity to do it continuously across hundreds or thousands of applications while simultaneously keeping up with the vulnerability backlog.

Agentic systems give us an opportunity to change that relationship. We can introduce more signals and feeds, correlate information that previously lived in disconnected tools, continuously reassess services as their architecture and behavior change, and identify where systemic risk is accumulating rather than counting findings. The security organization can increasingly ask where its next hour of effort will remove the most actual risk rather than asking which item is next in the vulnerability queue.

That is a fundamentally different operating model, and it is why I reject the idea that AI necessarily makes the AppSec job easier or gives security teams less to do. There may be more to do because we can finally cover things we never had enough people to cover before. Work that happens once a year can become continuous. Design guidance that reaches a fraction of developers can potentially reach all of them. Humans can operate at higher levels of the organization while autonomous systems handle more of the repetitive mechanics underneath them.

AI can simultaneously increase the volume of software we need to secure and increase our capacity to secure it. The appropriate response isn't to shrink the ambition of the security organization. It's to expand it.

An Actually AI-Native AppSec Playbook

That is ultimately what frustrates me about Anthropic's vision. The problem isn't that security policies shouldn't be made available to agents; they should. The problem isn't that AI shouldn't perform security review; it absolutely should. Automated security analysis, remediation, and tools like Claude Security are obviously going to be part of this future.

The problem is stopping there.

If we're going to seriously entertain a world in which product management, engineering, testing, deployment, and operations are reconstructed around autonomous systems, then telling AppSec to maintain the security guidance and run the AI-powered security review represents a remarkable failure of imagination. AppSec doesn't merely need an AI-native scanner. It needs an AI-native operating model.

That operating model should allow security practitioners to build and manage autonomous security capabilities across threat modeling, consultation, training, vulnerability management, incident response, secure design, and the other disciplines that already make up the security function. Humans can increasingly focus on novel problems, systemic risk, architecture, governance, and improving the machinery performing work beneath them, while autonomy expands in proportion to the assurance we've established around those systems.

It should also be an operating model built with discipline. Deterministic systems should establish what we can reliably know, while probabilistic reasoning is deliberately applied where judgment creates meaningful leverage. Evals and observability should be foundational, and trust should be continuously earned rather than assumed because yesterday's system performed well.

Most importantly, this gives the security organization an opportunity to stop spending nearly all of its energy managing the artifacts of risk and gain enough leverage to continuously manage risk itself.

The AI-native SDLC is going to require a different kind of AppSec practitioner and a different kind of AppSec program. We should start preparing for both. If everyone else in the SDLC is going to manage fleets of agents, security will too.

Maintaining Markdown isn't going to cut it.

Book a demo to see DryRun Security in action and learn how your team can verify real risk, guide remediation, and secure every code change.