Expert Summary
- AI has significantly lowered the skill floor for cyberattacks — phishing campaigns that previously required native language fluency and social engineering expertise are now accessible to less-skilled attackers using LLMs.
- Business email compromise (BEC) losses exceeded $3 billion in 2025 (FBI IC3 report), with AI-generated content cited as a contributing factor in the increasing sophistication of social engineering.
- The most effective defensive applications of AI in cybersecurity are anomaly detection at scale and automated vulnerability discovery — the same capabilities attackers are using offensively.
AI has fundamentally changed the cybersecurity threat landscape in both directions — lowering the cost and skill requirement for sophisticated attacks while also enabling defensive tools that operate at previously impossible scale. Organizations that don't understand this shift are underestimating their risk exposure.
AI-Powered Offensive Threats in 2026
Hyper-Personalized Phishing
Traditional phishing relied on volume and generic lures. LLMs have enabled attackers to:
- Scrape LinkedIn, Twitter, company websites, and public databases to build target profiles
- Generate personalized phishing emails that reference the target's recent projects, colleagues by name, specific company events, or financial situations
- Produce grammatically flawless content in any language, eliminating the spelling errors that previously helped users identify phishing
The Verizon 2025 Data Breach Investigations Report found that phishing emails generated or refined by AI had a 2.7× higher click-through rate than traditional bulk phishing in A/B test environments studied.
Deepfake Audio: Business Email Compromise at Scale
Deepfake audio voice cloning has become a primary vector for business email compromise:
How it works:
- Attacker gathers 10–30 seconds of target executive's voice from earnings calls, conference presentations, or YouTube
- Voice cloning model (ElevenLabs, many open-source alternatives) generates realistic audio in the target's voice
- Attacker calls finance or HR team impersonating the executive, requesting urgent action (wire transfer, password reset, vendor payment)
Real incidents:
- 2024: Hong Kong finance worker sent $25 million after deepfake video call impersonating company CFO
- 2025: Multiple Fortune 500 companies reported deepfake audio BEC incidents; FBI estimates $2.4 billion in deepfake-facilitated losses in 2025
Why it works: Finance teams are trained to be responsive to executive requests. The urgency framing ("I'm in a meeting, I need this done now") bypasses deliberation.
AI-Assisted Vulnerability Discovery
Attackers are using LLMs trained on vulnerability databases (CVE lists, exploit code repositories) to:
- Analyze public codebases for similar patterns to known vulnerabilities
- Generate proof-of-concept exploit code for newly disclosed CVEs faster than defenders can patch
- Discover configuration errors and exposed API endpoints at scale
Google Project Zero's 2025 report noted a significant increase in exploit code quality and speed of exploit development following public release of code-capable LLMs.
AI-Generated Malware
Malware that generates polymorphic code variants to evade signature detection is not new — but LLMs have accelerated this capability. AI-generated malware can:
- Generate novel code that behaves identically to known malware but has no matching signatures
- Adapt obfuscation strategies in response to detection attempts
- Generate convincing lures and social engineering scripts as part of the malware deployment package
AI-Powered Defensive Tools
The same AI capabilities that empower attackers also enable more powerful defenses:
Behavioral Anomaly Detection at Scale
Traditional security tools use rule-based detection — known bad signatures. AI-based tools learn normal behavior patterns and detect anomalies:
Email security (Abnormal Security, Darktrace for Email):
- Establish behavioral baseline for each user's email patterns
- Detect anomalies: unusual sending time, unusual recipients, unusual wire transfer request patterns
- Detect account compromise earlier than traditional tools by identifying behavior change before obvious malicious actions
Network detection (Darktrace, Vectra):
- Learn normal network traffic patterns
- Detect lateral movement, exfiltration attempts, and command-and-control traffic that evades signature detection
- Reduce mean time to detect from 200+ days (industry average) to hours in best-case deployments
AI-Assisted Penetration Testing
Defensive teams are using AI for:
- Automated attack surface mapping (discovering internet-exposed assets continuously)
- Vulnerability prioritization (EPSS scores, reachability analysis)
- Security code review at scale — LLMs can review codebases for security patterns faster than human reviewers
GitHub Advanced Security and Snyk Code AI now incorporate LLM-assisted code analysis to identify security vulnerabilities in pull requests before they reach production.
The Human Factor Remains the Dominant Vulnerability
AI attacks are sophisticated, but the dominant success factor in most breaches remains human behavior:
- Verizon 2025 DBIR: 68% of breaches involved a human element (phishing, credential theft, error)
- Password reuse: AI credential stuffing tools test stolen credentials against hundreds of services automatically — password reuse remains catastrophically common
- Social engineering: Even sophisticated technical controls are bypassed when an employee is manipulated into handing over credentials
The training gap: Most security awareness programs still focus on 2020-era threats (Nigerian prince emails, obvious phishing). Organizations need training specifically addressing AI-generated phishing and deepfake BEC scenarios.
Practical Defense Priorities
Priority 1: Wire transfer and financial authorization Implement dual-authorization for wire transfers above a threshold. Never approve financial transactions based on a single phone call, regardless of urgency. Implement a call-back procedure on known, pre-registered numbers.
Priority 2: Email authentication Enforce DMARC (with p=quarantine or p=reject policy), DKIM signing, and SPF on all outbound email. This prevents attackers from spoofing your domain in phishing attacks on your partners and customers.
Priority 3: MFA everywhere Multi-factor authentication on all accounts — especially email, VPN, and financial systems. Phishing-resistant MFA (FIDO2/passkeys) is significantly stronger than SMS-based 2FA.
Priority 4: AI-aware security training Update security awareness training to include AI phishing recognition, deepfake audio BEC scenarios, and response protocols for urgent executive requests.
Priority 5: AI-powered email security Behavioral AI email security tools are now cost-competitive with traditional secure email gateways for organizations of any size. Abnormal Security, Proofpoint Nexus, and Darktrace Email are the leading options.
Cyber insurance for small business: what it covers and what underwriters require →
How is AI being used to attack organizations in 2026?
The primary offensive AI applications are hyper-personalized phishing (LLMs generating convincing emails using public data), deepfake audio for BEC (fake executive voice calls authorizing wire transfers), automated vulnerability scanning, and AI-generated malware that evades signature detection by generating novel code variants.
What is the biggest AI-enabled cybersecurity threat for businesses?
Business email compromise enhanced by deepfake audio is the highest-impact threat. Attackers clone executive voices from public recordings then call finance teams requesting urgent wire transfers. Losses per incident range from $50,000 to $5+ million. The FBI identified BEC as the highest-dollar cybercrime category for the fifth consecutive year in 2025.
What should companies do to defend against AI-powered cyberattacks?
The four highest-priority defenses are: (1) multi-person verification for wire transfers, (2) security awareness training covering AI phishing and deepfake BEC, (3) email authentication enforcement (DMARC, DKIM, SPF), and (4) AI-powered email security tools that detect anomalous communication patterns rather than relying on signature matching.
