AI-powered phishing: how to spot attacks that look more real than ever

Every employee who has sat through a security awareness session in the last fifteen years was taught roughly the same checklist. Watch for bad spelling. Be suspicious of a generic “Dear Customer” greeting. Hover over links before clicking. Distrust urgent demands for money. That checklist worked because it described real, reliable properties of the phishing emails of that era – the ones written by attackers who were not fluent in English, working at volume, and not putting much craft into any single message.

That phishing still exists, and the checklist still catches it. The problem is that it is no longer the phishing that gets people into trouble. The dangerous email arriving in your staff’s inboxes today has perfect grammar, uses the recipient’s real name and real context, sounds like the person it claims to be from, and asks for something that fits plausibly into a normal workday. It does not trip a single item on the old checklist. An employee who has been trained to look for spelling mistakes will read that email, find nothing wrong, and act on it – and the training will have made them more confident, not less.

This article is the practical-defense companion to the rest of the Sequentur AI cluster. The how cybercriminals are using AI to attack small businesses article walks the full attacker playbook; this one narrows in on the single most common attack in that playbook – phishing – and answers the question a business owner actually has: if the old advice no longer works, what does my team look for now, and what should the email system be catching before a human ever sees it. It is written for SMB owners, operations managers, and in-house IT generalists who need their people and their tooling calibrated for phishing that no longer looks like phishing.

Short answer

AI-generated phishing is dangerous because it removed every surface tell that security awareness training was built around. It has no spelling or grammar errors, it is personalized with the recipient’s name and real business context, and it mimics the writing style of the person it claims to be from. The old “spot the bad email” approach fails against it because there is no longer a bad email to spot. The defense splits into two halves that have to work together. The human half stops teaching people to grade the writing and starts teaching them to react to the request: any message asking for a payment, a banking change, credentials, gift cards, or an unusual data export gets verified through a separate channel before action, no matter how clean and familiar it looks. The technical half catches what people now miss – DMARC, SPF, and DKIM to stop domain spoofing, advanced email filtering with click-time link rewriting, anti-impersonation rules, and anomaly detection that flags a message as unusual even when its contents look perfect. Neither half is sufficient alone. The rest of this article covers what AI changed, the new signals that still work, and the technical controls that do the catching.

What makes AI-generated phishing different

It is worth being precise about what actually changed, because every defensive decision below follows from it. AI did not invent phishing. It changed four specific things about it, and each one disables a piece of the traditional defense.

Traditional phishing	AI-generated phishing	Which old advice it breaks
Awkward grammar, misspellings, odd phrasing	Fluent, polished, error-free in any language	“Look for spelling and grammar mistakes”
Generic greeting – “Dear Customer”, “Dear User”	Uses the recipient’s real name, title, and team	“Watch for generic greetings”
Generic pretext, obvious mass-mail feel	Real context – active projects, real vendors, real coworker names	“Be suspicious if it does not sound relevant”
Does not sound like any specific person	Mimics a named sender’s tone and writing style from real samples	“Check whether it sounds like the sender”
One template blasted to thousands	Hundreds of individually personalized variants at the same cost	“Targeted attacks only happen to big companies”

The single most important line in that table is the last one. Personalization and scale used to be a trade-off for attackers – they could send one carefully crafted email to a CFO, or ten thousand generic ones, but not ten thousand carefully crafted ones. AI removed the trade-off. Producing a hundred personalized, polished, context-aware phishing emails is now the same amount of work as producing one. The volume went up and the quality went up at the same time, which is the combination that defenders find hardest to absorb.

The personalization is built on reconnaissance, and the reconnaissance is cheap too. LinkedIn supplies the org chart, job titles, and reporting lines. The company website and press releases supply active projects and partnerships. Social media supplies travel schedules and the names of colleagues. Old data breaches supply email formats and sometimes old passwords. An attacker can have all of this summarized into a clean profile of a specific employee – who they report to, what they are working on, which vendors they deal with, when invoices typically go out – in a couple of minutes. The phishing email is then written against that profile. When staff receive a message that references the real Henderson contract and their real Thursday deadline, it does not feel like phishing, because every signal they were trained to check comes back clean.

Why the old “check for bad grammar” advice now backfires

This deserves to be stated plainly because it is the single most common gap in SMB security awareness programs, and it is worse than just being outdated.

Training employees to equate polish with legitimacy does not merely fail against AI phishing – it actively helps the attacker. A perfectly written, well-formatted email reads as more trustworthy to someone whose mental model is “phishing looks sloppy.” The cleaner the AI-generated message, the more it confirms the staff member’s instinct that it is real. The training has inverted: the property it taught people to treat as reassuring is now the property of the most dangerous messages.

The same inversion applies to most of the rest of the legacy checklist:

• “Generic greetings are a red flag.” AI phishing opens with the recipient’s actual name and often their role. A personalized greeting now reads as a trust signal, which is exactly backwards.
• “It will not sound relevant to your work.” AI phishing references real projects, real vendors, and real colleagues pulled from public reconnaissance. Relevance is no longer evidence of legitimacy.
• “It will not sound like a real person you know.” AI mimics a named sender’s writing style from samples of their genuine emails. Sounding like the boss is no longer proof it came from the boss.
• “Be suspicious of urgency.” This one is partly intact, but weaker. AI-generated attacks now build plausible, specific reasons for the urgency – a real deadline, a real trip, a real deal – rather than generic pressure. Urgency alone is still worth noticing; urgency is no longer disqualifying on its own.

None of this means awareness training is worthless. It means the content of the training has to be rewritten. A program that still leads with “look for spelling mistakes” is calibrated to a threat that has largely been retired, and it is giving staff false confidence in a signal that now points the wrong way.

The signals that still work

If the writing quality is no longer a usable signal, staff need something to replace it. The good news is that there are signals that AI does not erase – they just sit at a different layer. They are about the request and the situation, not the prose.

The request itself is the signal. This is the central shift. Train staff to react to what is being asked, regardless of how clean or familiar the message is. A short list of request types should trigger an automatic pause, every time, from any sender:

• A payment, wire transfer, or change to banking or payment details
• A request for credentials, MFA codes, or a “re-login” through a link
• Gift card purchases – still one of the most common BEC payloads
• An unusual data export, employee records request, or tax document request (W-2 and payroll fraud spikes every January through April)
• Any deviation from the normal process – “skip the usual approval”, “keep this between us”, “I am in a meeting, just handle it”

A perfectly written email asking for one of these deserves more suspicion, not less. The polish is not reassurance; it is irrelevant. The request is what matters.

Channel and context mismatches. Even a flawless email can sit oddly in context. The boss who never emails about finance suddenly emailing about a wire. A vendor who always sends invoices as PDFs suddenly sending a payment-portal link. A request that arrives by email when that kind of request normally happens in person or by phone. A reply that appears in a real thread but subtly changes the banking details from an earlier message. AI writes the message well; it does not always know what is normal for your business. The mismatch between the request and the established pattern is a signal AI cannot fully launder.

Reply-in-thread and lookalike senders. The most convincing phishing is not a fresh email – it is a reply injected into a genuine, existing conversation, sent either from a compromised real mailbox or from a lookalike domain. Staff should be taught to glance at the actual sender address, not the display name, and to watch for near-miss domains: a swapped letter, a .co instead of .com, an extra hyphen, a rn standing in for an m. This is one of the few mechanical checks from the old era that still earns its place, because AI improves the writing but cannot change the fact that the attacker does not control your real domain – unless you have left it unprotected, which the technical section covers next.

Unexpected links and attachments. Hovering over a link still has value, but with a caveat: attackers increasingly send a clean link that redirects to a malicious page hours after delivery, which defeats a hover-check done at read time. The durable version of this lesson is not “inspect the URL” but “do not authenticate or download because an email told you to – navigate to the service yourself.” If an email says your Microsoft 365 password expired, the staff member opens a browser and goes to the portal directly rather than clicking through.

The honest framing for staff is this: you are no longer expected to spot the fake, because increasingly you cannot. You are expected to verify the request. That is a lower bar, it is achievable, and it works whether or not the message looked legitimate.

Verification: the habit that does the real work

Detection skills degrade as AI improves. Verification habits do not, because they do not depend on the employee being able to tell a real message from a fake one. This is the most important paragraph in the article: the durable defense against AI phishing is not better detection, it is a verification habit that applies regardless of whether the message looked legitimate.

A workable verification rule for a small business is short enough to fit on one page:

• Any payment, any change to banking or payment details, and any urgent financial request is verified through a second, pre-agreed channel before action. If the request came by email, verification happens by phone – on a known number from your own records, never a number supplied in the message.
• Verification is mandatory regardless of how certain the staff member is. “I recognized the writing” and “I recognized the voice” are no longer evidence of anything. Removing the judgment call is the point – the employee does not have to decide whether this particular email is suspicious, because the rule fires on the request type, not on a suspicion.
• The rule is blameless and expected. Verifying a request from the actual owner is not an insult and must never be treated as one. If staff fear looking paranoid, they will skip the step exactly when it matters.
• There is a fast, safe reporting path. When staff are unsure, reporting a message needs to be one click and carry zero risk of getting in trouble – including when they have already clicked. An employee who fears blame will stay quiet, and silence is what gives an attacker time.

This single habit is disproportionately valuable because it defeats more than email phishing. The same out-of-band verification step neutralizes AI-enhanced business email compromise, voice cloning, and deepfake fraud at once, because all three rely on a request being acted on without a second-channel check. The voice cloning and deepfakes article covers the cloned-voice version of the same attack – a phone call that sounds exactly like the CFO – and the defense is the identical verification protocol. Build the habit once and it covers the whole family of impersonation fraud.

What technical controls catch that humans now miss

Because human detection is less reliable than it used to be, the technical layer has to carry more of the load. The goal of email security is to remove as many AI-generated phishing messages as possible before a person is ever asked to judge them. None of the following controls are new, but their importance has gone up sharply now that the human backstop catches less.

Email authentication – SPF, DKIM, and DMARC. These three records do not stop your staff from receiving phishing, but they stop attackers from sending phishing that appears to come from your domain. SPF lists the mail servers allowed to send as your domain. DKIM cryptographically signs your outbound mail. DMARC ties the two together and tells receiving servers what to do with mail that fails both – monitor, quarantine, or reject – and sends you reports on who is sending under your name. Without these, anyone can spoof your domain and most receivers will deliver it. With DMARC at an enforcement policy, the lookalike-sender problem shrinks to genuinely different domains, which are easier to catch. Google and Yahoo began requiring DMARC for bulk senders in 2024, and the direction of travel is stricter, not looser.

Advanced email filtering with click-time protection. Basic filtering – the kind bundled with a mailbox by default – catches known-bad attachments and known spam sources, but it is not built for targeted phishing. Advanced filtering adds the layers that matter against AI phishing: link rewriting that re-checks a URL at the moment of click rather than only at delivery time (this is what catches the clean-link-now, malicious-later trick), attachment sandboxing that detonates files in isolation before delivery, and anti-impersonation rules that flag messages trying to look like your executives or your known partners. For a Microsoft 365 business, this is the Defender layer; the phishing attack prevention for small business article and the Microsoft 365 security hardening guide cover the specific settings, and third-party platforms can layer on top for defense in depth.

Anomaly detection. This is the control most directly aimed at the AI problem. Where signature and reputation filtering ask “does this message match something known to be bad,” anomaly detection asks “is this message unusual for this organization” – a first-time sender impersonating an internal name, a payment request that breaks the normal pattern, a login or send from an unexpected location, a reply that deviates from a thread’s established behavior. Because AI-generated phishing is specifically engineered to look legitimate, the behavioral question – is this normal for us – catches a meaningful share of what content inspection misses.

MFA on every account, no exceptions. When phishing succeeds anyway and a password is captured, MFA is what stops the password from being enough on its own. It has to be universal: one mailbox or one remote login without MFA is the gap that AI-scaled credential attacks are built to find. MFA is not invincible – adversary-in-the-middle phishing kits can relay an MFA prompt in real time – which is exactly why MFA alone is not enough and it belongs inside the layered stack rather than standing alone. Phishing-resistant methods such as passkeys and hardware security keys raise that bar considerably.

DNS filtering as the click-time safety net. Some phishing will get through filtering, and someone will click. DNS filtering checks the domain behind a clicked link against threat intelligence and blocks resolution to known-malicious or newly registered domains, showing a warning page instead of the phishing site. It protects devices on and off the corporate network, which matters for remote staff. It does not catch a phishing site on a domain not yet flagged, but it catches a real share, and it is inexpensive and fast to deploy.

The layered logic is the same as it has always been – no single control stops phishing, and what one layer misses the next one catches. What AI changed is the weighting. The human layer used to do a lot of the catching; it now does less, so the technical layers have to be in place and properly configured rather than treated as optional.

10 things small businesses get wrong about AI phishing

The recurring misconceptions and gaps:

1. Still teaching “spot the bad grammar.” Training calibrated to pre-AI phishing. Polish now reads as trustworthy – the exact wrong lesson.
2. Treating personalization as a trust signal. Your name, role, and real projects in an email mean the attacker did reconnaissance, not that the message is safe.
3. Grading the writing instead of the request. The defensible skill is reacting to what is being asked, not assessing how well it was written.
4. No out-of-band verification rule. The single habit that defeats AI phishing, BEC, and voice cloning at once – and the one most often missing.
5. Trusting the display name. AI improves the prose; it does not control your domain. The actual sender address and lookalike domains still need a glance.
6. Hover-checking links at read time. Attackers send clean links that turn malicious hours later. The durable lesson is to navigate to services directly, not to inspect URLs.
7. No DMARC at enforcement. Leaving SPF, DKIM, and DMARC unconfigured lets anyone spoof your domain to phish your own customers and staff.
8. Relying on default mailbox filtering. Basic filtering is not built for targeted phishing. Click-time link protection and anti-impersonation rules are what catch AI phishing.
9. MFA with exceptions. One non-MFA account is the gap AI-scaled credential attacks are designed to find.
10. Punishing the employee who clicks. Fear of blame produces silence, and silence gives the attacker time. Reporting has to be fast and blameless.

Time to calibrate your phishing defenses for AI

A practical sequence for a typical 20-50 person SMB updating its phishing posture for AI-generated attacks:

Phase	What happens	Time
Email authentication	Verify SPF and DKIM, move DMARC to a quarantine or reject policy, review the reports	1 week
Advanced filtering review	Confirm click-time link rewriting, attachment sandboxing, and anti-impersonation rules are on	3-5 days
MFA audit and gap closure	Confirm MFA on every mailbox and remote login, close exceptions, plan phishing-resistant methods for high-risk roles	1 week
Verification rule	Write and roll out the out-of-band verification rule for payments, banking changes, and sensitive requests	2-3 days
Awareness training refresh	Rewrite training around the request and verification habits, not grammar tells. Deliver to all staff.	1-2 weeks including delivery
Reporting path	Set up a one-click report button and confirm reporting is blameless	2-3 days
Anomaly detection and DNS filtering	Confirm behavioral email detection is active; deploy or verify DNS filtering across all devices	1 week
Simulated phishing	Start a recurring simulation program using AI-grade phishing samples, not legacy ones	Ongoing, first run within 2 weeks
Total elapsed time	From “we should look at this” to a phishing posture calibrated for AI	3-5 weeks

Most of this is calibration of controls a reasonably run business already partly has, not net-new construction. The three-week version is where the technical layers are mostly in place and the work is updating the training and the verification rule. The five-week version is where filtering, DMARC, or anomaly detection need to be built or replaced.

What is next in this content series

This article covered the practical defense against AI-generated phishing – the new signals, the verification habit, and the technical controls that catch what people now miss. The pieces around it go deeper:

• Voice cloning and deepfakes – the cloned-voice and deepfake-video version of the same impersonation attack, and the verification protocols for finance, hiring, and vendor onboarding
• How to evaluate whether an AI tool is safe for your business to use – the questions to ask before approving any AI vendor
• How to introduce AI tools to your team without creating security gaps – the controlled-rollout playbook
• What to do if an employee leaks business data through an AI tool – the incident response walkthrough

If you have not read them yet, the upstream pieces in this series are the shadow AI wake-up call, the AI acceptable use policy template, the data-side breakdown of AI vendor terms, the AI security risks overview, the AI and HIPAA guide for healthcare, the Microsoft Copilot rollout guide, the AI governance framework article, and the how cybercriminals are using AI to attack small businesses attacker-side overview.

If your phishing defenses are being managed inside a broader managed cybersecurity services engagement, the email security configuration, the anomaly detection, the training program, and the response capability all belong inside it.

How Sequentur can help

If you want help configuring email authentication, upgrading filtering and anomaly detection, rolling out a verification rule, or rebuilding security awareness training around the threats that exist now, schedule a call.