Security Awareness Training: What the Evidence Says and How Mid-Market Owners and Operators Should Run It

Security Awareness Training: What the Evidence Says and How Mid-Market Owners and Operators Should Run It

2026-10-23

The takeaway: Keep security awareness training, but stop expecting it to stop phishing on its own. A study of more than 19,500 employees, which UC San Diego calls the largest of its kind to date, found that common training formats barely moved click rates. Train for a handful of specific behaviors, measure reporting instead of clicks, and put technical controls behind your people so one bad click doesn’t become a breach.

What the biggest study found

Over eight months, researchers from UC San Diego and UC San Diego Health ran a randomized controlled experiment that sent ten simulated phishing campaigns to more than 19,500 employees. The paper, by Grant Ho (University of Chicago), Ariana Mirian, Geoffrey M. Voelker (UC San Diego), and colleagues, was presented at the IEEE Symposium on Security and Privacy in May 2025 (Understanding the Efficacy of Phishing Training in Practice). Here’s what they found:

  • Annual training didn’t show up in the results. There was no significant relationship between recently completing annual awareness training and failing a phishing simulation.
  • “You clicked, here’s a lesson” training helped only a little. Users who got embedded training were 9.5% less likely to fail in relative terms, but in absolute terms the best-performing groups had only 1 to 4 percentage points lower failure rates on most lures.
  • Almost nobody engaged with it. Over half of embedded training sessions ended within 10 seconds, and fewer than 24% of users formally completed the material.
  • Most people clicked eventually. Over 56% of users clicked a phishing link at some point during the study.
  • The lure mattered more than the training. One simulated email fooled 1.82% of recipients; another fooled 30.80%. UC San Diego’s write-up says the low one was an Outlook password update and the high one was a fake vacation policy update (UC San Diego Today, September 17, 2025).

Fair caveats: this is one large healthcare organization, and the authors tested training “in their current and commonly deployed forms.” Their conclusion is that those forms are “unlikely to offer significant practical value in reducing phishing risks,” not that people can’t learn.

Why you still train anyway

Government guidance still includes training, but as one layer. The joint phishing guidance from CISA, NSA, FBI, and MS-ISAC (October 2023) tells organizations to “implement user training on social engineering and phishing attacks” and stresses “the importance of reporting” suspicious emails. It lists that alongside technical controls such as phishing-resistant multifactor authentication (MFA), DMARC email authentication, and email and DNS filtering (Phishing Guidance: Stopping the Attack Cycle at Phase One).

NIST’s updated guide, SP 800-50 Rev. 1 (published September 12, 2024), frames training as a learning program meant to “encourage behavior change as part of risk management,” complete with metrics and evaluation so you can tell whether it’s working (NIST SP 800-50r1).

And the attacks aren’t only in email. Verizon’s 2026 Data Breach Investigations Report (May 19, 2026) says social engineering by text and phone has a success rate 40% higher than traditional email phishing (Verizon 2026 DBIR announcement). A program that only drills email is training for last year’s attack.

A program built for how people actually behave

1. Teach four behaviors, not forty topics

  • Report anything suspicious, fast, with one button.
  • Confirm any payment or bank-detail change by calling a number you already have on file.
  • Never share an MFA code or approve a sign-in prompt you didn’t start.
  • Treat texts and calls asking for passwords or money exactly like suspicious email.

2. Make reporting the scoreboard

Click rates mostly measure how hard the test email was (1.82% versus 30.80% in the same organization). Track how many people report a simulated or real phish and how quickly the first report arrives. A fast report lets IT pull the message from everyone else’s inbox.

3. Keep it short and role-based

NIST’s guide calls for role-based training. Finance, HR, executive assistants, and IT admins face different lures, so give them scenarios that match their jobs, in a few minutes at a time.

4. Celebrate the report, don’t shame the click

If clicking gets someone publicly scolded, they’ll stop telling you when they click. You want the opposite.

5. Put controls behind the people

The study’s authors suggest technical countermeasures likely offer a better return, naming hardware MFA and password managers that only fill in credentials on the correct site. Add the CISA list (email filtering and DMARC), plus a payment verification rule your finance team can’t skip.

Questions to ask your IT provider

  • What’s our phishing report rate, and who triages reports, how fast?
  • Which phishing-resistant controls do we have today, and which accounts still lack them?
  • Do our simulations include text messages and phone calls?
  • How would we know if our training changed anything?

The bottom line

Training is a layer, not a wall. Keep it short, aim it at a few behaviors, measure reporting, and invest in the controls that catch what people miss.

If you want an outside view of your security program, Yellow Coop’s fractional CTO engagements cover exactly this kind of practical review. See what we do, how we engage, or get in touch.

Sources