For more than a decade, security awareness training has been marketed as the antidote to human error. It has been packaged as the cultural reset every company needs – gamified, engaging content that will turn ordinary employees into the first line of cyber defense. The demand is real. Global cyber losses are rising, attacks are accelerating, and phishing remains the single most common cause of compromise.
Yet beneath the polished marketing campaigns and upbeat dashboards lies an uncomfortable truth. According to industry veteran Cary Johnson, founder of Phishbusters and one of the most experienced phishing simulation specialists in North America, the cyber awareness industry is built on a measurement model that no longer deserves the trust of security leaders.
It is a bold claim and one that challenges the foundation of a sector now worth billions. But Johnson argues that the problem is simple: the people selling cyber awareness solutions are the same people defining how success is measured. Then they self-assess their own performance.
No one would accept that in any other corner of cybersecurity. You would never ask your MSSP to perform your SOC 2 audit. You would never let your endpoint vendor run your penetration test. Yet, for nearly 20 years, organizations have accepted vendor-supplied metrics as proof of success in human risk reduction. The result is a credibility crisis that is quietly shaping boardroom conversations across Canada. Security leaders say it out loud more often than vendors care to admit.
But does it work?
Johnson believes the question keeps coming back because the industry has never built a scientific framework for answering it.
A model built on activity rather than impact
At the core of Johnson’s critique is what he calls the activity bias that drives most awareness programs. For years, vendors have promoted engagement as the north star metric – more games, more training modules, more quizzes – more touchpoints that create an illusion of progress.
When people are busy, they feel productive, says Johnson, but busyness is not evidence of reduced risk. He argues that the industry has succeeded in selling entertainment but has failed to produce objective measurements that prove behavior change.
This activity-heavy model has side effects. It overloads users with well-intentioned but often outdated content sometimes described as hack lore. Alerts about public WIFI hotspots or airport charging stations may feel familiar but do little to protect against the sophisticated phishing campaigns now driving Canadian breaches. The cognitive load being placed on employees is not proportionate to the risks they face.
