Researchers have tested the effectiveness of various phishing training courses in a study involving 19,000 test subjects. The result: training courses help less than expected.
A large-scale study on the effectiveness of phishing training has shown that conventional training methods are less effective than often assumed. Researchers at the University of California San Diego tested various training approaches over eight months with more than 19,000 test subjects from the healthcare sector.
Scientific approach instead of laboratory tests
The study took a different approach from previous investigations, which often took place under controlled laboratory conditions. Christian Dameff from the UC San Diego Center for Healthcare Cybersecurity explained that they wanted scientifically sound results from the real working world. Previous studies on phishing prevention have shown contradictory results – from significant improvements to no measurable effect.
The participants were divided into five groups and received different feedback during simulated phishing attacks: from simple error pages to static information pages and interactive, context-related training modules.
Slight improvements measurable
The result is sobering. The various training approaches led to an average improvement of 1.7 percent compared to the control group. Researcher Ariana Mirian, Senior Security Researcher at Censys and PhD student at the University of California in San Diego where the study was conducted, questioned whether all this focus on training was worth it given the results.
The secondary findings of the study were also interesting. The choice of phishing bait proved to be a decisive factor: while only a few percent fell for fake Outlook messages, around 30 percent clicked on supposed information on vacation regulations or dress codes.
Develop realistic expectations
The long-term study also revealed that around half of all participants fell for a phishing attempt at least once over the eight-month period. Mirian emphasized that it is important to learn from these mistakes instead of punishing them.
Another finding: many users skipped the training content completely or completed it so quickly that there was hardly any time for it to have a learning effect. This points to the need to make training courses more attractive and user-friendly.
The researchers do not see their findings as an argument against phishing training, but as an impetus for improvement. Dameff stated that more empirical research and data-based approaches are needed. Companies should demand concrete proof of effectiveness from training providers.
Multi-level security strategies in demand
Experts recommend a holistic approach anyway: In addition to optimized training formats, companies should rely on automated protection solutions, regular security updates and an open error culture.
The findings could help to make training programs more targeted and correct unrealistic expectations. For companies, the results do not automatically mean doing without phishing training, but u