Completion vs. comprehension
Distinguish training assigned and marked complete from any evidence that comprehension or behaviour actually changed.
Independent IT security review
Check whether training changes what people do, not just what they complete.
Technical anti-phishing controls are only half the system. This review checks whether the awareness and simulation program changes real reporting behaviour, instead of producing a completion certificate nobody reads.
- Before any access
- Written question and scope
- Agreed evidence method
- No automatic tenant changes
Separate completion from behaviour change.
A 100% completion rate proves people clicked through slides. It does not prove they would recognize or report a real attempt.
Simulation click and report rates
Read simulated-phishing results over time, not one campaign, and compare click rate against report rate-not click rate alone.
Repeat clickers
Identify whether repeat clickers get a different response than a generic reminder email sent to the whole company.
Confirm simulations reflect real attempts, not last year’s template.
A simulation that no longer resembles the phishing the organization actually receives trains people to recognize the wrong thing.
Realism
Compare simulated lures against real reported and quarantined messages to check the templates still match current tactics.
Targeting
Confirm whether finance, executives, and other higher-risk roles receive scenarios matched to what actually targets them.
Fairness and trust
Review whether simulation design and consequences preserve trust in reporting, rather than teaching people that reporting gets punished.
Follow a reported message somewhere-not into a queue nobody reads.
A “report phishing” button is only a control if a person or process is actually watching what it collects.
Ownership
Identify who reviews reported messages, how quickly, and what happens after hours or on weekends.
Signal, not noise
Check whether real threat signal from reports gets separated from routine spam before it reaches whoever triages it.
Feedback loop
Confirm whether the person who reported a message ever learns whether it was real, which is what sustains future reporting.
Connect the program to the technical controls, not around them.
A training program and mail-flow protection should reinforce the same message, not contradict each other.
Consistent guidance
Check that training guidance matches what the organization’s own anti-phishing and link protections actually do.
Measured over time
Track report and click trends after each simulation round instead of judging the program on a single snapshot.
Executive visibility
Confirm leadership sees the trend, not just a vendor-provided completion score, when deciding whether the program is working.
Next step
Define the review before sharing evidence.
Start with the trigger, decision, and known boundaries. Do not send passwords, recovery codes, or tenant exports.