SOLVED: Why Phishing Email Passed Through Filters

In the modern world phishing emails are a fact of life for everyone from the richest companies to the poorest home user, and protecting against them by teaching your staff how to spot them is becoming less and less effective. Thieves are using AI tools to write perfect (often targeted called spear phishing) content and they are sending phishing emails from valid sources. This is not to suggest that a workforce frequently trained to be cautions isn’t an absolute requirement these days but it does mean that we all need to rely on backend filtering even more.

Today we received the following email:

The biggest problem with it was that we have no dealings with ray.ocn.ne.jp, but several other minor items did stand out as well:

1. The attachment name matches one of our website page titles
2. Sending the password for an attachment along with the email defeats the purpose of the password but does get the reader used to following instructions, and more importantly blocks the mail server from scanning its contents
3. The message claims it came from “Margaret Jackson”, but the warning bar (added by Microsoft), reports it came from “danna”
4. It seems less likely that a local Japanese company would have someone named “Jackson” working for them

All of these are clues something is wrong, but each is far from conclusive and can be easily explained away.

The rest of our investigation relied on the email message header, and we have a brief explanation of how to pull that HERE, if you don’t already know.

We sent the email header to the free Microsoft Message Header Analyzer (mha.azurewebsites.net) and it returned X-Original-X-MS-Exchange-Antispam-FeedType = PHISH and Phishing message = 9.25 but still allowed the email through (note that Spam Confidence Level was only equal to 1). How could this be?

Well, the domain was legitimate, the sender was legitimate and the attachment was password protected so it could not be checked by the mail server. When we say legitimate, we mean they were not being spoofed and existed without any flags like appearing on a black list. OCN for instance is Japans largest ISP and anyone, including hackers, thieves, or even bots) could set up accounts with them.

We then looked at:

1. security.microsoft.com
2. Email & collaboration
3. Policies & rules
4. Threat policies
5. Anti-phishing

The email was delivered not because it wasn’t recognized as phishing (it absolutely was, with high confidence), but because our anti-spam SCL policy didn’t catch it as general spam, and the anti-phishing policy action for phishing was set to the lowest level.

The solution to avoid this the future was to increase the PHISHING EMAIL THRESHOLD SETTING to at least 1 = Aggressive and that is what we did.

Here are the four threshold levels and what each means:

1. Standard (Default)
   • Description: This is the default setting. It takes action on phishing messages based on their detected confidence level:
     • Messages identified as very high confidence phishing will have the most severe actions applied (e.g., quarantine, or even rejection if configured)
     • Messages identified as low, medium, or high confidence phishing will have less severe actions applied, or might be delivered with a safety tip, depending on other policy configurations
   • Impact: This provides a good baseline level of protection, aiming for a balance between catching obvious phishing and minimizing false positives. It’s suitable for organizations just starting out or those with a very low tolerance for false positives
2. Aggressive
   • Description: This level increases the sensitivity of the anti-phishing filters
     • Messages that are identified as high confidence phishing are treated as if they were identified with very high confidence
   • Impact: This means that emails that are “pretty sure” phishing will be treated with the same severity as those that are “absolutely sure” phishing. This is a common and often recommended next step for organizations looking to boost their protection without becoming overly restrictive. It helps catch more sophisticated phishing attacks that might not hit the “very high confidence” mark on the default setting
3. More aggressive
   • Description: This level significantly increases sensitivity
     • Messages that are identified as medium confidence phishing or high confidence phishing are treated as if they were identified with very high confidence
   • Impact: This setting will catch an even broader range of phishing attempts, including those that might have subtle indicators. This comes with a higher likelihood of false positives, so careful monitoring of quarantined messages is crucial
4. Most aggressive
   • Description: This is the highest level of sensitivity
     • Messages that are identified as low confidence phishing, medium confidence phishing, or high confidence phishing are treated as if they were identified with very high confidence
   • Impact: This setting provides the strongest possible phishing detection, aiming to block almost any message that exhibits even the slightest phishing characteristics. It also carries the highest risk of false positives, potentially impacting legitimate email flow. This setting is usually reserved for organizations with a very high-risk profile or specific compliance requirements, and it requires vigilant monitoring and tuning

Conclusion

While it is a noble target expect all staff to always catch phishing email, it will not happen. The bad-guys are getting harder for humans to detect every day so we need to rely on anti-spam and anti-phishing filters being set to aggressive standards.