Tackling False Positives in Email Validation
What is a false positive? In email validation, this term is used when an email address is incorrectly identified as being valid or deliverable – in other words, it is flagged as being good when it is actually bad.
False positives are dangerous for senders, marketers especially because sending messages to a bad email address can ruin a sender’s reputation and possibly even get them blacklisted. It’s best to correctly identify email addresses before sending out messages to help ensure that you don’t get penalized for sending messages to bad email addresses.
What causes false positives in Email Validation?
The DOTS Email Validation service offers real-time validation and verification of email addresses. Email verification is handled by directly communicating with an email address’ host mail server(s) via Simple Mail Transfer Protocol (SMTP). SMTP, to put it simply, provides the rules and guidelines for how mail servers and clients should communicate and behave when sending mail.
SMTP is older than the internet as we know it and it precedes the World Wide Web HTTP protocol by almost ten years. At the time of its inception its inventors probably never dreamed that it would be abused by malicious users and overwhelming spam. If they did then they probably never would have created SMTP commands like EXPN and VRFY.
These SMTP commands are considered vulnerabilities, as they are intended to list and verify user mailboxes; however, since they are seen as vulnerabilities most mail servers provide a way to disable them: some like Microsoft Exchange come with them disabled by default, and others will simply return a false positive . Even though the RFC specifically states, “EXPN and VRFY MUST return only valid domain addresses that are usable in SMTP RCPT commands”, it is not uncommon to see these commands return false positives. These are some of the reasons why the Email Validation service does not use or rely on these SMTP commands when trying to validate an email address.
The most common cause for false positives comes from servers that are configured to not reject recipient requests for an email address that does not exist. Simply put, the server will not reject a bad email address and it will instead say that the email is good. At Service Objects, we ubiquitously refer to this as a catch-all domain.
This type of behavior was commonly seen by domains that enabled catch-all or accept-all email accounts. The feature was primarily intended to be used as a way for someone to never miss an email address. Before the days of spam, when email addresses were a new concept, people didn’t want to risk losing an email message because someone forgot how to spell their mailbox or if someone accidentally mistyped it. It didn’t take long, however, before these catch-alls started getting filled with spam, making them near unusable.
However, catch-all behavior gained popularity by admins in an attempt to thwart bots from mining and spamming their users. The reasoning was likely that if a bot could not trust the results being returned by the server then the bot would be forced to move on, and the mailboxes of the domain’s users would be protected.
Unfortunately, malicious users and bots generally don’t care about catch-all behavior, and this practice instead creates other problems, such as helping spammers generate backscatter spam. Backscatter generally occurs when the recipient server does not reject a bad email address and instead bounces it back to the sender. The sender, however, is forged or spoofed by a malicious user and so the unsuspecting sender is now the victim of the unsolicited spam. Backscatter spam also leads to other issues, such as excessive bandwidth, but to not get too sidetracked we’ll perhaps dedicate a blog to backscatter spam at a later time.
Some mail servers are protected by anti-spam solutions. These solutions are sometimes included in firewall(s) or in mail server(s) or are proprietary to the mail host provider. Solutions can vary in sophistication. Most solutions will incorporate filters and blacklists to try and identify spam and spammers; however, unless the spammer is blacklisted then many of these types of solutions will not reject the bad email address – leading to a false positive. The mail server will likely also bounce the message back to the sender instead (helping to generate backscatter spam).
Not all anti-spam solutions are configured to always accept all requests, however. Some anti-spam solutions may be configured to instead temporarily reject all requests from spam-like activity. This is the opposite of false positives and can instead lead to false negatives.
Other solutions may instead temporarily act as a catch-all when they encounter spam-like activity: behaving normally and rejecting email requests at first, but then switching to catch-all mode temporarily and without warning and then eventually switching back to normal mode. The flapping in behavior can make verification difficult, because if the sender does not know what mode the recipient domain is in, then it can lead to false positives.
How Email Validation can help
According to a recent analysis from Statista, “Spam messages accounted for 56 percent of e-mail traffic in March 2019” and moreover, “China generated the largest share of unsolicited spam e-mails with 15 percent of global spam volume”. With so much spam being thrown around it is not difficult to understand why the overall tolerance for spam-like activity it is so low.
Sure, a single false-positive leading to one bad email message being bounced may not be enough to ruin your sender reputation or get you blacklisted, but for marketers who blast millions of messages for email campaigns, a false-positive here and there can quickly lead to hundreds and thousands if not tens of thousands of false positives.
With how important email communication is nowadays, and the benefits that it brings to marketers, can you afford to get blacklisted? Don’t take the chance and minimize your risk by using a service like Email Validation. Our Email Validation service is highly adept at identifying both false positives and false negatives. Our service has years of experience and data behind it to help identify various server behaviors and patterns.