Posts Tagged ‘Email Addresses’

A Brief Look at the Journey of an Email Message

How do emails actually get from point A to point B?

Most people have no idea. They don’t stop to wonder how an email message is sent and delivered when clicking the send button. If you were to ask them they might reply with,  “Of course, it’s simple. It gets sent to the cloud.”

But an answer like “the cloud” is only part of the overall journey. For most, how an email address is sent is simply techno-wizardry magic, and the details don’t really matter to most people. It only matters that the email message is delivered, and how that happens is not important to them. The purpose of this article is to show you what really happens with an email message.

How Email Works

Details matter to us, but we understand that not everyone likes or needs to get bogged down by them. If you were to search for “How Email Works” then you would find a variety of articles ranging in detail and complexity. The path of how an email message is sent is often summarized in graphics and flow diagrams, with one of the simplest ones shown below.

This diagram, which reads left-to-right, shows the journey of email messages broken into five parts. Please keep in mind that it is not complete and that it fails to cover a couple of scenarios, but overall it is a good starting point. We will add some more details and cover some common scenarios as we move along.

  1. To start, a sender sends an email message. The sender will do this using an email client, which is sometimes referred to as a Mail User Agent (MUA). This can be an application running on the sender’s desktop or mobile device.
  2. The Sender’s MUA will establish an SMTP connection with their Mail Server, and that server will, in turn, relay the message to the recipient Mail Server.
  3. The Sender’s Mail Server then queries the Recipient’s Mail Server and routes a connection to it.
  4. The Sender’s Mail Server establishes a connection with the Recipient’s Mail Server and transmits the email message.
  5. The Recipient Mail server receives and stores the email message so that the recipient MUA may retrieve it via POP or IMAP.

These five steps are about as basic as one can get in describing the journey of an email message. The main takeaway from this flow diagram and others like it is that it establishes that communication is occurring between mail servers. Mail servers communicate using the Simple Mail Transfer Protocol (SMTP). This protocol dictates how mail servers are supposed to behave and send messages to one another.

However, not all mail servers are entirely compliant. Large service providers such as Microsoft, Yahoo and Gmail will likely have their own specialized systems in place for handling internal message communication over their vast networks. Even small businesses with only a few or even just one mail server may configure their networks and mail server(s) in a way that is not fully compliant. However, when it comes to external communication between mail servers, they are not expected to stray too far off from SMTP, as this can easily cause a breakdown in communication.

The Mail Server

Mail servers host the environment for one or more domains as well as the user mailboxes for those domains. They are responsible for sending and receiving email messages, and they are often made up of multiple agents that work together in order to accomplish this:

  • Mail Transfer Agent (MTA) – Is responsible for transferring email messages, often referred to as the either the SMTP Server, Mail Exchange or Mail Relay. Commonly communicates over port 25.
  • Mail Submission Agent (MSA) – Is responsible for communicating with the MUA and handing the email messages to the MTA for delivery. Commonly communicates over port 587.
  • Mail Delivery Agent (MDA) – Is responsible for delivering email messages to the local recipient’s mailbox and is also known as the Local Delivery Agent (LDA).

These agents work together to receive and deliver email messages for their users. Collectively they are considered the Mail Server, but some MTAs will include MDAs and some of the functions of the MSA. Therefore, it is not uncommon for MTAs, in general, to be referred to as the Mail Server.

Local Mail Delivery

In the diagram above we see the path that an email message takes when it is sent to a recipient with the same domain as the sender.

This sender still sends an email message using an MUA that connects to the mail server. Standard connection ports are port 587 and port 25; however, some mail servers may specify their own particular ports and/or require an encrypted connection for security. The MUA connects with the MSA to authenticate the Sender as a user of one of the domains that are hosted on the mail server.

The MSA then receives the email message from the Sender MUA and checks that it conforms to enforced policies. It then delivers the message to the MTA for transmission. Since the recipient domain is the same as the sender, the MTA does not have to communicate with another MTA. Instead, the MDA will handle the delivery to the recipient’s mailbox, where the Recipient MUA is then able to retrieve the message.

Sending External Mail

External mail delivery works similarly to local mail delivery. The sender MUA connects to its mail server where it is authenticated by the MSA. The MSA then receives the email message and passes it on to the MTA.

This is the point where external mail transmission will traditionally differ from local mail delivery. Instead of the MDA delivering the email message, it will be the Sender MTA that will transmit the message to the recipient MTA. It typically does this by querying a Domain Name Server (DNS) for a domain’s Mail Exchange (MX) records. An MX record is used to identify a domain’s mail server. Some domains may have more than one MX record and if so, then multiple records will be returned. Each record is assigned a preference number to prioritize which mail server would be preferred for a connection.

Receiving External Mail

When an MTA receives external mail, the overall process is generally the same as local mail delivery with the exception that the MSA is omitted from the process. This is because the MSA is responsible for communicating with the MUA only. When receiving external mail, the messages are coming from another MTA, commonly on port 25.

Some mail servers and networks are protected by firewalls and anti-spam systems.

Protected Networks

This diagram is an example of what external mail from another MTA to a protected system may look like. In this case a firewall with built-in spam protection.

Firewalls can protect mail servers in various ways. Firewalls are commonly used to block unwanted IP connections and often make use of various blacklists to do so. Ones with built-in spam protection often check for known spammers, and some will inspect the contents of the email message for known malicious links, domains, attachments and various other embedded objects. Spamming techniques are becoming more sophisticated and problematic. Therefore, protection methods are also evolving and becoming more sophisticated as well.

Microsoft, for instance, has developed its own protection service called Exchange Online Protection, and describes it as such, “Microsoft Exchange Online Protection (EOP) is a cloud-based email filtering service that helps protect your organization against spam and malware, and includes features to safeguard your organization from messaging-policy violations.”

The Microsoft EOP Overview page does a good job of explaining how their system works and is comparable to what one can expect from protected mail servers. In their case, they use three distinct levels of filtering: first, a check of the sender’s reputation, followed by subsequent checks against common mail flow rules, as well as, content often found in spam. Messages that pass all three levels of checks are then delivered.

How EOP Works

Piecing it Together

Now that we have gone over some of the basics of email delivery and reception, let’s put the pieces together.

Comparing this diagram with the five-step diagram at the beginning, we have:

  1. Sender MUA sending the email message.
  2. Mail Server, which is comprised of the MSA and MTA.
  3. DNS and Internet Cloud.
  4. Firewall/Anti-Spam protection and Mail Server, which is comprised of the MTA and MDA.
  5. Recipient MUA retrieving the email message.

Again, this is a very basic look at how email is sent and received. We have not gone into details for each step or the underlying protocols, we have barely mentioned security and encryption, and we have not gone over any of the large-scale email service provider systems. So, while the path of sending an email message can essentially be broken into five steps, the overall journey is actually much longer and more complicated. There is a lot that can go wrong, and if you are working on any kind of email marketing campaigns, you want to make sure that you use a validation system that knows its way around.

Many emails flying into a trash bin

Identifying Disposable Email Addresses: A Better Approach

Disposable email addresses – also known as burner emails, throwaway emails, temporary emails or fake emails – are commonly touted as a useful tool for keeping one’s personal or business email address private and clean of spam. Not to be confused with alias email addresses (which generally forward to a primary email address, and are therefore more likely to be read), there are different types of disposable email addresses, and they can work in a variety of ways.

In general, a user will submit a disposable email address instead of their real one, which in theory should help keep one’s own email protected from spam without their primary email and/or private data being exposed. (Note that we say “should”: there are some unscrupulous disposable email providers out there, so as with all things concerning the internet, users must be careful.)

Disposable email addresses may sound great for end users, but they can be problematic for legitimate businesses and marketers. One could easily argue that disposables are successfully doing their job when it prevents a marketer from emailing an end user, but this also means that businesses are forced to adapt their marketing strategies. One such strategy: trying to identify these disposable email addresses up front, to have a more accurate view of your email marketing assets.

A simple (but flawed) strategy: email lists

Disposable email addresses are commonly identified by static lists. There are many online communities that pool together their own lists of known disposable domains and email addresses. However, static lists are a poor long-term solution, as they can quickly become stagnant. Some communities do their best to keep their lists up to date, but there are still many potential problems with this strategy:

  • Lists often lack standardization, which can lead to implementation issues. There are many disposable services available worldwide, and some community driven lists and solutions are dedicated to just a single disposable service.
  • These lists frequently contain legitimate records for domains and addresses that are not disposable.
  • In order for a disposable to make it on to a list it first needs to be reported. By the time that happens, and the data makes it way into a solution, the list may already be partially outdated. Moreover, disposables frequently change and not all disposables are reported.
  • Using a list strategy requires constant vigilance. It’s trouble enough staying up to date on just one disposable service, but trying to stay on top of multiple others as well as new ones as they pop up is often a losing battle.

Lists of disposable email addresses are a reactionary solution at best. Worse, they only scratch the surface of the problem. Disposables are constantly changing, with new ones appearing and old ones disappearing all the time. It is impractical to rely on a simple list strategy to try and successfully identify a disposable.

A better approach: organic data aggregation

At Service Objects we like to look beyond simple lists. Instead of looking at one list to perform a simple straightforward disposable lookup, we take advantage of our wealth of data and our years of experience to not only dig deeper, but to also cast a wider net. Our email validation service doesn’t just look at lists, it looks at the whole picture as well as the nitty-gritty.

We observe various behavior patterns to better identify specific activities and ties to these activities, not just for disposables but for a variety of email types – malicious or otherwise. This allows us to assign values to these activities and even compare them against other activities. Using complex algorithms along with machine learning we can intelligently determine if a value is directly or indirectly related to a particular issue, such as being a disposable address.

As sophisticated as this solution is, note that we won’t always be able to successfully identify a disposable address. Sometimes all the variables don’t match up just right, and sometimes there just isn’t enough data. However, the service will still often be able to identify such email address as being malicious or potentially malicious, in which case you would likely want to reject the email address anyway.

The sophisticated solution

Disposable email addresses are a real headache for businesses and marketers. As with most things regarding email addresses, they are a much more complicated problem than one would normally think. A problem that requires more than a simple list as a solution. They call for a sophisticated solution.

Our DOTS Email Address Validation service keeps tabs on millions of domains. It monitors various behavior patterns and leverages multiple sets of data. As domains and data continue to grow, so does the service – becoming smarter and better. The service can adapt to the constantly changing disposables, making it better suited to identify them as they pop up. Not because it’s trying to keep up with them, but because it’s anticipating them.