In the context of email deliverability and MTA functionality, often, one of the more frustrating aspects, is being able to handle, categorize and react to bounce messages. A bounce is a type of email reply that you receive when the remote email server is unable to deliver your email. Here we’ll discuss a few tips on why it is important to handle them, the consequences of ignoring ISP responses and some helpful ways EmailSuccess can assist you in overcoming these difficulties.
1) Bounces and Reputation
Processing bounces is important for your Deliverability.
Historically speaking bounces were meant for people to understand why a message wasn’t being delivered, so that they could fix things and resume normal communication (eg. imagine calling the other party by telephone “your system is broken, please bring it up again” “you gave me the wrong address, what was it again?”). Today things have changed.
To give you an example as to why bounces are important for your Reputation, let’s look at it from the perspective of an ISP (Internet Service Provider) and see why some sending practices are not appreciated:
- ISPs don’t like senders that oversend to non-existent or invalid email addresses. One reason for this is that you are consuming the ISP’s resources for nothing. (‘I’ve already told you the email doesn’t exist, why are you sending to it again?’’). Another reason is that the ISP may assume that you are trying to harvest email addresses by trying out a combination of usernames or attempting to send to old email addresses found on the internet.
- ISPs don’t like senders that don’t follow their rules. Rules are in place to limit abuse and ease the workload of the ISP abuse team, why would they accept your emails if you don’t follow their rules?
Perhaps the most definitive consequence to a sender’s reputation is that all the information about your sending behavior gets fed to the ISP’s antispam/reputation engine (that “learns” from it), or worse, gets you instantly added to an internal or external blacklist. Ending up on such lists can greatly impact your deliverability, with consequences ranging from inbox placement problems, to messages being marked as junk and even total rejection.
2) Processing Bounces is Hard
As mentioned before, after years of abuse, the internet has evolved into a multitude of different implementations (such as SMTP and anti-abuse systems) and bounce handling has grown into a very fragmented landscape.
Here are some fundamental truths about Bounce Handling:
- Bounce messages can be synchronous (like the majority of bounces received by an MTA) or asynchronous (the message was accepted by the remote server but after some time, there was an out of bound bounce).
- Bounces can broadly be categorized between soft bounces (4xx) or hard bounces (5xx), where hard bounces mean you should never attempt to send the exact same message (or it will fail again).
Apart from these hard facts, everything else is highly dependent on the MTA software and/or ISP on the other side of the channel.
This is because the SMTP protocol is very old and has a lot of “history”. Each user having done things differently at one point and, usually, continuing to do so over time. One classic example is greylisting technology, that was invented and adapted to the SMTP bounce system at one point in time.
The global fight against SPAM can also hinder your deliverability. As a ‘’good’’ sender you would like to know why you are being blocked (reflected by the bounces), but for the ISPs, being too open about that information means they are helping attackers circumvent their defense.
This all also means that bounce messages don’t always follow RFCs (RFC 3463 – Enhanced Mail System Status Codes) and that you’ll need to parse the full bounce email to truly understand its meaning, which of course could be based on the specific ISP.
3) What EmailSuccess can do for you
Handle bounce categories
- EmailSuccess provides a powerful categorization tool through which you can create your own ‘’bounce categories” (though a default list to start toying with is also provided).
- Each bounce category can be assigned to a macro-category (issue type), to enable ‘’business oriented’’ analytics (hard-bounce, user-bounce, blocked-bounce, technical-bounce, other-bounce).
This enables you to work both on the more technical levels (single bounce category, ie. differentiate a bounce due to “low reputation” from a bounce due to a “blacklist hit”) and on a higher level (should you want to generally track all the “block-bounces” without going deep into the categorization aspect).
Use of these ‘’business oriented’’ macrocategories are, for example, the population of a suppression list after X number of hard-bounces (address non-existent), or after Y number of user-bounces (address non reachable due to configuration error of the final user, i.e. mailbox is full).
Handle bounce patterns
- EmailSuccess enables you, just like with the category list, to create your own patterns to assign a specific bounce to a category.
- Patterns can be enabled for a specific section of the SMTP conversation (error-stage). This is useful for synchronous bounces where some errors have different meanings if received during MAIL FROM/RCPT TO SMTP conversation or, for example, after DATA.
- Patterns can be created for specific Autotuning Providers, meaning that you can have specific patterns for specific ISPs and that these patterns have precedence over the generic ones when the system is handling bounces for a particular provider.
EmailSuccess allows you to also differentiate between processing why you have received specific errors vs. how the system should respond to the errors that require immediate attention. Some bounces reported by the ISP are really important and should be acted upon immediately to improve your deliverability.
Handle outcome patterns
- Outcomes decide what EmailSuccess needs to do next with the message: fail (stop sending and parse the bounce for bounce categorization), retry (reschedule with rescheduling rules and TTLs), ande graylist (reschedule following a greylisting logic).
- Just like with bounce patterns, you can create patterns for specific Autotuning Providers.
As an example, if you are receiving a specific soft bounce (4xx) that you know means graylisting for a certain provider, you can create an outcome pattern for that provider with the outcome “greylist”. Please note that, if the sending continues to fail and the system exhausts its configured retries, the bounce will eventually be parsed by the bounce categorization engine and get categorized accordingly.
Another example is that if you are receiving a specific soft bounce (4xx) that you don’t want to retry because the provider didn’t like it, you can create an outcome pattern with the outcome “fail” to fail it immediately.
Please note that all the information regarding bounces is accessible via the EmailSuccess user interface under the single message (full history of each retry).
You can also always export any information regarding outcome and bounce pattern matching to a csv like file or to a Kafka event stream server (see EmailSuccess Collectors) for further analytics outside of EmailSuccess.