We regret to inform you that a configuration error resulted in the outgoing email sent through the MailChannels Outbound Filtering Service between 3:20 PM UTC and 6:20 PM UTC on July 13th being inadvertently quarantined instead of delivered. The surge in load on our quarantine system caused delivery delays across the entire service.
We have now delivered all of the incorrectly quarantined emails. Messages that are released this way will not appear in the log search page on the console. If you need confirmation of delivery, please contact email@example.com.
Due to the delays introduced by this problem, some messages may have timed out of the queue. Non-delivery receipts are sent to the sender for all messages that time out this way. The timed out messages will have the following error message
"delivery temporarily suspended: conversation with
outbound-smtp-gateway.outbound.svc.cluster.local timed out
while sending end of data "
We deeply regret the issues caused by this configuration error. To ensure that the same problem does not happen again, we have expanded our monitoring of quarantine traffic to detect unusual fluctuations. We have also made changes to strengthen the resilience of our configuration change process in this part of the system to prevent human error.
Frequently Asked Questions
Did any of my email get lost silently - i.e. without the sender being notified?
A small number of emails may not have been released from the quarantine. If you need confirmation of delivery, please contact firstname.lastname@example.org.
Due to the delays introduced by this problem, some messages may have timed out of the queue. Senders will always receive a “non-delivery receipt” letting them know that one of their messages could not be delivered. The solution would be to resend these messages.
How can I be sure this won’t happen again? MailChannels has fully identified the cause of this failure and we will make changes to ensure it cannot happen again.
If you have any further questions or concerns about this, please do not hesitate to reach out to us at email@example.com