This morning, at around 5:00 AM Pacific Time, we experienced an unusually large number of connections to the database which stores the list of domains which are provisioned to use our inbound filtering service. This database is queried whenever a sender tries to deliver a message to our inbound mail servers. It became overloaded, and could not respond quickly to queries, which resulted in senders receiving a
451 SMTP error response.
We responded to the incident by adding additional database capacity, and restricting the number of concurrent connections that each client application could make. By 7:30 AM Pacific Time, mail traffic was flowing normally.
We are continuing to investigate the root cause of this issue. We will update this post-mortem when we have more information.