Tuesday
Feb192008
Building a email communication system
Tuesday, February 19, 2008 at 11:31PM
hi,
the website i work for is looking to build a email system that can handle a fair few emails (up to a hundred thousand a day).
These comprise emails like registration emails, newsletters, lots of user triggered emails and overnight emails.
At present we queue them in SQL and feed them into an smtp server on one of our web servers when the queue drops below a certain level.
this has caused our mail system to crash as well as hammer our DB server (shared!!!).
We have got an architecture of what we want to build but thought there might be something we could buy off the shelf that allowed us to keep templated emails, lists of recipients, schedule sends etc and report on it. We can't find anything
What do big websites like amazon etc use or people a little smaller but who still send loads of mail (flickr, ebuyer, or other ecommerce sites)
Cheers
tarqs
the website i work for is looking to build a email system that can handle a fair few emails (up to a hundred thousand a day).
These comprise emails like registration emails, newsletters, lots of user triggered emails and overnight emails.
At present we queue them in SQL and feed them into an smtp server on one of our web servers when the queue drops below a certain level.
this has caused our mail system to crash as well as hammer our DB server (shared!!!).
We have got an architecture of what we want to build but thought there might be something we could buy off the shelf that allowed us to keep templated emails, lists of recipients, schedule sends etc and report on it. We can't find anything
What do big websites like amazon etc use or people a little smaller but who still send loads of mail (flickr, ebuyer, or other ecommerce sites)
Cheers
tarqs
Reader Comments (4)
There was a good Ycombinator thread on this a while back: http://news.ycombinator.com/item?id=92022.
There are dozens of ESPs that can help with that. For the volume you're talking about you'll need to talk to a tier-1 player like Silverpop or YesMail! Both have good APIs for sending templated messages, transactional messages etc.
For 100% in house look at StrongMail. If your issue is the outbound MTA, look at Port25's PMTA product.
Hope this helps,
Chris
Here's a suggestion from http://developer.amazonwebservices.com/connect/message.jspa?messageID=76475:
I use authsmtp and I'm very happy with the service. I use postfix locally, configured to send all mail through the gateway so that the mail is sent more quickly from my web app.
http://authsmtp.com/
You may want to check out http://www.cloudemail101.org
It's a useful and unbiased overview of the cloud email space, including some key considerations and gotchas for migrating to the cloud. Despite the technology behind the Cloud, I am seeing more and more adoption of SaaS/cloud computing across all segments. Not surprisningly, many organizations are starting with email (as well as ancillary services like email archiving, anti-spam/anti-virus protection, and encryption) which makes perfect sense for a cloud-based model.
-Clark