Scaling IMAP and POP3

Just thought I'd drop a brief suggestion to anyone building a large mail system. Our solution for scaling mail pickup was to develop a sharded architecture whereby accounts are spread across a cluster of servers, each with imap/pop3 capability. Then we use a cluster of reverse proxies (Perdition) speaking to the backend imap/pop3 servers . The benefit of this approach is you can use simply use round-robin or HA loadbalancing on the perdition servers that end users connect to (e.g. admins can easily move accounts around on the backend storage servers without affecting end users). Perdition manages routing users to the appropriate backend servers and has MySQL support. What we also liked about this approach was that it had no dependency on a distributed or networked filesystem, so less chance of corruption or data consistency issues. When an individual server reaches capacity, we just off load users to a less used server. If any server goes offline, it only affects the fraction of users assigned to that server.


Best,

Erik Osterman

Comments

[Serge!]

As a load balancer Perdition sounds nice, however, how do you handle availability issues? E.g. if one of your "storage" mail servers explodes. Perhaps users will complain if they've lost their mailboxes.

[Pedro]

I have scaled POP3/IMAP by using IPVS and a cluster of servers running a POP3/IMAP server.
User accounts are on a LDAP server and storage is on a NFS.
IPVS takes care about load balancing (all servers receive same number of connections) and is fault-tolerant.

Its easy to scale because servers are Xen paravirtualized machines, and setting up a server are as easy as deploy a new disk image.

But perdition is a good product with useful features too :)

[U K Achary]

Hi
I am trying use Nginx as a proxy POP3 server.
My goal is use it as follows
Pop3/web client <-> nginx_proxy_pop3_server <-> Actual POP3 server.

Can somebody please provide a example proxy POP3 configuration for Nginx.

I really appreciate any help in this regard.

Thanks and regards

Achary

[atif.ghaffar]

Achary,

you can find all from the nginx wiki.
http://wiki.codemongers.com/NginxImapProxyExample

Authentication example with a php app
http://wiki.codemongers.com/NginxImapAuthenticateWithApachePhpScript

And another with embedded perl.
http://wiki.codemongers.com/NginxImapAuthenticateWithEmbeddedPerlScript