Tuesday
Apr222008
Simple NFS failover solution with symbolic link?
Tuesday, April 22, 2008 at 8:52PM
I've been trying to find a high availability file storage solution without success. I tried GlusterFS which looks very promising but experienced problems with stability and don't want something I can't easily control and rely on. Other solutions are too complicated or have a SPOF.
So I'm thinking of the following setup:
Two NFS servers, a primary and a warm backup. The primary server will be rsynced with the warm backup every minute or two. I can do it so frequently as a PHP script will know which directories have changed recently from a database and only rsync those. Both servers will be NFS mounted on a cluster of web servers as /mnt/nfs-primary (sym linked as /home/websites) and /mnt/nfs-backup.
I'll then use Ucarp (http://www.ucarp.org/project/ucarp) to monitor both NFS servers availability every couple of seconds and when one goes down, the Ucarp up script will be set to change the symbolic link on all web servers for the /home/websites dir from /mnt/nfs-primary to /mnt/nfs-backup
The rsync script will then switch and the backup NFS will become primary and backup to the previous primary when it gets back online.
Can it really be this simple or am I missing something? Just setting up a trial system now but would be interested in feedback. :)
Also, I can't find out whether it's best to use NFS V3 or V4 these days?
So I'm thinking of the following setup:
Two NFS servers, a primary and a warm backup. The primary server will be rsynced with the warm backup every minute or two. I can do it so frequently as a PHP script will know which directories have changed recently from a database and only rsync those. Both servers will be NFS mounted on a cluster of web servers as /mnt/nfs-primary (sym linked as /home/websites) and /mnt/nfs-backup.
I'll then use Ucarp (http://www.ucarp.org/project/ucarp) to monitor both NFS servers availability every couple of seconds and when one goes down, the Ucarp up script will be set to change the symbolic link on all web servers for the /home/websites dir from /mnt/nfs-primary to /mnt/nfs-backup
The rsync script will then switch and the backup NFS will become primary and backup to the previous primary when it gets back online.
Can it really be this simple or am I missing something? Just setting up a trial system now but would be interested in feedback. :)
Also, I can't find out whether it's best to use NFS V3 or V4 these days?
Reader Comments (5)
Have you looked at DRBD + Heartbeat? This would be a much better and more reliable approach. The short, short version is that DRBD replaces your rsync script (and more) and Heartbeat replaces your Ucarp/symlink. Here is a nice guide on how to set it up:
http://howtoforge.com/high_availability_nfs_drbd_heartbeat
Thanks for your reply Dave. The thing that put me off DRBD was that if there is any corruption, apparently it gets replicated on the other drive. Plus there is a bit more flexibility with rsync, e.g. not deleting files immediately on the backup server.
I'll give it another look though I think.
Thanks my firends
I am trying to look at similar option. Did anybody tried the above symbolic Link option in production? Curious to know if this works or did anybody faced issues with rsync?
instead of rsync you could use lsyncd. it monitors for fs changes and updates copies of files almost realtime.