Monday
Feb182008
How to deal with an I/O bottleneck to disk?
Monday, February 18, 2008 at 1:19AM
A site I'm working with has an I/O bottleneck.
They're using a static server to deliver all of the pictures/video content/zip downloads ecetera but now that the bandwith out of that server is approaching 50Mbit/second the latency on serving small files has increased to become unacceptable.
I'm curious how other people have dealt with this situation.
Seperating into two different servers would require a significant change to the sites architecutre (because the premise is that all uploads go into one server, all subdirectorie are created in one directory, etc.) and may not really solve the problem.
They're using a static server to deliver all of the pictures/video content/zip downloads ecetera but now that the bandwith out of that server is approaching 50Mbit/second the latency on serving small files has increased to become unacceptable.
I'm curious how other people have dealt with this situation.
Seperating into two different servers would require a significant change to the sites architecutre (because the premise is that all uploads go into one server, all subdirectorie are created in one directory, etc.) and may not really solve the problem.
Reader Comments (1)
The only option you have if you're unable to reengineer your solution so that it uses less I/O is to add more of it. There are a number of things you can do to add I/O capacity and keep all you data in a single sub-directory. You could:
- Buy new disks with better I/O performance and add more disk controllers.
- Add more disks to the server.
- Look at the way your RAID set is configured.
- Make sure that OS, logfiles and filestorage is running on different disk controllers.
- Attach a SAN (This would of course be the most expensive solution but configured correctly it would also be able to provide you with a massive amount of I/O compared to what a couple of local disks can do.)