Friday
May222009
Distributed content system with bandwidth balancing
Friday, May 22, 2009 at 10:41PM
I am looking for a way to distribute files over servers in different physical locations. My main concern is that I have bandwidth limitations on each location, and wish to spread the bandwidth load evenly. Atm. I just have 1:1 copies of the files on all servers, and have the application pick a random server to serve the file as a temp fix...
It's a small video streaming service. I want to spoonfeed the stream to the client with a max bandwidth output, and support seek. At present I use php to limit the network stream, and read the file at a given offset sendt as a get parameter from the player for seek. It's psuedo streaming, but it works.
I have been looking at MogileFS, which would solve the storage part. With MogileFS I can make use of my current php solution as it supports lighttpd and apache (with mod_rewrite or similar). However I don't see how I can apply MogileFS to check for bandwidth % usage?
Any reccomendations for how I can solve this?
It's a small video streaming service. I want to spoonfeed the stream to the client with a max bandwidth output, and support seek. At present I use php to limit the network stream, and read the file at a given offset sendt as a get parameter from the player for seek. It's psuedo streaming, but it works.
I have been looking at MogileFS, which would solve the storage part. With MogileFS I can make use of my current php solution as it supports lighttpd and apache (with mod_rewrite or similar). However I don't see how I can apply MogileFS to check for bandwidth % usage?
Any reccomendations for how I can solve this?
Reader Comments (4)
Isn't MogileFS just on the disk to the server side and not the server to the client side which is what you probably need to monitor? There are many bandwidth monitors and the interface level and since it's your program feeding out the byte stream you know how much you are sending.
Another way would be to have a master server(director).
You send all the bandwidth usage to master and it can select the one with least used bandwidth.
Read the lighttpd/apache logs for the bytes send from time to time or even with tail process and send it to the master server.
We do the same to report the traffic usage and for blocking for our video service.
best regards
Atif
An example is here.
http://publisher.media-streamer.net/1306300/104/index.html
You see that the flash/js components are loaded from the publisher but the feed actually goes from another server.
MogileFS supports multiple copies of each file across servers for redundency, so when retrieving a file from the mogile tracker it selects the device with lesser IO usage. Preferably I want the tracker to return the server with lesser bw usage without having to rewrite anything in mogilefs.
Ideal case would be to have a bandwidth aware plugin for mogilefs that selects host based on bw+location in addition to IO, and could replicate most accessed files in order to gain more bandwidth / shorter travel distance. As I have the overall server capacity to serve the requests needed already, I do no wish to move to a CDN.