Strategy: Asynchronous Queued Virus Scanning

Wednesday, January 16, 2008 at 11:01PM
Todd Hoff in Strategy, job queue, vrius scan

Atif Ghaffar has a nice strategy to deal with virus checking uploads:

Upload item into a safe area. If necessary, the uploader blocks waiting for a result.

Queue a work order into a job system so all the work can be distributed throughout your cluster.

A service in your cluster performs the virus scan and informs the uploader of the result.

Move the vetted item into your system.

This removes the CPU bottleneck from your web servers and distributes it through your cluster. Keep your web servers providing prompt service to users. Let your cluster do the heavy lifting. This minimizes response time and maximizes throughput. A similar system can be used for creating thumbnails, transcoding, copyright checks, updating indexes, event notification or any other kind of intensive work.

Article originally appeared on (http://highscalability.com/).