How Flickr Handles Moving You to Another Shard

Colin Charles has cool picture showing Flickr's message telling him they'll need about 15 minutes to move his 11,500 images to another shard. One, that's a lot of pictures! Two, it just goes to show you don't have to make this stuff complicated. Sure, it might be nice if their infrastructure could auto-balance shards with no down time and no loss of performance, but do you really need to go to all the extra complexity? The manual system works and though Colin would probably like his service to have been up, I am sure his day will still be a pleasant one.

Comments

[john allspaw]

Colin snuck up upon the man behind the curtain! :)

The process isn't manual, actually, it is automatic. But even when auto-balancing, you need to make sure that the data isn't changed while it's being moved. Migrating accounts are for a very small time locked while they are moved, unlocked when they are done. What happened to Colin can happen when a user just happens to hit their shard at the time they are being migrated, which doesn't happen very often. I suspect that his account was done in a lot less time than 15 minutes, probably seconds at absolute most. Note that actual photos aren't being moved, just metadata. :)

That 15 minute possibility comes from the part where sometimes the migration could take longer depending on what's happening on the source or target shard at the time. Our auto-balancing shard migrations happen with feedback from the actual load on the site, so more migrations happen at low traffic periods to avoid this exact thing. :)

Colin should win some sort of prize for catching it in the act.
-john

[Dustin Puryear]

I have a feeling that by "manual" he probably meant not-migrated-on-the-fly. In other words, that it isn't a completely transparent process at Flickr. :)

Todd, good point about transparency of a move not always being needed. I think far too often people try to make too much invisible, especially when it's completely unnecessary, as with Flickr.

--
Dustin Puryear
Author, "Best Practices for Managing Linux and UNIX Servers"
http://www.puryear-it.com/pubs/linux-unix-best-practices

[chmac]

Here's a http://www.bytebot.net/blog/archives/2007/10/11/moving-to-another-shard">direct link to the post.

http://www.callum-macdonald.com/" title="Callum" target="_blank">Callum