Wednesday
Aug082007
Partial String Matching
Wednesday, August 8, 2007 at 5:32AM Is there any alternative to LIKE '%...%' OR LIKE '%...%' in MySQL if you have to offer partial string matching on a large dataset?
Wednesday, August 8, 2007 at 5:32AM Tuesday
Aug072007
Can you profit from the coming Content Delivery Network wars?
Tuesday, August 7, 2007 at 2:18AM Playing like the big boys may be getting cheaper. The big boys, like YouTube, farm the serving of their most popular videos to a third party CDN. A lot of people were surprised YouTube didn't serve all their content themselves, but it makes sense. It allows them to keep up with demand without a large hit for infrastructure build out, much like leasing computers instead of buying them. The problem has been CDNs are expensive. Om Malik reports in Akamai & the CDN Price Wars that may be changing. CDN service could be becoming affordable enough that you might consider using them as part of your scaling strategy. Akamai, once the clear leader in the CDN field, is facing strong competition from the likes of Limelight Networks, Level 3, Internap, CDNetworks, Panther Express and EdgeCast Networks. This commoditization may be bad for their stock prices, but it's good for website builders looking for new scaling strategies. EdgeCast, for example, passes on the cost savings when when their bandwidth costs drop. Other services lock you into fix cost contracts. So competition is good. New cheaper, faster, and easier possibilities for scaling your website are coming on line. Maybe CDNs can help you.
Related Articles
Tuesday
Aug072007
What qps should we design for in making a MySpace like site?
Tuesday, August 7, 2007 at 1:32AM We are currently building a high traffic portal like myspace. What is the qps that we have to keep in mind and develop the site so that it can be scalable as the traffic grows?
Saturday
Aug042007
Product: Cacti
Saturday, August 4, 2007 at 3:33AM Cacti is a network statistics graphing tool designed as a frontend to RRDtool's data storage and graphing functionality. It is intended to be intuitive and easy to use, as well as robust and scalable. It is generally used to graph time-series data like CPU load and bandwidth use. The frontend is written in PHP; it can handle multiple users, each with their own graph sets, so it is sometimes used by web hosting providers (especially dedicated server, virtual private server, and colocation providers) to display bandwidth statistics for their customers. It can be used to configure the data collection itself, allowing certain setups to be monitored without any manual configuration of RRDtool.
Saturday
Aug042007
Try Squid as a Reverse Proxy
Saturday, August 4, 2007 at 1:47AM This scalability strategy is brought to you by Erik Osterman: My recommendations for anyone dealing with explosive growth on a limited budget with lots of cachable content (e.g. content capable of returning valid expiration headers) is employ a reverse proxy as mentioned in this article. In the last week, we had a site get AP'd, triggering 100K unique visitors to a single IIS server in under 5 hours. It took out the IIS server. Placing a single squid infront of the server handled the entire onslaught with a max server load of 0.10 on a modest Intel IV 3Ghz. It's trivial to implement for anyone interested...
Friday
Aug032007
Running Hadoop MapReduce on Amazon EC2 and Amazon S3
Friday, August 3, 2007 at 3:00PM Excellent article on using Hadoop in Amazon's services environment to solve real problems for very little money. It's excellent because it shows how the stack works together and it actually seems like something a real human could do.
Friday
Aug032007
Scaling IMAP and POP3
Friday, August 3, 2007 at 11:43AM Just thought I'd drop a brief suggestion to anyone building a large mail system. Our solution for scaling mail pickup was to develop a sharded architecture whereby accounts are spread across a cluster of servers, each with imap/pop3 capability. Then we use a cluster of reverse proxies (Perdition) speaking to the backend imap/pop3 servers . The benefit of this approach is you can use simply use round-robin or HA loadbalancing on the perdition servers that end users connect to (e.g. admins can easily move accounts around on the backend storage servers without affecting end users). Perdition manages routing users to the appropriate backend servers and has MySQL support. What we also liked about this approach was that it had no dependency on a distributed or networked filesystem, so less chance of corruption or data consistency issues. When an individual server reaches capacity, we just off load users to a less used server. If any server goes offline, it only affects the fraction of users assigned to that server. Best, Erik Osterman
Thursday
Aug022007
Product: Mashery
Thursday, August 2, 2007 at 9:38AM All APIs are different. At its core, an API provides direct access deep into a web service (lower case - a service that is provided on the web) and turns it into a Web Service (upper case) that people can use as a building block. What makes it an API is the infrastructure that sits in front of it, attracts developers to use it, secures it from misuse and provides the metrics and management needed to turn an internal web service into a Web Service managed through an effective distribution channel, and providing strategic and/or financial benefit. While each API is different, the infrastructure I have described is consistent across virtually all of them, so it is neither economical nor effective to reinvent the wheel for each API someone wants to release. It is similar to the concept of an adserver - all websites have different content and fucntionality, but the concept of selecting and serving an ad, tracking it, and targeting it is pretty consistent across sites; as a result, there are many sites that use a handful of adserver providers. In addition to allowing companies to focus on their core business without having to build peripheral, non-core services, using a third-party service who is focused on providing that service allows you to benefit from ongoing development and enhancement, and from features that would be prohibitively expensive to build for just a single provider. As for an example? check out sites such as developer.trulia.com or developer.compete.com, our first two customers (we have many more, but I like to give props to our early adopters). In addition to documentation and community, they have developer key issuance, instant self-service developer provisioning, usage and rate throttling, and tracking. What you don't see, but our clients enjoy, is a dashboard where they can assign different access levels, rates or limits to each developer on a key-by-key basis, customize error messages and other API parameters, and see detailed reports of API usage on a developer-by-developer or overall basis. Building all of that takes time and money; we offer it as an instantly-deployable on-demand service with no up-front investment, and our customers seem to find it an excellent value. Oren Michels, CEO Mashery
Thursday
Aug022007
Multilanguage Website
Thursday, August 2, 2007 at 12:13AM Hi , someone can point me to some good resurce about how to bulid a multilanguage website ? the only resource i have found is this http://www.indiawebdevelopers.com/technology/multilanguage_support.asp thanks! p.s. great site ;)
Wednesday
Aug012007
Product: MogileFS
Wednesday, August 1, 2007 at 11:53AM MogileFS is an open source distributed filesystem. Its properties and features include: Application level, No single point of failure, Automatic file replication, Better than RAID, Flat Namespace, Shared-Nothing, No RAID required, Local filesystem agnostic.