The email sent will contain a link to this article, the article title, and an article excerpt (if available). For security reasons, your IP address will also be included in the sent email.
This is a guest post by Jon Vlachogiannis. Jon is the founder and CTO of BugSense.
BugSense, is an error-reporting and quality metrics service that tracks thousand of apps every day. When mobile apps crash, BugSense helps developers pinpoint and fix the problem. The startup delivers first-class service to its customers, which include VMWare, Samsung, Skype and thousands of independent app developers. Tracking more than 200M devices requires fast, fault tolerant and cheap infrastructure.
The last six months, we’ve decided to use our BigData infrastructure, to provide the users with metrics about their apps performance and stability and let them know how the errors affect their user base and revenues.
We knew that our solution should be scalable from day one, because more than 4% of the smartphones out there, will start DDOSing us with data.
We wanted to be able to:
- Abstract the application logic and feed browsers with JSON
- Run complex algorithms on the fly
- Experiment with data, without the need of a dedicated Hadoop cluster
- Pre-process data and then store them (cutting down storage)
- Be able to handle more than 1000 concurrent request on every node
- Make “joins” in more than 125M rows per app
- Do this without spending a fortune in servers
The solution uses:
- Less than 20 large instances running on Azure
- An in-memory database
- A full blown custom LISP language written in C to implement queries, which is many times faster that having a VM (with a garbage collector) online all the time
- Erlang for communication between nodes
- Modified TCP_TIMEWAIT_LEN for an astonishing drop of 40K connections, saving on CPU, memory and TCP buffers
For more information on the BugSense architecture please keep on reading...
Return to Article