This is a guest repost by Rinat Abdullin, who worked on HappyPancake, the largest free dating site in Sweden. Initially written in ASP.NET and MS SQL Database server, it eventually became overly complex and expensive to scale. This is the last post in a nearly two year long series of engaging articles on the evolution of the project. For the complete list please see the end of this article.
Our project at HappyPancake completed this week. We delivered a simple and scalable foundation for the next version of largest free dating web site in Sweden (with presence in Norway and Finland).
Journey
Below is the short map of that journey. It lists technologies and approaches that we evaluated for this project. Yellow regions highlight items which made their way into the final design.
Project Deliverables
Project deliverables included:
- Deployable full-stack application with major features implemented.
- Domain model captured in software design (back-end and front-end) and a set of declarative use-cases (acting as living documentation, system outline and behavior test suite).
- Configured environments for development and continuous integration (docker container).
- Strategies for further evolution and scaling of the system.
- Code for migrating existing production deployments to a new version of software.
Final high-level architecture is simple to reason about and scale. It was designed to be that way.
Logically the entire solution consists from backend modules (represented by golang packages) and elements of Facebook Flux architecture (grouped together in namespaces by naming conventions).
Such structure helps to maintain the project as it grows in size and complexity.
This design also helps to scale the deployment to handle higher loads.
We can scale backend by:
- moving individual modules to bigger servers;
- launching multiple instances of a single module;
- switching storage of an individual module to clustered solution, moving it to bigger servers or even pushing to the cloud.
We can scale frontend by simply launching new instances behind the load balancer.
Solution structure also provides a natural way to split the work between the developers. Given the established published language (contracts of API and events), we can also bring in more developers, assigning them to work on individual backend modules or frontend namespaces.
Lessons Learned
- Picking the right technology can reduce the development effort.
- In my next project I'll try to focus even more on divide and conquer approach - isolate a small part first and then evolve it, limiting the amount of work in progress.
- It is crucial to establish feedback loop as early as possible, involving all stake-holders. This builds trust and helps to avoid surprises.
Related Articles
- On Hacker News
- The complete HappyPancake article series. Just the titles give you an indication of how the project zig zagged over time. Introduction; New Team; Language is an Implementation Detail; Moving Forward with Golang; Starting with FoundationDB; Evolving the Stack and learning Nanomsg; Designing for Throughput and Low Latency; Containers, virtualization and clusters; Benchmarking and tuning the stack; Change of Plans; Back to Basics; Messaging - Heart of a Social Site; Event-driven week; Reactive Prototype; Tactical DDD; Emergend Design Faces Reality; Minimal Viable Product; Almost Demo; Our First Demo; Scala, Modular Design and RabbitMQ, Distributing Work; Smarter Development; Delivering Features and Tests; Data, Use Cases And New Module; Back from the Vacation; Native Performance; Features, Use Cases, Rendr; Getting started with Node.js, Lazojs; Web development, the good parts; Reactive User Experience; Feeds, Chat, Online list and CSS; To ReactJS and Facebook Flux; Project Complete.