GAMIFICATION.COM - The Official Blog of Bunchball

WorkWithKyle.com - “Queuing” - #gamification #server #engineering

Cross-posted from WorkWithKyle.com - where Bunchball engineer Kyle Vigen is blogging about Bunchball’s server infrastructure.

When we left off last time I was describing our philosophy of performing pre-computation at write time to improve our read performance. At first we applied this approach only to leader boards, but as we added new features we continuously incorporated them into this framework, and it wasn’t long before we were doing quite a bit of work at write time - updating leader boards, news feeds, the data warehouse, etc… So, in solving our read response time problem, we were gradually creating a write response time problem. We needed to figure out a way to do the pre-computation without sacrificing write performance:

Parallelization:

One option was parallelization. Updating leader boards, adding to the news feed, etc… could all be done independently. So, we could modularize the work into a handful of self-contained “operations”, allowing us to realize significant performance improvements simply by kicking off a thread for each operation. This solution had a lot of nice properties: 

1. It would be relatively easy to implement 
2. It would reduce our response time to the response time of the longest operation. 

Still, we knew that parallelization is never a panacea, and we would have to balance its benefits with the downsides: the possibility of bizarre synchronization bugs and increased complexity. Luckily, since we chose independent units of work and relied, primarily, on the underlying database for synchronization, we were able to sidestep most of the risks. So, after a little internal discussion, we decided to throw together a simple multi-threading implementation.

This solution reaped huge benefits, but it didn’t handle a couple scenarios very well, notably:

1. Traffic spikes would cause us to spin off a ton of threads, and could overwhelm our servers
2. If any of the individual operations failed, we had to roll them all back
3. Though we sidestepped most of the new synchronization issues, lock contention increased

So, we decided to complement parallelization with another classic software paradigm, queuing.

Queuing:

Queuing, as I’m sure you know, is a simple concept: instead of doing work immediately you put in a queue and do it “when you can”. And though it isn’t applicable in all situations, it’s widely used for both time-insensitive and bulk operations.

There are a fair number of open source queuing solutions available, but after evaluating the options, we decided to build our own in house because we we wanted something simple that we had complete control over. We’re currently improving our model to make it more robust and flexible so I’ll hold off on the details until we’ve finished, but for now it suffices to say that I view it as probably the most important component in our back-end architecture - the paradigm that powers many of our most complex features.