A sad day for scrobbles everywhere

As from Yesterday I am no longer gainfully employed by Last FM. After a lot of soul searching I have decided to move on. This decision was made in part because I feel I’d gone about as far as I could and in part because the vision of the future for the MIR team is one that I couldn’t really see myself being part of.

I’ve worked at Last FM for about 3 years now. I worked in the MIR (Music Information Retrieval) team. To be fair, the name was more than slightly mis-representative of what we did and was more of a hangover from the early days, but it is also a well known discipline in the music research world.

Basically, the engineering side of Last FM is split into three main teams (four if you include the desktop client): web, data and MIR. The web team are responsible for the face of Last FM; the website and user experience. The data team are the heart of Last FM, responsible to looking after the terabytes of data Last FM manages. MIR could be considered the brains of Last FM. Our remit was to innovate; to try and come up with new ways to engage users.

Aside from that, due to the specialist skill-set of the team, we also developed most of the data-services that are critical to Last FM. The most well known (some might say infamous) is Radb (which stands for Row Accumulating DataBase). This is a data service that has one very special and specific task, to store and serve all the user scobbles in real time.

Now this may not sound like a big job but it really is. Last FM has billions of scrobbles and hundreds of thousands of users all scrobbling at the same time, 24/7. Not only do these scrobbles need to be stored in real time, they also need to be served up in real time and the data has to be stored robustly; in a way that is completely fault tolerant.

Now, I can’t go into the specific details of how Radb does this but I will say that Radb is a distributed database made up of banks. Each bank is a collection of shards and the shards in each bank mirror each other. Users are distributed over banks so that there is both redundancy and scalability.

Of course, nothing is perfect and during my time at Last FM we had our fair share of problems with user scrobbles and Radb; however, most of these problems were not directly due to the data-service (external factors, such a melting data-centres) and in all cases the services were restored to working order within 24 hours (often within an hour or so) and with no significant loss of scrobbles.

During my time at Last FM I was responsible for this service (among others). Looking after this service, both maintaining it and adding new features,  was a significant scalability challenge but it was also a lot of fun. The service is written in C++ (compliant with the 03 standard) and designed to be very efficient, and it is!

Unfortunately, it seems the future for Last FM is going to be Java. The cost of hardware nowadays is significantly cheaper than it was a few years ago and Java is a language that allows for the development of services far more rapidly than in C++. This being the case the powers that be in Last FM have decided that going forward they plan to focus mainly on Java.

Now, I can do many things and I can code in many languages but Java is not one of them. For that reason (and some others), following discussions with the management team, by mutual agreement I have now accepted a voluntary redundancy package with immediate effect. Of course, this does mean I now get a nice purple (my favorite colour) “Alumna” badge on my Last FM avatar rather than the rather ostentatious red “Staff” badge – every cloud has a silver lining!

My parting from Last FM was very amicable and I’d like to wish them and all my ex-colleagues all the very best. I’m sure we’ll still stay in touch and have plenty of beer sessions at The Old Fountain. In fact, the thing I’ll miss the most will be Jill’s lasagna on a Thursday lunchtime… just kidding! I’ll miss working for Last FM and all my colleagues. I can honestly say it was one of the best places I’ve worked and I’ll continue to be a loyal user of their product.

I was fortunate to work with some very smart people and I learned a lot from my time at Last FM. It is a shame they have decided to change the direction the company is going but I understand that in these hard times it is necessary for companies to make decisions that are not always popular (with staff or customers) but sometimes things just are what they are.

My currently plans are to take a month out (it is nearly xmas and I go on vacation in a couple of weeks), rest up and decide what I want to do next. I guess I’ll have to update my CV (watch this space, it’ll be uploaded here soon) and then join the millions of other unemployed looking for a new job. Interesting but exciting times ahead. I’m slightly scared of what the future holds but at the same time looking forward to a new challenge with a new company.

In the meantime  I’d like to wish both Last FM and all the staff the very best for the future and I look forward to seeing you all at the next office party 🙂