I am awestruck with Spotify’s recommendation engine – the curation works much better than Apple, Amazon and pretty much everybody else.
What’s the magic behind Spotify’s recommendation algorithm? What makes it so special? Yes, it is machine learning et al – but why haven’t others like Apple, Amazon managed to beat Spotify at curation and personal recommendations?
There is more to Spotify than just algo.
Because of “never ending playlist”, how is Spotify impacting the musical world? Learn about all this (and more) in this collection.
Spotify doesn’t actually use a single revolutionary recommendation model. Instead, they mix together some of the best strategies used by other services to create their own uniquely powerful discovery engine.
To create Discover Weekly, there are three main types of recommendation models that Spotify employs:
- Collaborative Filtering models (i.e. the ones that Last.fm originally used), which analyze both your behavior and others’ behaviors.
- Natural Language Processing (NLP) models, which analyze text.
- Audio models, which analyze the raw audio tracks themselves.
Take a look at various Implicit Matrix Factorization for Collaborative Filtering being used at Spotify and how to implement a small scale version using python, numpy, and scipy, as well as how to scale up to 20 Million users and 24 Million songs using Hadoop and Spark.
(Chris is Engineering Manager - Recommendations and Personalization at Spotify)
The paradox of the Spotify experience is that while it enables listeners to tumble down their own personal niche rabbithole, it’s also geared towards casual, passive listeners. The front page is dominated by pre-existing playlists; the weekly pre-eminence of New Music Friday emphasises the imperative to stay up-to-date with what’s popular (or, rather, what Spotify has determined will be popular).
Active choices – to listen to a particular album, for instance – morph into passive listening, as the default setting is for algorithms to automatically choose what you listen to next rather than the music stopping.
“Centralised influences and centralised incentive models ... create boring ass music. It creates boring ass culture,”
The main ingredient in Discover Weekly, it turns out, is other people. Spotify begins by looking at the 2 billion or so playlists created by its users—each one a reflection of some music fan’s tastes and sensibilities. Those human selections and groupings of songs form the core of Discover Weekly’s recommendations.
“Playlists are the common currency on Spotify. More users knew how to use them and create them than any other feature,”