I'm going to make the assumption that the people who read this column use GMail and not other email services from the previous Ice Age. Now, the next time you read your email, pay attention to the ads that appear on the extreme right. Open an email thread about someone speaking about jobs and you will find ads that offer you jobs. Open a thread that mentions a tourist destination and you will find offers for unbelievably low air fares to that destination. What actually happens here is that Google uses some massively complex computing to predict what sort of ads you might find relevant, and among the many complex statistical methods they use, the one that we are interested in is the Collaborative Filtering (CF) method.
A more visible example of Collaborative Filtering happens on online shopping sites like Amazon, which were among the pioneers of CF. When you attempt to buy an item, Amazon will tell you that people who bought this also bought certain other items which, they hope, will entice the current user to fork over some more of his hard-earned cash. It's quite simple when buying, say, a mobile phone. Amazon will tell you that people who bought this phone also bought a certain Bluetooth headset so that they might buy both.
It gets more interesting with books, the product category that Amazon started with back in the day. Bookworms are always looking for the next book to chew through, and typically rely on book reviews in newspapers and magazines and the occasional visit to the bookstore. This is clearly not optimal because publishers do all kinds of unethical things to get favourable reviews. The blurbs rarely print bad reviews. The last time I checked, practically every book at my local store was “a New York Times Bestseller.”
Here's where the Internet provides an elegant solution. Amazon analyses the data of millions of customers and comes up with recommendations like “people like you also read these books.” The method is as follows. Look for users who rate books in a similar pattern to the current user and predict book titles that he/she is likely to enjoy. What this does is remove institutional bias and use the opinions of like-minded people to offer recommendations. Amazon, of course, isn't doing this for the benefit of humanity. It's doing it to sell more books.
A few years back, the DVD-rental and online-movie-streaming service Netflix announced a competition inviting programmers to design a movie-recommendation algorithm that did better than their current system. Like books, movie tastes tend to be clustered. Sci-fi fans. Chick-flick fans. Akshay Kumar fans, etc. What Netflix wanted to do was improve the engine that suggested to their paying customers what movie they should be renting next based on what people like them saw and enjoyed. The winner took home a million dollars.
But there is a dark side to Collaborative Filtering. As people continue to move away in droves from dead-tree versions of newspapers (at least in the West), iPad apps like Zite and Flipboard use CF methods to generate highly personalised newspapers for every reader. Eli Pariser, in his insightful TED talk on “Filter Bubbles” points out that this can be dangerous in the long run. People will tend to custom-design filters that only show them news and opinions they agree with or are aligned with ideologically.
In short, every person will live in his own echo chamber of collaboratively filtered agreement. But again, I believe technology will always have a solution for the problems it creates. It's not too hard to envision an anti-CF engine, one that shows alternate opinions and tastes. And perhaps one of them will introduce me to interesting people who don't use GMail.
