Collaborative Filtering

Instead of asking opinions to each individual, you might try to determine an "average opinion" for the group. This, however, ignores your particular interests, which may be different from those of the "average person". You would rather like to hear the opinions of those people who have interests similar to your own, that is to say, you would prefer a "division-of-labor" type of organization, where people only contribute to the domain they are specialized in.

The basic mechanism behind collaborative filtering systems is the following:

• a large group of people's preferences are registered;
• using a similarity metric, a subgroup of people is selected whose preferences are similar to the preferences of the person who seeks advice;
• a (possibly weighted) average of the preferences for that subgroup is calculated;
• the resulting preference function is used to recommend options on which the advice-seeker has expressed no personal opinion as yet.

If the similarity metric has indeed selected people with similar tastes, the chances are great that the options that are highly evaluated by that group will also be appreciated by the advice-seeker. The typical application is the recommendation of books, music CDs, or movies. More generally, the method can be used for the selection of documents, services or products of any kind.

The main bottleneck with existing collaborative filtering systems is the collection of preferences (cf. Shardanand & Maes 1995). To be reliable, the system needs a very large number of people (typically thousands) to express their preferences about a relatively large number of options (typically dozens). This requires quite a lot of effort from a lot of people. Since the system only becomes useful after a "critical mass" of opinions has been collected, people will not be very motivated to express detailed preferences in the beginning stages (e.g. by scoring dozens of music records on a 10 point scale), when the system cannot yet help them.

One way to avoid this start-up problem is to collect preferences that are implicit in people's actions (Nichols 1998). For example, people who order books from an Internet bookshop implicitly express their preference for the books they buy over the books they do not buy. Customers who have bought the same book are likely to have similar preferences for other books as well. This principle is applied by the Amazon web bookshop, which for each book offers a list of related books that were bought by the same people.

There are even more straightforward ways to collect implicit preferences on the web. One method is to register all the documents on a website that have been consulted by a given user (cf. Breeze et al. 1998). The list of all available documents, with preference 1 for those that have been consulted and preference 0 for the others, then determines a preference function for that user (cf. Breeze et al. 1998). Using a similarity metric on these preference vectors makes it possible to determine neighborhoods of users with similar interests.

More info:

• Collaborative Filtering Research Links: a list of papers about collaborative filtering, with abstracts and links to the full papers
• Collaborative Filtering Resources at Berkeley and at the SIGGroup
• Breese J.S., Heckerman D. and Kadie C. (1998), Empirical Analysis of Predictive Algorithms for Collaborative Filtering, Proceedings 14th Conference on Uncertainty in Artificial Intelligence, Madison WI: Morgan Kauffman.
• Nichols D.M. (1998) "Implicit Rating and Filtering", Proc. Fifth DELOS Workshop on Filtering and Collaborative Filtering, Budapest, Hungary, 10-12 November 1997, ERCIM, 31-36.
• Resnick P, Iacovou N., Suchak M., Bergstrom, and Riedl J. (1994), "GroupLens: An open architecture for collaborative filtering of netnews", Proceedings of ACM 1994 Conference on Computer Supported Cooperative Work, Chapel Hill, NC: ACM, 175-186.
• Shardanand U. and Maes (1995), Social information filtering: Algorithms for automating "word of mouth", Proceedings of CHI'95 -- Human Factors in Computing Systems, 210-217
• A. Chislenko's essay on Automated Collaborative Filtering and Semantic Transports
• Fifth DELOS Workshop on Filtering and Collaborative Filtering, Budapest