There has been an interesting exchange regarding the ‘bibtip’ recommender system on the NGC4LIB mailing list over the weekend. This explored some of the issues with recommender systems, specifically in the context of libraries, including ‘the ant problem’  – where recommender trails become self-perpetuating, the need for a critical mass of data for a successful recommender system, and the  need to base recommender systems on ‘high cost’ behaviour

This triggered a few thoughts on my part:

1. This is a difficult problem, and commercial organisations are willing to put significant resource into it. Would some library organisation (OCLC, Library consortiums, national or international groups of libraries) be interested in running a competition along the lines of the Netflix competition http://www.netflixprize.com/?

2. We would be in a much better position to build a critical mass of recommender (and other) information if we had a single source of bib records which acted as a hub for linking. OpenLibrary has a goal that might fulfil this, but Worldcat is clearly a good starting point as well. The library community really needs to exploit linking (creating a link seems to me to be a high(ish) cost behaviour). Wikipedia entries appear highly ranked in Google rankings and it must be partially related to it becoming a de facto standard for linking basic reference information – if we could emulate that so there was a central resource to which people ‘just linked’ when they cited bib information, then this would really start to exploit the latent information available in the web. (We are really late on this one, and have lots of catching up to do – I would guess Amazon must be the main receiver of ‘bib’ linking on the web currently).

On NGC4LIB Kevin Kidd said: “When it comes to things that really require critical mass – like tagging, reviews and ratings – we need to begin to develop platforms that can link users, usage and bib data across universities (I am really not qualified to comment on the needs of public libraries in this context). We have the technology now to begin doing this.”

I would argue all the technology is there, and has been for years. If there was a recognised hub of bib information which people used to link to, and tag (using existing bookmarking services like delicious, digg etc.), this would work right now – technology is simply not an issue. Being slightly less ambitious, the number of catalogues that offer the ability to easily bookmark a bib record without including session info in the URL is lamentable – which would be so simple from a technology perspective.

3. Just to highlight another approach to recommender systems, bX, developed by Herbert van de Sompel and others – this is an attempt to exploit user behaviour information gathered by OpenURL resolvers. I guess it will also be susceptible to some of the problems discussed on NGC4LIB, however, my guess is that this captures higher cost behaviour than looking at OPAC records.