MetaLib/SFX – usability and organizational impact

A study from Jonkoping University.

From the library perspective MetaLib/SFX offers a single point of entry for multiple library resources, and MetaLib/SFX are updated centrally, which is much easier than local libraries trying to keep track of the all available resources.

The study was based on a series of workshops with library staff, and how they expected it to affect their work, and with end-users of MetaLib.

The study was qualitative, but it seemed to indicate that MetaLib resulted in better exposure of all available resources, but the end-users find it difficult to interact with MetaLib.

MetaIndex – ‘Clearly a Success’

A presentation from Ari Rouvari and Ere Maijala – seemingly tireless developers from the National Library of Finland where they work on MetaLib.

MetaIndex is a product to build index into MetaLib – it harvests metadata from other resources, and creates a searchable index on a central server. However, from the users perspective, it is just another federated search target for MetaLib to search.

It relies on the OAI-PMH to harvest metadata, and you can use it when you can’t do a federated search using a standard protocol (like z39.50, or SRU/SRW)

Some examples of it in use are DOAJ and E-LIS, Electra, E-thesis local collections, Music collection of Pirkanmaa Polytechnic.

In Finland they have a Nation wide unlimited license to for MetaIndex, so they can create as many as they want. However, they found that they had to be managed centrally.

Overall, they have found the MetaIndex an extremely powerful, and felt that the importance of OAI-PMH and the ability to create indexes is comparable to OpenURL and SRU in terms of impact and importance.

MetaIndex – ‘Clearly a Success’

A presentation from Ari Rouvari and Ere Maijala – seemingly tireless developers from the National Library of Finland where they work on MetaLib.

MetaIndex is a product to build index into MetaLib – it harvests metadata from other resources, and creates a searchable index on a central server. However, from the users perspective, it is just another federated search target for MetaLib to search.

It relies on the OAI-PMH to harvest metadata, and you can use it when you can’t do a federated search using a standard protocol (like z39.50, or SRU/SRW)

Some examples of it in use are DOAJ and E-LIS, Electra, E-thesis local collections, Music collection of Pirkanmaa Polytechnic.

In Finland they have a Nation wide unlimited license to for MetaIndex, so they can create as many as they want. However, they found that they had to be managed centrally.

Overall, they have found the MetaIndex an extremely powerful, and felt that the importance of OAI-PMH and the ability to create indexes is comparable to OpenURL and SRU in terms of impact and importance.

Students’ Experience of MetaLib and Google Scholar

This is by Glenn Haya from Stockholm University and Else Nygren from Uppsala University. This was a study done in Sweden, where there is a national MetaLib consortium.

It may be interesting to compare this with the analysis Marco did last year of Google Scholar.

The study has just be completed – although it looks like quite a weighty tome – you can read it online (still waiting for a link on this).

The big questions they hoped to answer were:

What happens when a student sits down with Google Scholar or Metalib and tries to search for material? (interestingly there seemed to be more written about MetaLib than Google Scholar here)
What role does instruction play in the experience?

They used 32 students, and were asked to do searching, one group with training, and one without. Each student was recorded using software calle ‘More’ (or Moray?), which records audio, video, mouse clicks, keyboard strokes. These were advanced and intermediate undergraduates who were writing a thesis – and the searching was on their thesis topic.

It’s worth noting that the MetaLib implementation lacked certain functionality that is available in the product in general – specifically the personalisation aspects. Also it looks like SFX is implemented within Google Scholar.

Anyway – down to the analysis:

They found that there were specific phases to the search process:
Navigation
Beginning
Search
View Results list
(View Metadata)
View Full-text
Save Full-text

They graphed how much time was spent on each of these phases as their search session progressed.

Some interesting results on the quick set screen – the search immediately tried to select multiple quick sets.
Then, where there was a problem, then the searcher just added another search term – and so got far too specific in their search – and got no results. Then hits the back button – which doesn’t work. The user never got to a full-text article.

To compare Google Scholar and MetaLib:

Phase Google (Time %) MetaLib (Time %)
Beginnging 0.6 3.9
Searching 15.3 15.9
Viewing Results 30.9 11.2
Viewing Metadata 5.1 11.0
Viewing Fulltext 30.0 3.9
Saving Fulltext 2.1 1.0
Navigation 15.8 52.9

About half the searches carried out in MetaLib did not produce a results list.

Training made a large difference in how many articles were saved when using MetaLib (from 12 to 30). However, compare these to Google – 37 to 48.

Looking at the quality of the articles found, what was interesting was that before training with Google scholar only 21% were peer-reviewed. However, after training, this became 48%. With MetaLib the percentage stayed persistent at 42% before and after training. Some suspicion that the improvement with Google was due to the use of the SFX link in the Google Scholar results, leading the users to pay more attention to those articles available from the University via SFX.

Some considerations – Google is essentially a familiar interface, and the user expectations are geared towards this. MetaLib is a more complex application, and is designed to do more than Google Scholar. It is also true that perhaps for Undergraduate level Google Scholar is ‘good enough’?

A comment for MetaLib team is that in v4, the default search will be keyword – but it is currently phrase – which leads to the problems with multiple terms mentioned above.

However, I think the biggest issue here is the amount of time spent on Navigation in MetaLib. Whatever way you interpret the results, 50% of the ‘searching’ time spent ‘navigating’ surely is a bad thing?

Decoupling the interface

Since there has been some talk at the conference of de-coupling the user interface from ‘back office’ systems (tomorrow we have several sessions on Primo, which is the approach Ex Libris is taking to this), I thought that this development blogged by Lorcan Dempsey was worth a look – a user driven approach to decoupling the experience.

Library way: “E41ST uses Flex 2, Amazon web services, and ‘your public library’ to connect an Amazon browsing experience with a look up in a local public library.

Decoupling the interface

Since there has been some talk at the conference of de-coupling the user interface from ‘back office’ systems (tomorrow we have several sessions on Primo, which is the approach Ex Libris is taking to this), I thought that this development blogged by Lorcan Dempsey was worth a look – a user driven approach to decoupling the experience.

Library way: “E41ST uses Flex 2, Amazon web services, and ‘your public library’ to connect an Amazon browsing experience with a look up in a local public library.

Deep linking in Ex Libris products

The last session this morning is a Q and A with Ex Libris. I’m not blogging most of it, as I wouldn’t want Ex L to feel inhibited in their discussion, and most of it isn’t terribly interesting to the world at large.

However, one question has come up, that was also raised in the integration session this morning, which is about providing persistent deep links into Ex Libris products.

For example, this might be to deep link to a ‘search’ in the library catalogue, or to deep link to a specific resource within MetaLib.

This cannot be just for Ex Libris products – when we link to a library catalogues web interface to do a search, each vendor has a different link syntax. Perhaps some implementation of SRU?

Standardisation would make work such as the work we have done with Moodle and deep linking into our library systems easier to share, and transferable over different product versions and suppliers.

Deep linking in Ex Libris products

The last session this morning is a Q and A with Ex Libris. I’m not blogging most of it, as I wouldn’t want Ex L to feel inhibited in their discussion, and most of it isn’t terribly interesting to the world at large.

However, one question has come up, that was also raised in the integration session this morning, which is about providing persistent deep links into Ex Libris products.

For example, this might be to deep link to a ‘search’ in the library catalogue, or to deep link to a specific resource within MetaLib.

This cannot be just for Ex Libris products – when we link to a library catalogues web interface to do a search, each vendor has a different link syntax. Perhaps some implementation of SRU?

Standardisation would make work such as the work we have done with Moodle and deep linking into our library systems easier to share, and transferable over different product versions and suppliers.

MetaLib and Usability

Nick has left a comment on the post on MetaLib developments regarding usability (as opposed to accessibility), and I thought it was worth a post. In fact, a later session today covers a usability study.

I’d agree that there are usability issues for MetaLib. However, one message that has been consistently sent to Ex Libris over the last few years is that changing the interface is a huge amount of work for libraries, as it means retraining patrons, and this is bad for PR as well as requiring large amounts of work.

I’m very happy that we aren’t seeing substantial changes, as although it may not be an incredibly intuitive interface, the majority of our users have now been using the interface for at least a year.

In terms of continued development of the interface, Ex Libris has been asked about development of the Aleph OPAC in the light of Primo, and have given a committment to keep developing this, and I guess that the same will be true of the MetaLib interface.

One other point that has come through strongly at this conference is that with a complex product like MetaLib, it is unlikely that Ex Libris can deliver an interface that suits everybody (anybody?). Also, many institutions are interested in integrating MetaLib into other environments (portal, VLE, etc.) This leads to the x-server being even more important – and this is the method by which individual institutions can provide an interface which suits their particular requirements.

MetaLib and Usability

Nick has left a comment on the post on MetaLib developments regarding usability (as opposed to accessibility), and I thought it was worth a post. In fact, a later session today covers a usability study.

I’d agree that there are usability issues for MetaLib. However, one message that has been consistently sent to Ex Libris over the last few years is that changing the interface is a huge amount of work for libraries, as it means retraining patrons, and this is bad for PR as well as requiring large amounts of work.

I’m very happy that we aren’t seeing substantial changes, as although it may not be an incredibly intuitive interface, the majority of our users have now been using the interface for at least a year.

In terms of continued development of the interface, Ex Libris has been asked about development of the Aleph OPAC in the light of Primo, and have given a committment to keep developing this, and I guess that the same will be true of the MetaLib interface.

One other point that has come through strongly at this conference is that with a complex product like MetaLib, it is unlikely that Ex Libris can deliver an interface that suits everybody (anybody?). Also, many institutions are interested in integrating MetaLib into other environments (portal, VLE, etc.) This leads to the x-server being even more important – and this is the method by which individual institutions can provide an interface which suits their particular requirements.