I’m at Internet Librarian International today and tomorrow (I’m speaking at the end of today). This session is being delivered by Joy Palmer (MIMAS) and Brian Kelly (UKOLN).

Starting off with Brian Kelly – with a focus on the impact of the Social Web:

First thing to maximise impact of content – set it free! So he encourages people to record and disseminate his talk via many methods – and applies Creative Commons license to enable this.

Brian highlights there is increasing acceptance of ‘Social Web’ as of value to us as professionals – so time to start asking questions about impact and showing value. Brian believes we need to reshape the Gartner hype cycle curve – avoid the ‘hype’ being too much, and jump the ‘trough of disillusionment’ straight to productivity.

Brian showing stats about his blog http://ukwebfocus.wordpress.com/ – but stressing stats in isolation are meaningless – need to give context. But also stresses the dangers of ‘rankings’ for blogs – measurement leads to ‘winners’ and ‘losers’?

Arguments against gathering stats:

• We know we are doing a good job
• Users happy with services
• Gathering stats distracts from core businesses

But:

• Government (and other bodies) unconvinced
• Maybe some services need to be ‘retired’
• Evidence could be in favour of more investment in some services

Need to understand trends… e.g. how effective are mailing lists? About 3rd of the audience saying still important way of discussing things, about 3rd saying people have moved onto other mechanisms – RSS, Twitter, blogs etc.

Brian shows two pieces of evidence – lists aimed at web managers and web professionals have dropped significantly in use, whereas lis-link although has dropped slightly still very well used… – shows importance of context.

What types of metrics?

• Blog usage statistics – but beware of automated tools artificially boosting stats (and other reasons why stats vary)
• Technorati – Some say Technorati rankings not worth screen they are projected on, but Brian saying this is worthwhile because no effort and people will pay attention

People’s behaviour changes over time – Brian now seeing impact of Twitter referrals in terms of what drives traffic to his blog – so he knows to focus in this area to bring more traffic. He can also see in stats the impact of him adding a ‘subscribe by email’ to his blog – big leap in number of hits – so makes sure he always includes this option on blogs now.

What to do?

• Register with Technorati
• Look at Wikio for blog rankings as well

But Brian stressing limitations – league tables can be flawed – blogs may stop at end of project and drop from rankings etc. – but important to have a ‘story’ to explain.

Brian noting that the effort of tweeting a post is absolutely minimal – and substantial return – look at effort. Recommends registering with bit.ly to get stats on use of any shortened URLs you post using the service.

Finally Brian suggests that we can identify popular services and contribute to reach audience – so Wikipedia good example.

Now over to Joy Palmer to talk about moving ‘Beyond Usage Stats’ – or demonstrating value and marketing services when you have no money:

Joy faces the challenge of understanding the impact of JISC national services – Copac, Zetoc, Archives Hub … – on the UK knowledge economy.

Wanted to use audience research to drive engagement – needed to understand:

• Perceptions
• Value
• Unique selling points
• … but most of all BENEFITS

Benefits drive engagement (in the commercial world = SALES).

Often we think we know who are users are, and how they benefit from our services – but if you question and dig you may find answers are not what you expect, or that you know less than you thought.

Statistics only give very partial information about usage, and give little or know information about how a service is actually valued and where it fits in the users’ context. That’s not to dismiss statistics completely – as Brian outlined you can interpret statistics in useful ways.

However, there is a lot that statistics don’t tell you. Understanding impact and value means rethinking performance. Joy going to outline what they did at MIMAS:

Looked at who exactly is using the services – but beware you may find biggest users are not the people you are funded to provide services to – e.g. Archives Hub has a great deal of use from family historians – which is great, but not the target of the core funding for the service.

Looked at questions of market penetration (and defining market) and other areas…

Joy recommending the JISC “Guide to Researching Audiences” – did it on a shoestring, over Christmas! Carried out online surveys (over 3 weeks), and then follow up interviews.

Polite insistence to get surveys filled in – e.g. popup on COPAC website. Also found bribery effective – amazon vouchers for interviewees. Used 4 staff to do 12 x 30 minute interviews.

Wanted to understand:

• If users UK based
• Were they from HEIs
• What their role was (academic, student, librarian)
• …

Got lots of good feedback – much more feedback on Zetoc and Copac than on Archives Hub – perhaps because screened for people from UK HEIs – and for Archives Hub lots of users outside this core audience.

However, still left with holes in knowledge…

Features are not benefits!

‘simple and convenient’ is not a benefit – it is a feature.

Need to put yourself into the mindset of the user – far too often we talk about features of our services – which can be useful but doesn’t clearly communicate what your product is and what it will do for the user.

‘create new knowledge’, ‘further your career’, ‘gain recognition’ – these are user benefits – and this is the language MIMAS have started to use on the homepage of the Archives Hub.

E.g. – we don’t need to sell ‘cross-search’ but the benefits of serendipity.

Quotes from users very valuable – and can be mined for benefits.

e.g. on Zetoc alerts “The alerts have enlightened me on many topics”

What next for MIMAS in this area?

• Need to do more market analysis – who aren’t we currently reaching
• Need to develop marketing startegy
• …

Key message – you don’t have to spend a lot – 3-5 interviews can give you a huge amount of valuable information. This is NOT A SCIENCE – there is no set way of doing this, although there are some methodologies you can adopt.