Last changed 4 July 1998 ............... Length about 900 words (6000 bytes).
This is a WWW document maintained by Steve Draper, installed at

Web site logical path: [MIRA] [MIRA workshops] [Dublin workshop] [programme] [this page]

More details on Dublin workshop

Proposals for papers for the final MIRA conference

Each MIRA member will probably want to publish a paper to represent the result of their work in MIRA. But all papers will be refereed and will only be published in the conference book if they pass this process. As the deadline for submission is less than 6 months away (January 1999), it is time to start planning these papers. At the Dublin MIRA workshop, there will be sessions for each of us to try out an outline of a paper on each other, and get some useful feedback in time to act on it before the deadline. Each MIRA member is advised to to prepare an outline for a paper and present it to a group in Dublin. This is voluntary, but hopefully can be helpful.

The plan is to divide into small groups of about 6, and each present a short outline and see what comments the rest of the group make about its suitability and interest.

Working groups

At the last MIRA workshop, several working groups were set up. Each working group is expected to present a report at Dublin. Furthermore, we would like there to be a set of exercises for people to participate in at the final conference. This will give them a personal experience that is probably much more convincing that just an academic paper would be. For instance, Fabio's working group is looking at the amount of disagreement between people in making relevance judgements. It should be possible to produce an exercise that demonstrates this, in which anyone can make some relevance judgements and then compare their own with other people's. At Dublin, we hope each working group will present an exercise like this for us all to try out: if it works well, we can use it at the final conference.

Sessions on relevance/ information needs

At the Dublin MIRA workshop there will be a series of 3 related sessions. They will explore issues, often discussed in the literature but not brought into mainstream consideration in IR evaluation, about the complexities of "relevance" and "information need". The aim of each of these sessions is to bring out and juxtapose as many different points on the theme as possible, so that the workshop can consider how they might be put together. Hence many very short talks, each making a few distinctive points, would be most useful.

The first session is on "information needs" (what are they, how many different types are there?). The second is on scalar, non-binary relevance judgements i.e. on arguments for getting away from the binary relevance judgements used in most test collections and recording each judgement at least as a number or ranking. The third is on multiple types or dimensions of relevance: on why even a single number is inadequate.

In my (Steve Draper's) view, the papers by Stefano Mizzaro are the best summary in these areas so far, and we should use them as a starting point. They offer a framework that tries to cover all the issues, and have copious references to the literature that are useful in relating the various different terms and distinctions that have been made. It may be that the discussion will show us how the framework has to be extended to cover everything (the second of these sessions run by Fabio probably does not fit directly), but it would be helpful if everyone has looked at Stefano's work first. One of his papers, also published in Interacting with Computers is available on the web . (He has another one with an even more comprehensive literature review.)

For the first session, on information needs, a useful starting point is his view of that as a dimension (dimension 2, section 2.2 in his paper). I have summarised his 4 dimensions on a web page, but for a fuller explanation and references to the earlier literature, you need his paper.

Stefano is coming to the workshop, and is interested in learning how he needs to update his framework.

Web site logical path: [MIRA] [MIRA workshops] [Dublin workshop] [programme] [this page]