Last changed 25 Apr 2003 ............... Length about 9,000 words (57,000 bytes).
(This document started on 17 Apr 2003.) This is a WWW document maintained by Steve Draper, installed at http://www.psy.gla.ac.uk/~steve/chi03/report.html. You may copy it. How to refer to it.

Web site logical path: [www.psy.gla.ac.uk] [~steve] [chi03] [this page]

Report on the CHI 03 Doctoral Consortium

By Steve Draper,   Department of Psychology,   University of Glasgow.

This is a complete second draft of the report, and no more feedback has come in after April 2003.

Contents (click to jump to a section)

Preface

This is a report on running the Doctoral Consortium at CHI 2003 (held April 2003 in Fort Lauderdale, Florida, USA). The panel was:
  • Clarisse de Souza PUC-Rio (Pontificia Universidade Catolica), Rio de Janeiro, Brazil
  • Steve Draper (chair) University of Glasgow, UK
  • Anthony Hornof University of Oregon, USA
  • Lisa Neal EDS, Boston, USA

    This report is written most directly for next year's CHI doctoral consortium chairs, Jean Vanderdonckt and Liz Churchill. However it may be of some interest to other years and other conferences. It also has some points about student posters as well as the doctoral consortium.

    This report, while to some extent informed by feedback and experience, is only my opinion, not policy nor evidence-supported fact.

    Introduction

    From the viewpoint of someone running part of a conference, the worst situation would be to have compulsory constraints and instructions without recorded reasons, whereas the ideal would be to have:

    CHI is nearer the worst position, although better than many conferences. There is:


    In this report based on my experiences as (sole) Doctoral Consortium chair at CHI 2003, I will try to move the organisational knowledge on this a bit further up the spectrum. However I won't provide much detailed actions and model emails and text, as I don't think I did those all that well. The biggest source of my own practice at the larger scale decisions was Gilbert Cockton, who was both conference co-chair and had been a doctoral consortium chair at CHI recently, constrained by the policy document; while at the shorter scales (how to do the minute by minute face to face stuff) I had some previous experience in doctoral consortiums at other conferences.

    The CHI recipe for a doctoral consortium

    I've done doctoral consortiums at other conferences, and I wasn't particularly keen on the CHI format for doctoral consortiums when I took the job, but I now think this is a particularly good recipe and other conferences should consider adopting it. This report is in part about expressing reasons for this recipe which were not apparent to me in advance.

    The recipe, in summary, now is:


    Evaluation of the 2003 doctoral consortium

    There was no big evaluation of the doctoral consortium, although I did collect comments by email from the student participants, collected here. However I now believe for these reasons that it was very successful:

    Pre-planning vs. spontaneity

    A big issue in organising anything is the balance between pre-planning and spontaneity (or "just in time decisions"!). I skimped on timely advanced organisation, mainly due to character defects. On the other hand it worked, and it is possible to reduce flexibility by doing too much advance planning, and to make it all feel burdensome by telling people your plans in advance when they don't really need to know until the moment.

    In writing out reasons in this report, I am trying to provide future organisers with material for thinking about and planning doctoral consortiums. It doesn't follow that they should adopt all my actions, nor that they should make detailed plans in advance. Better understanding of the issues allows good on the spot decisions just as much as better advanced planning.

    Summary of the key requirements

    This is a summary: for some rationale, see the main report below.

    1. Good quality discussion of each student's work in turn.
      Hence: the time per student devoted to this is an important metric (about 30 minutes at CHI 03), you may need to rein in the panel if their comments show more quantity than quality, may need to further encourage student comments but only if they are good quality, consider inviting visitors to join in here and there, get everyone to read the papers in advance, keep the talks short.
    2. Students getting to know each other well.
      This is the foundation of a peer group of new HCI researchers, introduces them to the great breadth of topics in HCI, and lays the foundation for good discussion. Organise socialising together, support student comments on each other's work.
    3. Students having prolonged access to the panel faculty.
      Not just their formal comments in the sessions, but chatting at meals.
    4. Students making contacts with "notable" researchers.
      Programme of inviting them to meals with the doctoral consortium, encourage students to approach them elsewhere, perhaps use your own contacts to prompt their visiting the student posters.
    5. Students present a poster as well.
      Good for the students, good for promoting further contact-making, good for integrating the doctoral consortium with the rest of the conference.
    6. Talks/discussions within the doctoral consortium on other aspects of being a doctoral student.

      Minor aims

    7. Subsidising students by getting funding for them.
    8. Making it a truly international group.
      Promoted by subsidy, by mentoring, and by careful selection of the panel.
    9. Giving them practice (and feedback) on giving talks.
      Talking to 20 strangers is obviously better practice than doing it in their home department, but not as scary as to a big conference audience.
    10. Giving them practice at contributing to discussion.

    Main report on Doctoral Consortium at CHI 2003

    I will deal with points here in order of their importance to the overall success of the doctoral consortium (not in chronological order of the action required, nor in some logical structure about the type of issue).

  • Devoting substantial time to discussing each student's research.
    The CHI formula of 15 students over two working days allows 45 minute slots for each student, of which only 10 minutes need be given to their talk (not least, because their documents can be precirculated and read by all in advance). This is strongly supported by student feedback as a key feature. One student said that this was the only time they had had such prolonged attention to their work apart from one on one sessions with their advisor, and how useful it was to get other perspectives. Another student said they had been in the doctoral consortium at CSCW, which had the same number of students in half the time, and how much better this format was. Note how sensitive this is to the overall time allocated: if the student talks take 15 minutes, then a extending each slot from 30 to 45 minutes doubles the time on this top priority activity of discussion for the cost of only 50% increase in time overall. The policy document might be tightened in this respect to stress that the full two days for 15 students is important.

    The focus on discussion is because this is what you need the face to face situation for, while learning about their work can largely be done by reading in advance.

    In addition, a significant benefit is (as one of the students remarked in feedback) hearing the comments made to other students. The time other than when their own work is being discussed is not dead time, nor useful only if they are making comments themselves: significant "incidental" learning goes on when they are only acting as bystanders.

  • The students get to know each other well.
    This is important for several reasons: (a) founding contacts in the field that may remain important to them for years; (b) hence experiencing the great range of kinds of research done in HCI; (c) making them feel very relaxed during the consortium and for the rest of the conference as part of a group they feel at home with. (One of them actually said this, but it was obvious in their relaxed demeanour after the doctoral consortium, and in the way they continued to seek each other out for evening socialising, sit next to each other in talks, etc.) This is an important part of a good conference experience, and is particularly missed by most first time attenders. (d) As a consequence, feeling able to make contributions to discussion, and to receive comments without getting too anxious.

    While no special action needs to be taken for this, it is actually probably one of the most important benefits for the participating students. It is also experienced by student volunteers, but possibly something more might be done for student poster presenters. For the doctoral consortium, it will usually happen automatically given two days in a nearly closed session, with most meals held together: another reason for these aspects of the standard formula.

  • The third of the big three aspects is inviting other researchers to meet the students at meals during the doctoral consortium. There is no doubt this is a big hit with the students: at one end one student this time went round saying it had changed her life (after lunch with the person's whose published framework she was trying to use in her PhD), and at the other a student saying he wished there had been more such guests so he could have met someone more directly relevant to his own work.

    It is also rather easy to arrange, since everyone I invited, whether they could come or not, said they were delighted to be asked. Furthermore, those who came all seemed to enjoy it: after all, it is a free meal with company that is non-threatening but very pleased to meet them. Given that, you should consider asking anyone: keynote speakers, the very famous, the more modest but whom you actually admire the most. A good tip is to ask, where possible, couples (whose plans are obviously often linked). If you organised this well in advance (which I did not) you might be able to persuade some to travel earlier than they otherwise would have. If not, you will be largely limited to those on the conference committee plus those giving tutorials or workshops: but clearly this is actually a very rich pool for these purposes. Finally the people running the following year's doctoral consortium should probably be invited for one meal, and will probably want to sit in for part of the doctoral consortium to see how it feels.

    Given you have arranged the guests, don't forget that the students probably don't know who they are, however famous. So brief the students on them before the meal, and explicitly pick out the students who would get the most out of them and tell them to sit next to the guests. You can do this by a) reading the references cited by each student; b) looking at the short summary of each student's research in your notes. Perhaps I should additionally have got people to change places after each course. Remember the bad thing about conference meals is that you can only talk to the 2 or 3 people next to you and may have nothing to say to them; and the next worst thing is that old hands all stick together and never meet new people. Students are obviously the most vulnerable to these traps. A little work on active introductions can turn it into the best conference experiences for the doctoral consortium students: meeting someone new who works on something very similar to you.

    In 2003, worries about budget meant the conference provided a cheap lunch in our room, and booked the dinner at the main hotel. This was extremely convenient for me: less organisation and still more importantly, no time and trouble spent in travelling to a restaurant.

  • In the feedback, one of the students suggests taking the "guests" idea further by reserving an hour (say) for having multiple guests in, and dividing up the students in much smaller groups between them.
  • As the conference wore on, I found myself extending this principle. I went over the students' work again and thought of people at the conference who might be interested in their work, and did some work suggesting they visited that student's poster at the designated poster times. When I got this right, this was obviously great for the student, but also in fact welcomed by the contacts too, few of whom think systematically about checking out student posters, but who are in fact interested in new work in their area.

    With hindsight I would recommend thinking this out and doing some of it in advance by email and by a bunch of notes on the personal messages board: giving them advance notice raises the chance of them doing it. The absence of a delegate list at CHI is an obstruction to this: lobby for a delegate list at least for you. A list of paper presenters would do almost as well for this purpose.

    Ask the rest of the panel faculty to help with this. You could suggest this long before the conference to alert them. Then near, but still before, the conference, pass them what lists of attenders you have been able to get, and ask them to consider using their contacts a little.

    Give the students a 5 minute pep talk during the doctoral consortium on seeking out contacts in their area of research. I did this and pointed out:


    One of the doctoral consortium students immediately went off and applied this, and came back to report success.

  • Another consequence of being more active in connecting students to other researchers and contacts outside the consortium is that you will find yourself needing a list of short summaries of their research areas, to help you carry out planning introductions etc. I recommend, therefore (I wish I'd thought of this in advance), making them submit a 30 word (2-3 line) summary of their topic and what their contribution to it is. You will find yourself using this repeatedly. At other conferences this has been tried anyway for all submissions as a help to assigning referees, as a handy summary for all attenders in selecting talks to go to, and as a way of making the author think about and be explicit about what value their paper has as a contribution to research. Here you will want to use it a) for assigning reviewers from the panel, b) in choosing an order for the talks, c) perhaps in considering what guests to invite, d) in considering approaching contacts to come see their posters. Of course you may well edit your copy of these summaries, but getting authors to provide the first version is still helpful.

    I recommend that you lobby the conference chairs to make all conference applicants do it for all categories. If that doesn't get accepted require it anyway in the call for doctoral consortium submissions. If you don't do that, do it by email for those you accept. (I believe William Newman got it done for one CHI (2001?). It was done after submission but for the conference programme for HCI 2002, and I think Janet Finlay has agreed to do it for HCI 2004.)

    Additionally (just for consortium submissions) you might suggest to them that if their work was particularly inspired by one paper, then they say what the paper that inspired them was as an addendum to the 30 words. This may apply to, say, a third of submissions: for instance a thesis trying to apply Dey's framework for context, or Julian Orr's paper on "war stories". It's important to prompt comments on whether the thesis work does or does not seem to capture the spirit of the paper; but also because research may be inspired by something, yet the actual work turns out to lead to somewhere quite different, which is important but painful to recognise.

  • Function of the panel of faculty.
    The most important thing is to be able to cover all the research areas that the selected students have. In my estimate, I can find something of some kind to say about any talk, but something worthwhile about only 75% of them. (I estimated that in advance, and in fact found that to be true on the day as well.) So two panelists can probably cover it, but four is much more comfortable and certain. In 2003, the student topics ranged from low-level input device design up to highly social issues. (They might have ranged from actually constructing new physical input devices, out to pure ethnomethodological approaches.) I was a bit worried in case there were software engineering methods, or design in the art school sense to cover: but not this time it turned out.

    So topic coverage is the most important issue. Complete coverage is impossible, but a good range is feasible. In fact even if no panel member had any deep comments, it would still probably be satisfactory for the student, since simply dealing with a new audience and explaining why their work is important to the general HCI audience, is valuable. If one panellist has deep comments that is good. If two do, that is very good. In most cases this year, all four panellists had something to say, but you should probably point out that there is no need, and perhaps it isn't even desirable, for all panellists to say something to all students.

  • Selecting the panel.
    Since only a day or two's expenses are paid, they pretty much have to be going to that CHI anyway. Ideally, the panel of four should be balanced in all these respects:

  • Selecting the students.
    The most important principle (stated in the call and in the policy document) is that students should be at a stage to benefit: not too near completion to change anything, not too early to have much to talk about. Next, the policy is to pick the best (as opposed to those who most need help). In 2003 we had 41 applicants, of which about 35 would have been satisfactory to accept, but only 15 places. Consequently you may well have many of similar merit, and could choose to balance some other aspects: gender (you might have to avoid selecting too many females if you want equal numbers), research area, country of origin. There is no one way to define the latter, since we had many students born in one country, educated in another, and doing their PhD in yet another.

  • Mentoring.
    One of the students we accepted, from Athens, told me he would never have been accepted without a lot of help in writing his extended abstract from a CHI mentor. You may want to make sure this is in place, as it is probably important to achieving genuinely international equality of access.

  • Students all also present a poster.
    This is important to integrate them with the wider conference: it lets others see what the doctoral consortium students are working on, and conversely it also gives a structure for other delegates to meet them, if they want to. The students didn't much enjoy having to stand by their poster, but in fact many did make further contacts by doing so. (A pep talk on making contacts should perhaps talk about this as well.)

  • It would be a good idea to improve the support for presenting posters: perhaps get together with the student poster committee member for this, but it may be particularly important for doctoral consortium students, as they mostly aren't thinking of the poster as a primary concern. I didn't pay attention to this at the time, but noticed several small problems all of which would have to be addressed before the conference:

  • Students' extended abstracts are published in the conference proceedings. This is important in getting them published, and accustomed to being so; in allowing others to look up their work during the conference; and for a wider audience to be able to find them by search afterwards.

  • Student talks within the doctoral consortium:
    The main value is in getting discussion of each student's work, not in them presenting. Partly because they have written a 2-page abstract, and all have read it in advance, it is feasible for them to give only a short talk. I instructed them to talk for only 10 minutes and that if they took longer, it would reduce the discussion time in which they could receive comments. In the event, some kept to 10 minutes, many went to 15 minutes, one went to 20 minutes. I didn't pressure them to keep to 10 minutes beyond that general warning: again, the 45 minute slot takes the pressure off. A particular issue here is that some students had made a significant shift in their work since applying, and at least two turned out to have a depth of further relevant work they hadn't even mentioned in their abstracts.

    I also told them to be explicit about what issues they would particularly like us to discuss afterwards, and they all did that. Some started their talk with this, so we would bear it in mind during their presentation.

    This is the text of two advance emails I sent about giving the talks:

    Aim to speak for only about 10 minutes, in order to allow lots of time for discussion from both the panel and the other consortium members. You can assume everyone will have read your extended abstract at least.

    There are basically two possible aims for a talk (or a spectrum):
    a) Showing off, making people think you and your work are wonderful
    b) Getting new ideas from the audience.

    If you want to look good (e.g. in a job talk) then you hide all uncertainties, and talk only of accomplishments. If you want to learn from the audience, then you use the talk to state your problem, discuss your uncertainties (e.g. the arguments both for and against things).

    Since the consortium is closed, you don't have to worry much about showing off, and should think about what you most want to learn from the audience, and what you need to say in your talk to get them interested and in a good posiiton to comment on the issues you most want discussed.

    If your thesis is well developed and you are using the consortium to check that outsiders don't see flaws in it you hadn't anticipated, then you will want to give an overview of all your work. I think my advice then would be to concentrate on presenting the logical skeleton argument at the core of your thesis rather than details e.g. "The literature suggests these 3 ideas, and I conducted an observational study that also suggests similar points plus this additional idea; from that I want to argue that my design of my software should centre on these 4 features. Is that a reasonable leap from observation to design ideas, or have I included untested assumptions in that leap?"

    However it is up to you to use your talk to promote the discussion you most want to hear; and that will probably mean ruthlessly pruning your talk, leaving out a lot of the detail that in other kinds of talk would be important.

    I sent a long message a while ago about the talks. To paraphrase the main point:
    In this consortium, you should not think about how to interest an audience that isn't sure it wants to be there or listen to you, but on how to start a discussion that will be of the most use to you yourself.

    In my experience this might well include:
    a) A quick overview to warm the audience up and remind them of what they should have read.
    b) Say what you feel is the most interesting aspect of it all. I.e. what is really neat, what your personal enthusiasm in the project is.
    c) What it is you would most like comments on: and you may want to spend some time presenting details of this aspect in order to get those comments. This will be very different for different people. It might be "I'm confident about my implementation but am scared someone will be nasty about my statistics" or "I'm happy about all the technical details but worried it could all seem boring to an outsider: what are the exciting implications I might suggest my work leads to?" or "I've got a literature review and an implementation and a user study, but do these all fit together into a single argument i.e. a single thesis?" (I've had students that should have worried about this one a lot more.)

    The talks seemed to me to work fine.
    Having said that, practice at giving a talk is a worthwhile aim in itself, and these talks are roughly the length of a conference short paper. This is practice at giving a talk to a group of strangers (not in the student's home department), although in a relatively friendly and supportive atmosphere. One of the panellists persuaded us to have a discussion of presentation issues, and this might be a worthwhile standard feature. If you are particularly keen on this, you could reserve the last 5 (or 2?) minutes of each slot for comments on the presentation style (as opposed to content), and address issues such as how to handle difficult questions, whether to rehearse and how else to improve, and not appearing too self-doubting.

  • Discussion constituents.
    The main ingredients are the panellists' comments, comments from other students, and whether the speaker manages to retain anything from the discussion. Anthony Hornof made a crucial suggestion about the last point at the start, which we adopted: that students should find a partner (who wasn't presenting in an adjacent slot) to take notes on the whole of the discussion, since obviously the presenter might be too anxious to remember it all accurately. I in effect chaired the discussions and, without this being a planned policy, I looked round the panel to see who wanted to start this time, usually the other panel members went next, then me, then other students commented. On the second day I kept the slots to 45 minutes fairly strictly, but on the first day I let them run on until discussion waned. I regret this reduced time for other things, but it may still have had the advantage that a good way to encourage discussion from all is not to cut anyone off, and allow discussion to proceed until they are sure they can say something as interesting as the current discussant. With 45 minute slots, there is almost, but not quite, enough time for discussion to run until no-one has anything to say. Consequently there is not a big problem here, but we could ask ourselves what would further optimise the quality of contributions.

  • Panellists' comments.
    There was some prior email discussion among the panel about how to organise our comments e.g. should we divide up the students and agree who should lead for each one. In fact this was unnecessary. In most cases all four had something to say, and never was there a student who had everyone floundering for a comment. With hindsight I wish I'd reminded the panel in advance that it wasn't necessary to make a comment for every student, though that is nice if they found that easy, and that they should feel free to concentrate on the ones they felt most interested in. Having four on the panel provides a very comfortable cushion: not everyone needs a comment on every talk, and if one has a crisis and misses part of the session, and another does no advance homework it will still all work smoothly.

    A potential problem is having panel members who talk too much, regardless of the quality of what they say or time constraints. I was clearly the biggest threat of this kind on this panel, but putting myself last to comment and being the one responsible for time-keeping as well, largely contained this problem (I think!).

  • Content of comments.
    An issue raised in the feedback, but for which I'm not sure of a settled view, is whether the comments should include suggestions about major changes of method etc. On the one hand, a student with less than a year to go cannot consider learning a new method and running new studies. On the other hand, exposure to contrasting methodological attitudes is one of the more valuable things that can be offered, not least in order to give the student practice in defending their work to those who don't share their assumptions. There is some discussion of this in the feedback.

  • Student comments.
    I didn't plan specially for this in advance. However, again because of the 45 minute slots, there was plenty of time, and there were some student comments from the first talk onwards. On the second day, for other reasons, I kept calling on a student to make the first comment, and that set a precedent that had other students pitching in comments before the panel made any. Maybe I was lucky, or maybe that would be a good deliberate plan: to be lax with time keeping for the first talk or two, to encourage hesitant students to make comments and set a precedent; then later to call on students before the panel. Another possibility would be to assign students in turn to lead the questions. On the other hand, since in my view even experienced panellists may not have anything worthwhile to say about every talk, simple assignment may not lead to good quality questions. Another variant would be to instruct them in advance to prepare comments on at least three papers of their choice. Alternatively, I suppose a more systematic chairman would keep a note of who had asked questions, and begin to call explicitly on those who hadn't ever asked one rather than only waiting for volunteers.

  • On day two, next year's doctoral consortium chairs sat in on a few talks.
    I had ordered the talks so that the ones most likely to be of interest to one of them, Liz Churchill, were scheduled then. Liz joined the discussion, right in her area of expertise, and both students and panellists later told me how impressed they were with this contribution. In fact this was not the first contribution by outsiders: two different student volunteers, attending the session for occasional chores, got interested and contributed (useful) comments. This raises the possibility of including guests more deliberately. With 15 + 4 people in the core doctoral consortium group, a couple of extras in no way felt like an intrusion on the group's cohesion: certainly not after the first morning. It might be possible to suggest that meal guests, or others you happen to know are nearby, sit in for a talk or two. I had suggested to the student volunteer organiser that it might be good to allocate student volunteers who had applied but been rejected by the doctoral consortium, but I'm not sure if this was done systematically.

  • This of course interacts with ordering the talks.
    Most of the students would prefer to speak early, to get the anxiety over with, and to make sure they get a fair share of the time. However someone has to go last. I did pick a student I thought better able to cope with this to go last, and I did take care not to erode the 45 minutes for the last talks. I made up an order at breakfast on the first day, and so didn't give them advance notice of the order (apart from the first speaker whom I'd warned the night before). After all, this is not like a conference with people rushing off to other talks after giving one. I roughly ordered them with similar topics next to each other (although I wondered if the opposite policy would be better, so that someone with specialist interests wouldn't have a long period with nothing near their own topic), and with a view to the interests of the two scheduled visitors. Having the 30 word descriptions would have made this easier, and I would now consider the possibility of inviting more visitors to the discussions.

  • Other talks/activities.
    Most found it hard to sit through continuous sessions, and wanted frequent breaks. Since they usually talked a lot to each other during the breaks, these were in any case useful in their own way. It does however reduce the time left for other things.
    There was certainly an appetite for some other kinds of discussion or short talks by the panel. My timekeeping wasn't strict enough to do as much of this as might have been good. I was uncertain in advance what these students would like to hear (after all, they are better than the average PhD student, and they come from different places, so I didn't expect my standard spiels to PhD students to be necessarily appropriate). At the start, I got them to suggest a list of topics they would like to hear about ("what are the general issues about being a doctoral student that you find difficult?"), and we covered some (*) of them:

    The panel hadn't overtly agreed what we might talk about, but I had alerted them to the general possibility and we had probably each thought a bit about it. This turned out well, with each topic we did talk about being contributed to by several panel members rather than a single prepared talk. Particularly useful, even though brief, were Anthony Hornof's remarks as part of his self-introduction on how the doctoral consortium is a form of the kind of peer discussion that is so central and essential to any researcher's work.

    In addition, and following some previous CHI doctoral consortium practice, there was a women-only subgroup at one of the lunches on being a woman in this (or any) research field. It seems clear this is greatly appreciated by the women participants (though the need for it is less extreme than it used to be, with strong female applications to the doctoral consortium easily equalling the male ones, and finding good female panel members being little problem). Unsurprisingly the men felt a bit left out. Since I was warned about this well in advance, I should have dreamt up a suitable alternative (it must be too long since I was in a men's group). In fact it would have been interesting to discuss whether we men had picked academic life as more easily compatible with being an active father, whether we'd move to suit a partner's career, and whether we can cope with the present strong female competition in HCI. There is also now a group of women CHI participants (contact Judy Olson) who probably want to make contact with the doctoral consortium participants. This year they had a birds-of-a-feather session, next year perhaps it will be still more organised.

  • Introductions.
    Unlike many meetings, in this case everyone already knew everyone else's official face: by reading the students' abstracts etc., and the web pages of the panel members. In fact, I now realise it is possible, easy, and useful to start the community in advance by digital means. I should have asked the students too for their web page URLs and/or photos, and put up a simple web page to collect these for the doctoral consortium. It is unlikely that people, especially the panel, will feel they have time for email communication with strangers, but allowing everyone mutual pre-familiarisation in their own time seems worthwhile. (I only partially thought of this at all because of a request in advance from a student, and it is reinforced by some of the feedback afterwards.)

    The original recipe called for starting with dinner the night before, but since I couldn't find a flight that got me there in time, this was changed to drinks in the hotel bar. In fact not everyone turned up for that (only 2/5ths of the students and half the panel). Thus I regard a prior social meeting as both not really necessary and apparently not wanted by a substantial subset (who undoubtedly were already in town); although offering it is certainly the friendly thing to do. On the other hand (see feedback), it is highly valued by another, perhaps larger, subset.

    I began the first day's session by short self-introductions. I had everyone write a nameplate to put on the table in front of them. I went round the panel asking them to cover: their previous doctoral consortium experience, their CHI experience, their own research topic, their area of expertise, in what ways they were interdisciplinary. Then round the students, asking them to cover: how to pronounce their name, the countries they came from, how many conferences of any kind they have been to, and something / anything about themselves that is not in their (generally very professional) applications to the doctoral consortium. Thirdly, I asked everyone to suggest possible topics (put up on the flipchart) they would like discussed ("what are the general issues about being a doctoral student that you find difficult?"), starting with a couple of example topics I'd elicited in the bar the night before. My real aim in these introductions was to get everyone's voice heard briefly, in order to set an unconscious precedent; while the content was just to complement the more serious content in what we'd read on the web about each other. Anthony took the opportunity to make two important contributions: suggesting students get a colleague to take independent notes of their discussions, and putting his view on what the doctoral consortium was about (see above). This all took about an hour.

  • Changing seats.
    At the start of the second day, the students switched some of the name plates around to get people to sit in different places. A further suggestion in the feedback was to do this every half day at least.

  • Refereeing process.
    You have to invent one. I had about 42 applications for 15 places. I pre-screened out about 5 who seemed clearly not at the right stage of their PhD. You have to decide how much reviewing your panel will do: after all, if they don't like reviewing they'll want to avoid it, and if they do like it they will already have done a bunch for other parts of CHI.
    The main issue is that rejected applicants would have liked feedback, which I didn't provide, though I got a number of explicit requests for it. This would require the panel to generate written feedback useful to applicants (as opposed to useful to the selection: which will often be summaries of content, and comments on research methods). In my view, the trouble is that applicants want to know why they were rejected, but the main answer is: too few places for all the acceptable applications. It is not true that writing a better application would have made them a better person more deserving of a place. Applying to the doctoral consortium is not a lifelong skill worth practising.

  • One of the features for CHI03 which I strongly approve of was putting all the student opportunities together in a coordinated way (see the website and call for participation): the doctoral consortium, student posters, and student volunteers. We encouraged people to apply for them together, though this means coordinating acceptances to student posters and the doctoral consortium. This is clearly best for conference goers, organising it from their point of view, and particularly for students who will not be familiar with CHI.

  • Application documents.
    Applicants were required to submit:

  • Application process.
    This year, at least, they sent these to me direct by email. Quite a lot of work (about 42 applications), but I didn't have to learn some foreign software I couldn't control. They were required to use PDF format, but in fact some couldn't: since I could convert Word to PDF easily, this wasn't a problem. Some supervisor letters, sent separately, were late, but not a big problem.

  • Acceptances to student posters and the doctoral consortium must be coordinated, since some apply to both. This means accept/reject emails have to be coordinated too.

  • Get Carol Klyver in during the consortium to cover the expenses reimbursement process for the students. (15 minutes at least)

  • Get the student poster chair in to talk briefly about how to man/present their posters at the designated times?

  • The web page call promised that applications would be acknowledged by return. I would change that wording. In fact three different acknowledgements are needed (and perhaps some are better done by web page than by 3 times 42 personal emails). a) I got some kind of email submission from you; b) I decoded your attachments OK and they seem in the right format; c) I have now got all your bits including your supervisor's letter of recommendation.

    I now regret not creating a web page in my own space, and having the conference site point to it. That would allow me to announce what stage the process was at e.g. "I have now sent acknowledgements to the 41 applicants" (so if I was late, I wouldn't get 41 enquiries, yet applicants could tell if they had indeed lost an email to them): this can address the issue of how someone can detect whether an email has probably got lost. It might be a way, if no-one objects on privacy grounds, to show each applicant's application stage (perhaps on a separate, unadvertised, web page): whether all 5 documents have arrived (or which are still missing).

    Don't forget in sending out rejection and acceptance emails that you should also tell the accepted who else was accepted, and who is on the panel; and circulate to all the abstracts and personal statements of the accepted ones. Furthermore they wanted to know how many applications there had been (i.e. how lucky/good they were). Another (unadvertised) web page under your direct control is handy here. The conference committee want this information: so also tell the participants.

  • The students wanted ribbons on their badges similar to those accepted for student posters only. Certainly helpful when presenting their posters.

  • Other post-acceptance administration.

  • The expenses of the doctoral consortium are covered more or less completely by a grant from the NSF. Because the money must be paid to a US academic, it was applied for by Gary Olson. He said it only took 5 minutes, re-using an old application. Currently the NSF is very happy with this, and fully expects to fund next year's in Vienna. Carol Klyver suggests that the amount should be raised: it hasn't increased despite inflation for a while, and the increasingly international group means not all expenses are covered.

  • The policy document is out of date (not revised since 1994) in the dollar amounts mentioned, and in only offering 1 day's expenses to panel members, while a full 48 hours labour is now required. In my opinion you should lobby to have this changed now. The money should be recalculated, the $1500 limit reviewed, and both policy and the amount of the NSF grant revised together.

    Student poster recommendations

    I would contribute the following points to the wider experience from the organisers of the student poster stream:

    Better guidance and above all examples of the "sketch" required for submission. Set a file size limit and stop them sending complete posters that only print out on giant colour printers: this is inconvenient for the review process.

    Provide advice on other things presenters might bring to a poster: business cards, printouts of the abstract in the (CD only) proceedings, whether they are interested in job offers. A photograph of themselves so others can find them at the conference.

    Given student inexperience, perhaps develop a little advice on presenting a poster: why it's worthwhile, what to do, how not to feel embarrassed, lonely, and rejected. (Send this out with the acceptance email? Put it in their conference packs?)

    Perhaps organise a little student-only bonding e.g. special poster preview for all doctoral consortium and student posters to look at each others' posters, to foster community among the students at the conference.

    Having the posters in the "commons" where all tea and coffee breaks are served (and other stands and exhibits are) is excellent.

    Mixing up categories of poster by research topic was a great idea. Scheduling other posters and student posters for different times (but same place) was a great idea: reduce crowding, but raise the chance of people noticing other posters on the same research theme.

    The second student poster presentation scheduled slot was a dud in 2003, as everyone was directed elsewhere for tea, coffee, and a plenary talk at that time. They wuz robbed.

    Poster board size: warn about leaving a margin if the rim is metal, and give the actual size the poster can be, not the furniture-centered view of how much space the board takes up in the hall.

    Partial Action summary for CHI 04

    While the main part of this document was about rationale and what was done because it seemed natural, this is a summary of some of the things that I didn't think to do but would wish to do in future.

    Get policy document changed. Review the amount of money applied for to NSF.

    The call for participation:


    Get a delegate list out of the organisers so as to be able to target people to visit the doctoral consortium students' posters. AND so as to invite "notables" to meals. A list of paper presenters would do almost as well. These are best obtained a week or two before the conference, but prepare the organisers for this request.

    Better support for the poster presentations.

    Precirculate the students with the list of panel/faculty, plus their URLs. In fact start the doctoral consortium as a digital community by collecting everyone's web page in one place, plus the accepted students' documents (abstracts etc.) (perhaps by a URL to a page with all the documents and/or a big zip file for mass downloading and printing). Make it clear everyone should have read all the abstracts in advance.

    30 word descriptions required in the call, or anyway later. You will need them, but also probably good on a web page for everyone.

    Ask for a flipchart so you can have things like the talk running order permanently displayed.

    A student volunteer is not really essential, but nice in case running around has to be done for this and that.

    Summary: what to aim for and how to achieve it

    We might distinguish between what we want and how to get it; between: a) Aims, desired outcomes, requirements, b) Implementation, techniques, methods.

    The aims, major and minor, are summarised above in "Summary of the key requirements". The major methods are in effect embodied in "the CHI recipe for a doctoral consortium", also summarised above. All that remains is to summarise a few points about smaller scale methods that probably are important in achieving the aims.


    Web site logical path: [www.psy.gla.ac.uk] [~steve] [chi03] [this page]
    [Top of this page]