(Back up to central page on the debate)
The discussions were timely as EPSRC is about to undertake a review of the Human Factors sector of the IT & Computer Science Programme, with a view to identifying the important research priorities for the future. The issues raised at this meeting are important in this respect and we would welcome comments from the community on what was discussed.
If you have any thoughts or comments, please send them to me:
Nigel Birch
IT & Computer Science Programme
EPSRC
Polaris House
North Star Avenue
Swindon
SN2 1ET
Email: nigel.birch@epsrc.ac.uk
Thank you.
Whilst our definitions of user are broad, and can include academic communities outside of IT and computer science, the industrial and commercial communities are obviously extremely important. If the results of the research we support in academe is failing to reach them, we need to know why.
Nigel Birch (Associate Programme Manger, Human Factors, EPSRC, and Chair of the session) introduced the session and posed the questions. He also outlined EPSRC's mission as a grant awarding body.
Ros Goldstraw (Head of Management, Psychology, Linguistics & Education, ESRC) explained the ESRC's role in the funding of HCI research and the position of Anne Anderson as the Cognitive Engineering Programme Co-ordinator. She informed the audience that the funding of HCI research would continue with the new Technology and People Programme.
The Panellists then proceeded to introduce, in pairs, the issues which they
believed relevant.
She outlined the ESRC's mission and how they are the largest funder of research and postgraduate training in social and economic issues. As had been pointed out in the keynote address earlier that morning from Darrel Rhea (Cheskin+Masten), these issues were becoming increasingly important for HCI and HCI researchers. She described the way in which the Council's research themes were identified after discussion with academics, industry and other research councils.
She then outlined the objectives of the Cognitive Engineering Programme which was aimed at trying to understand how people make decisions and from this, how to design systems that would provide more effective support.
She believed that the following issues all need addressing with regards to the problems encountered with transfer of research:
Charles Brennan stated that one of the main issues was the need to bridge the gap between research and application - this all came down to communication.
There was a need to provide ways in which to ease communication between academic and industrial environments. There was no single way to do this. Different situations called for different solutions. Some of the routes used by BT have been:
Successful collaborations and take-up of academic research therefore meant looking at the issues around the interface between the two organisation. Such issues as:
Tony Solomides, UWE, asked whether these last four issues weren't really a result of differences in value.
Charles Brennan agreed that there was a cultural difference and that the best way to overcome this was to try and take everything into account from the beginning.
Anne Anderson commented that academics were very good at gathering data and statistics - the clash came at the interpretation of the research and the analysis of the results, that academics were not good at producing hard recommendations applicable to a particular company as they did not understand the company's perspective. Charles Brennan agreed, adding that, because companies often tried to draw out firm conclusions from their academic partners by questioning the findings, this could lead to feelings of suspicion, that there was an implied criticism - this was not the case. Harold Thimbleby said that this `clash' is much less of an issue with engineering academics.
Karen Carr believed that there was a need to separate what research needed to be done from the issue of what needed to be done to transfer the results of the research into applications in industry. This was an important point as in many industries, different people are involved in these two processes.
In terms of what needs to be researched, comments were often made about rapidly changing goals. This is because the people, involved in making these decisions are at the sharp end of the business and are focused on the short term projects and things that change in response to the outside world. These people were not the same as those who occupied positions where overall strategic directions for the company were important.
Once a research project had been agreed, the results needed to be fitted into a process where basic research was acquired by the company and translated into a more applied direction (matching the research onto the company's strategic plan). From here the results were fed closer and closer towards shorter term projects, where a technology demonstration needed to be shown to convince the company that the results of the research are indeed relevant.
Within the process research could either be applied to products - basic research leading to new products, or, in providing new tools. Both kinds of research needed to reach a particular level before it could be sold to those people involved in the short term research. Whilst it was possible to interest the strategic planners in a project, they would not become involved in overseeing the company's involvement. Any collaboration would be with those involved in shorter term projects which would need convincing of the value of the work.
Karen Carr believed that the need was to focus on the middle men between basic research and industrial applications. There were various ways in providing these `middle men'. One way was industry providing corporate research centres, like the Sowerby Research Centre at British Aerospace. The centre acted as the liaison between academics and the various business units in BAe. Many of those working in these business units do not know what they want in 5 to 10 years - they expect the Centre to tell them. The Centre thus needs to maintain links with academe to keep track of new developments that might be useful to BAe. The centre then tries to take some of this research and develop it into a format that would be more useful to the company.
BAe is lucky in being able to afford something like the Sowerby, and centres like it were only found in large companies (BT was another example). There were other ways of providing these links, however, which were of relevance to smaller companies:
a) Academics specialising in particular industry areas
b) Academic equivalents of trade associations (as mentioned in Technology Foresight) - a forum where academics were represented and could interact with the kind of people who were in industry Trade Associations. This could be a good way of identifying strategic directions.
c) Case studies and collaborative work was very useful as it showed the business units the value of the work and allowed the provision of feedback to the researchers.
Jon Oberlander considered that the major problem was that not everyone spoke the same language and there was an analogy between the problems of linking academe and industry with those of the communication problems between nations speaking different languages. The analogy extended to suggesting possible solutions.
Firstly though, there were solutions in the language domain that don't work: you could not force one side to speak the other's language. In research terms, this meant that industry should not expect academe to carry out short term research. Universities are not well placed to do this and the effects would be detrimental in the long term.
There were some solutions that might work, however:
a) Encourage the establishment of management consultancy style intermediaries (for example the people at Sowerby) who understood both sides. They would as interpreters not just translators.
b) Encourage Networking. There were various ways of doing this:
- Long term relationships founded on data sharing, which leads to an
understanding between the collaborators, of how the organisations work and
giving the opportunity to build a longer term relation with the original
partners and giving knowledge of others in the sector.
- Applied maths style workshops: intensive workshops involving groups of
academics and industrialists. The aim would be to have academics identify
possible uses for some of their `old' research and to gain an understanding of
current industrial concerns that could be tackled by new research.
- "emigrations" and exchange visits: moving people trained in academe in to
industry to work, as well as supporting shorter visits by academics to industry
and vice versa. The problem was finding appropriate places to visits. As an
isolated academic it was difficult to know who you might want to spend time
with.
He used the following slide to identify what he believed were the main issues:
| Research the social, pragmatic aspects of knowledge transfer |
| - specify bottle necks more clearly |
| - identify good practise |
| - help policy makers become more realistic about the kinds of communication that were possible |
| - need public, publicised studies of the real world problems |
Steve Draper, Psychology, Glasgow University asked Karen Carr if she could give an example of basic research going through into production.
Dr Carr replied with an example of auditory warnings. The university (which already had an established reputation) had carried out research into the number of signals created which eventually led to the Eurofighter. However, this could be a long process (15 - 20 years). There were also case studies, such as those in virtual manufacturing and design, which had assisted engineers.
Charles Brennan added that BT had just taken up Phil Barnard's (MRC Applied Psychology Unit) work done on communication in cognitive sub systems and this was being applied to BT's work on multimedia systems.
Another audience member asked whether there was scope for the use of appropriate technology for a smooth transition.
Jon Oberlander replied that the problem was not in relaying the individual messages - it was the actual getting of people together in a social context. Charles Brennan added that what would be useful was something that allowed you to identify those people in the other community with whom you had things in common.
Steve Draper was concerned that the discussion so far was missing the deep underlying issue, namely that neither academics nor industrialists could foretell what the outcome of any given piece of research might be. Yet, the whole discussion seemed to be based on the assumption that we could know in advance what the likely benefits of a piece of research were likely to be.
Jon Oberlander agreed that you couldn't foresee the outcomes, but by fostering open-ended relationships and closer contacts you were likely to identify opportunities for exploitation.
John Long observed that there is a need to make a distinction between the respective roles of industrialists and academics. The differences include both the outcome of their activities, ie their products, and the fitness-for- purpose of the latter. The product of academic research is knowledge, for example in HCI a method or a user model, and its test of fitness-for-purpose is the known ability of the knowledge to support design. The product of industry is an industrial or commercial artefact, such as a programmable washing machine or a software package for office administration whose fitness-for-purpose is to meet user requirements. If the academic research output is appropriate in general, it will be able to support design across a range of industrial companies, that is be generic with respect to companies. To be appropriate, academic research needs to take seriously the operational problems of companies, but the research problem selected should be general over companies. Academic research may need to work with individual companies to acquire knowledge such as HCI methods and models, but these should be generally applicable by companies and not just individual ones, or those involved in the original research. Academia offers industry knowledge intended to help solve some of the operational problems.
Dr Anderson agreed with this and restated her belief that the role of an academic was to generate and interpret data. The application of the knowledge was best done by the industry.
She summarised the difficulty in take up as being the organisational dimension to the industrial environment, which was generally not addressed by academic research:
There is also the need to continually defend your skills, as HCI was seen as a `soft science'. In summary, the intrinsic value of the research was not necessarily enough, there were the politics of the organisation and the business pressures that need to be addressed if new working practices or methods were to be adopted by a company. In addition to your research skills, you needed to be aware of budgets, to know how to make a business case, to keep track of events elsewhere in the organisation that will affect the take-up of any research (projects cancelled, senior management changes, etc.).
This may sound a pessimistic view, but she emphasised that academic research was needed, but it had to take account of these other pressures if it was to be successfully transferred into a industrial context.
Harold Thimbleby introduced himself as a computer scientist. He repeated the reason for having the Panel was the statement by an industrialist a couple of years ago at an HCI conference, that they did not use HCI. As all of us, including the Panel, had a stake in HCI, he thought it would useful to step back and choose some other area, say, quantum thermodynamics. An industrialist could say that they were not interested in this. But the academic researcher would counter by saying that they found it an interesting area to work in. However, the industrialist was more than likely using the knowledge in the company's manufacturing processes without knowing it. There was thus an educational problem in bringing home this realisation and it was clear that that quantum thermodynamics was appropriate for Research Council funding, both in terms of interest and usefulness.
HCI, however was not the same as an area like quantum thermodynamics - which was not out to change the world. HCI is about improving things and is thus a proactive discipline. It is also a public issue. Worrying about what industry was up to was too narrow. There were actually more general users out in the world, ordinary people, than there were industrialists able to take up the research. One of the goals of HCI should be, therefore, to motivate end-users (e.g. consumers) to make HCI decisions and discriminating arguments.
There were many examples of usability problems with products in general and with computer products in particular. In the case of computing, it seemed that people are happy to buy products that do not work, and moreover, they assume it is their fault if they do not work - as witness the willingness to buy upgrades.
In this sort of atmosphere, HCI was completely useless.
That's why EPSRC and ESRC should continue to fund it. They need to fund it at least until we have some principles, criteria and laws of HCI and indeed an ethical approach that meant that industry would pursue the stated goals of HCI, whatever those turn out to be.
Computer firms were not going to solve our problems. Their goal is to make money. A goal of HCI researchers should be to change people's perceptions.
A final thought: wasn't ironic that computer `windows' were the only windows you need training to look through!
John Long commented that academia, industry and Research Councils have a genuine and difficult problem as concerns HCI. The problem is the relationship between the specific and the general. For example, academic knowledge is supposed to be general, an HCI method or model which supports design. However, the knowledge can only be acquired from particular instances, and needs to be applied to particular instances. Industry is supposed to provide both kinds of instance, academia is supposed to provide the generic knowledge, and the Research Councils, via peer review, are supposed to assess the likely success of acquisition from and application to instances of general knowledge which does not yet exist. This three way relationship is complex and difficult to achieve in practice, because consensus examples of HCI generic knowledge, its instance acquisition and application have yet to be documented.
Nigel Birch responded that EPSRC does see the interaction between academics, industry and end-users as important and there was an ongoing discussion and consultation process within the IT & CS Programme to this end.
Charles Brennan commented that it was in Industry's interest to ensure that products were usable. However, the further out you go the more general the product became as the model of the `user' becomes more abstract.
Anne Anderson remarked that the Research Councils were not there to fund a company's specific problems. To be funded a project must be addressing a problem that can be disseminated into the public domain. It was the job of the company to exploit and apply the work to their own ends. However, where there were problems originating in an industrial environment that did have a general application, joint collaborations between academics and industrial researchers could be considered.
The question was asked "How did the Panel feel about getting a matrix on comparative products - selling usability rather than features?". Professor Thimbleby stated that he did not think this was possible until there was a change of culture and people started to complain, as at the moment it did not matter if people complained. If there were obvious standards, then complaints would be effective. Anne Anderson made the comment that the Research Councils should be funding the methodology. Lesley Trenner agreed with this statement but said people have to be aware of the alternatives.
Discussion then followed on what responsibility the Universities had in training - should it be research, research with some job training or exchanges with Industry. Charles Brennan commented that he believed it was important to build the relationship with the Institution not the individual.
Steve Draper made the final remark in the discussion by remarking "that the attempt to do a project then realising your preconceived idea was incorrect in practise happens too often - the bottom line is encourage basic research".
To conclude the meeting, Nigel Birch summed up the discussion saying that it seemed the three messages were that the Councils should fund basic research, that users needed to be involved, and that we should all complain more. He thanked the Panel for their efforts in preparing for the session and the audience for participating in an interesting discussion.
Report compiled by Michelle Kennedy and Nigel Birch.