Crossmodal Attention and Multisensory Integration:
Implications for Multimodal Interface Design
One of the most important findings to emerge from the field of cognitive psychology in recent years has been the discovery that humans have a very limited ability to process incoming sensory information. In fact, contrary to many of the most influential human operator models, the latest research has shown that humans use the same limited pool of attentional resources to process the information impinging on each of their senses (e.g., hearing, vision, touch, smell, etc). As such, the decision to develop multimodal interfaces that stimulate more than one sense need not always be beneficial in terms of improving human performance, for example, in terms of avoiding sensory overload. Instead, interface designers should realize that the decision to stimulate more senses actually reflects a trade-off between the benefits of utilizing additional senses and the costs associated with dividing attention between different sensory modalities. In this presentation, I will discuss some of the problems associated with dividing attention between eye and ear, as illustrated by talking on a mobile phone while driving. I also will highlight some of the important but understudied issues related to the synchronization and desynchronization of input to each of the senses. I hope to demonstrate that a better understanding of the senses and, especially the links between the senses that have been highlighted by recent cognitive neuroscience research, will enable interface designers to develop multimodal interfaces that more effectively stimulate the user’s senses.
Biography
Charles Spence is a University Lecturer in Experimental Psychology at Oxford University. His particular research interests lie in using cognitive neuroscience techniques to both identify and understand the information-processing limitations that constrain our ability to process the inputs from arriving from each of our senses (e.g., hearing, vision, touch, smell, etc). His research calls for a radical new way of examining and understanding the senses that has major implications for the way in which we design everything from household products to multimodal user interfaces.
Charles is a consultant for a number of multinational companies advising on various aspects of multisensory product design, and he has also conducted human-computer interaction research on the Crew Work Station on the European Space Shuttle. Charles is currently researching the attentional problems associated with using mobile phones while driving, as well as issues related to the synchronization/ desynchronization of different sensory inputs.
Charles has published more than 70 articles in top-flight scientific journals over the last decade. He has received both the 10th Experimental Psychology Society Prize and the British Psychology Society: Cognitive Section Award in recognition of his achievements. This year, he is the recipient of the Paul Bertelson Award, recognizing him as the European Cognitive Psychologist of the Year.