New Techniques for Evaluating Innovative Interfaces with Eye Tracking

Sandra Marshall

San Diego State University and EyeTracking, Inc.

    Computer interfaces are changing rapidly, as are the cognitive demands on the operators using them.  Innovative applications of new technologies such as multimodal and multimedia displays, haptic and pen-based interfaces, and natural language exchanges bring exciting changes to conventional interface usage.  At the same time, their complexity may place overwhelming cognitive demands on the user. As novel interfaces and software applications are introduced into operational settings, it is imperative to evaluate them from a number of different perspectives.  One important perspective examines the extent to which a new interface changes the cognitive requirements for the operator. 

    This presentation describes a new approach to measuring cognitive effort using metrics based on eye movements and pupil dilation.  It is well known that effortful cognitive processing is accompanied by increases in pupil dilation, but measurement techniques were not previously available that could supply results in real time or deal with data collected in long-lasting interactions. We now have a metric—the Index of Cognitive Activity—that is computed in real time as the operator interacts with the interface.  The Index can be used to examine extended periods of usage or to assess critical events on an individual-by-individual basis.

    While dilation reveals when cognitive effort is highest, eye movements provide evidence of why.  Especially during critical events, one wants to know whether the operator is confused by the presentation or location of specific information, whether he is attending to key information when necessary, or whether he is distracted by irrelevant features of the display.  Important details of confusion, attention, and distraction are revealed by traces of his eye movements and statistical analyses of time spent looking at various features during critical events.

    Together, the Index of Cognitive Activity and the various analyses of eye movements provide essential information about how users interact with new interface technologies.  Their use can aid designers of innovative hardware and software products by highlighting those features that increase rather than decrease users’ cognitive effort.

    In the presentation, the underlying mathematical basis of the Index of Cognitive Activity will be described together with validating research results from a number of experiments.  Eye movement analyses from the same studies give clues to the sources of increase in cognitive workload.  To illustrate interface evaluation with the ICA and eye movement analysis, several extended examples will be presented using commercial and military displays.

    [NOTE: Dr. Marshall’s eye tracking system will be available to view at Tuesday evening’s joint UIST-ICMI demo reception]

Biography:

Dr. Marshall is President & CEO of EyeTracking, Inc. and Professor of Psychology at San Diego State University.  Her research in cognition and assessment has received federal funding for the past twenty years and has had important theoretical and practical impact.  Early research on problem solving culminated in the book Schemas in Problem Solving. Her recent work has focused on the use of eye tracking to understand cognitive activity in training and performance on military simulations. In research sponsored by the U.S. Office of Naval Research, the U.S. Air Force Office of Scientific Research, and the U.S. Defense Advanced Research Projects Agency (DARPA), Dr. Marshall developed new methods for assessing cognitive strategies and cognitive workload based on eye measures.  The techniques are now being used to evaluate interfaces for military and non-military applications.