In Blade Runner (1982), Roy Batty, the leader of an outlaw group of Nexus-6 replicants, undertakes a quest to “meet his maker”. When he encounters Chew, the genetic designer of his eyes, he mocks: “Chew, if only you could see what I’ve seen with your eyes.”
The exciting news is that eye-tracking technology now allows us to objectively track and record what viewers are looking at, for how long, and in what ocular sequence.
Eye trackers work by reflecting infrared light onto the eye, which is then captured by a sensor. The sensor allows the captured movements to be tracked and specialised software overlays these movements over what was being viewed.
The eye fixates when gathering information and move between points in saccades (fast eye movements) (see image 1 below).
Where we attend to, and what draws our attention, is of interest as this process is driven both by our conscious intentions and by unconscious reactions to visual stimuli such as objects, textures, and movements.
Looking at gaze behaviour can hint at the cognitive drives behind them. Even more interesting is the ability to look at the gaze behaviour of groups of people in relation to similar stimuli.
This is made possible through the use of heat maps (see image 2 above), which effectively show the weighting of all the viewing occurring and gaze plots, which show the location of the fixations made as well as the sequence in which they were made (see image 3 below).
What we’ve done
In our recent studies we used a Tobii x120 eye tracker to record 12 subjects watching scenes from three films and analysed their fixation data using Tobii Studio Software.
We are part of a group of multidisciplinary researchers that together form the Melbourne-based Eye Tracking and the Moving Image group. This group has recently been analyzing data collected from three films:
- UP (Docter, 2009)
- Monsters Inc. (Docter, 2001)
- Saving Private Ryan (Spielberg, 1998).
Data for this study was collected and early analysis of fixations completed with a focus on narrative development, characterisation, and sound in relation to gaze patterns; further in-depth analysis is planned over the next three months.
The three films were chosen because they had very strong and directed storylines, interesting soundscapes, and allowed us to consider whether there were differences in the way live action and animation might be viewed.
The fact that two of the films were made by the same director is also of real interest, although not the focus of the research presently.
In terms of UP we are looking at what the eyes track and focus upon as viewers watch the film’s opening montage sequence:
This is where Carl and Ellie, the two central characters, get married, fall deeper in love, and live out their lives together, until Ellie eventually dies. It is a short but emotive sequence, set to a subtle musical score, but without a word of dialogue spoken.
We are finding that hot spots emerge around the character’s mouths, as if our viewers are conditioned to look for identification through the way a film’s central characters “speak”, and are searching for narrative clarification through dialogue exchanges that actually never emerge.
We are also finding that objects and motifs that actively move the story along and define character mood are picked out of the mise-en-scène and gazed at even when the scene is fluid and action is taking place across the filmic space (see image 3 above and 4 below).
Our eyes, then, seem to be active in finding emotive objects even when they are found in a busy scene. There is a caveat here of course: the “extra” attention to significant objects may well have been because the scenes were free of dialogue.
In terms of both Monsters Inc. and Saving Private Ryan, we created a control condition in which our viewers watched the two scenes with and without sound. Both scenes were “action” sequences, although the Ryan Beach Landing scene is hyper-realistic, infused with photo-realism and documentary-style aesthetics:
So far our results have been a little inconclusive with only slight variance in where viewers look when the scenes are played with and without sound. But, with regards to Monsters Inc. we have found when the sound is off, viewers actually migrate slightly away from the character’s mouths to focus on their eyes, and to objects that are pregnant with narrative information, such as the red warning light on the door (see images 5 and 6 below).
This, we suggest, is because the viewer is searching for non-verbal signs to confirm what might be taking place and because the eyes, culturally speaking, are where “truth” is to be revealed.
On one level this contests our results for UP, where the mouth was gazed at; on another, it confirms the way that impressionable objects are intensely gazed at.
Briefly, with Saving Private Ryan we found that when the scene was played without sound, gaze patterns slightly migrated to objects that could be predicted to be key to what might happen next.
That is, there was a greater level of prediction when the sound was off, as viewers searched for visual clues to where the action might “explode” next.
For both Monsters Inc. and Saving Private Ryan we measured pupil dilation, and we found that there was a close correspondence between the presence of sound and larger pupil sizes.
That is, when various elements of sound design were present, pupil enlargement indicated that the viewer was experiencing an increased emotional response, giving weight to the importance of sound in the affecting power of cinema.
So, what have we seen with your eyes? The power of your insight, the way you search for meaning and follow narrative and aesthetic clues, the importance of sound – of hearing and seeing to not only where you look but also to how you feel when you do so.