Reinventing the museum experience with eye-tracking

The customer experience in museums is mostly the same for everyone. It is linear. Our major cultural institutions do not propose differentiated customer experiences. The emerging use of Big Data in the museum space makes it possible to apprehend all the variations in visitor behaviour and to glimpse ways of improving it. In this article, I develop the idea of using eye-tracking to understand better how art is consumed and to propose new visitor experiences.

Summary

• Introduction
• Use of eye-tracking (oculometry) to understand how the Works of Art are viewed
• Results of research in oculometry applied to Art
• How to use eye tracking in the museum space
• Perspectives for the museum visitor experience
• Sources

Introduction

In a previous article I reported on the use of Big Data in art museums. The finding was bitter because when data is generated, it revolves mainly around the works themselves. Visitor data is sorely lacking.
The megadata research programme launched by the Louvre is helping to define new perspectives through the collection of data on visitor behaviour (see this research on visitor movements inside the Louvre).
After having sketched out how existing museum systems could contribute to the analysis of visitor behaviour, I wanted to go a step further. How can we understand the visitor’s relationship with a particular work? This is the question that led me to think about the applications of oculometry.

Painting is writing. Painting makes us see the outside world.

Daniel-Henry Kahnweiler

Understanding the visitor’s perception of the Work of Art through eye-tracking (oculometry)

Art has undergone exciting developments over the centuries. Modes of representation have changed, and new forms of creation have exploded in the last 100 years. The aesthetic revolutions initiated by individual artists have sometimes led to incomprehension on the part of their contemporaries. We will thus remember the scandal that the impressionists had unleashed at the Salon of 1863. The aesthetic shock and the resulting incomprehension were the subjects of heated debate at the time. This situation is wonderfully summed up by Kahnweiler, the merchant who put the Cubists in the spotlight and who discovered Pablo Picasso and Juan Gris (among others).

Painting is writing. Painting makes us see the outside world. It creates the external world of men, and when this creation is new when the signs invented by painters are new, this discomfort is born, this conflict that makes people see objects, but no longer see them at all as they are accustomed to seeing them.

Daniel-Henry Kahnweiler in “My galleries and my painters”

Art is only a way for the artist to materialise his reality. It is necessary to appropriate the artist’s writing to understand his interiority. This appropriation takes time. It requires from the art lover, the museum visitor, an attentive, repeated observation, to penetrate the essence of the Work of Art and thus approach the truth of the artist. To help the museum visitor grasp this reality, to penetrate the mysteries of the work, it is necessary to understand how he consumes it. This is where eye-tracking could play an exciting role.

When Picasso painted the Demoiselles d’Avignon and initiated the cubist movement, amateurs were taken aback. This new way of writing painting was not yet understood.

How do our eyes perceive a painting?

Research such as that of Quiroga and Pedreira (2011) has shown that some regions of the paintings invariably attract the eye’s fixations. In this research, the authors made modifications to the works to analyse the path of the eyes before and after modifications. They show that the removal of even minute details can lead to significant changes in the way the eye moves around the surface of the painting. One of the most exciting conclusions, however, is that seeing the work the first time will influence the way you look at it the second time. The authors write:

the perception of art is a very complex process conditioned by factors at different levels. On the one hand, there are basic visual principles, such as contrast and saliency, which introduce some uniformity in the gazing pattern of different subjects by driving the attention to particular areas and, on the other hand, there are also more complex cognitive factors, such as previous experience and knowledge, which introduce a large variability across subjects.

Where does the eye concentrate when looking at different Works of Art? (Source)

Other research has focused on the place that can exist the dynamic and static aspects of the objects depicted in the paintings. A vibrant image (a character in action, see below) leads to more pronounced and more prolonged fixations.

Other research has investigated the role of external factors in the work. Ambient light thus influences the way the eye travels across the canvas (Bhattacharjee and Pal, 2018). But who would have guessed that our character affects the way we look at a painting? Research by Italian researchers shows that our empathy influences the way we look at a work of art.

Implementing eye-tracking in the museum space means giving oneself the means to understand how visitors decode the works and to use this knowledge to change how they are explained.

The use of eye-tracking within the museum space

On the one hand, eye tracking has been used for decades in marketing (see an example of an application from Twitter here) and for more than 20 years in the field of Art.
On the other hand, experiments have shown that museums can reinvent their customer experience by using the megadata they are likely to generate from large visitor flows.

Rather than confining eye-tracking to theoretical research, it is time to bring this technology into the museum space to understand how Works of Art are consumed.
Implementing eye-tracking in the museum space means giving ourselves the means to understand how visitors decode the works and to use this knowledge to change the way they are explained entirely.

Isn’t it paradoxical that the explanations given to visitors are all the same? Whether on the cards that accompany the works, via audio guides or digital applications, nothing is done to offer a personalised experience to the visitor. However, expectations and visitor experience vary according to the visitor’s profile. The expectations of Chinese visitors are fundamentally different from those of a loyal visitor, for example. Knowledge of a Work of Art also changes the way it is read (Massaro et al. 2012).

Although eye-tracking is now a technique that has been well mastered, its use outside the laboratory poses several challenges. Measuring equipment must be calibrated, and precise distances must be respected to collect reliable data. The space in the museum does not lend itself to this type of measurement. But the revolution may be closer than we think. Tomorrow’s visit could be made without continually having to look away from the work to consult a screen or an explanatory cartel. The Church of St. Peter in Leuven (Belgium), a Mecca for medieval art, proposes a unique visiting experience in collaboration with Microsoft. The visitor is equipped with HoloLens 2 glasses that allow the superimposition of information thanks to augmented reality. From there, to see in these new glasses, a medium for the analysis of eye movements is only one step that I will not hesitate to take.

The museum experience of tomorrow, more educational and significant than ever before

The use, in the museum, of technological tools such as connected glasses makes it possible to envisage radically different and much more effective experiences from an educational point of view.

On the one hand, unlike current visitor media (digital applications, cartels), connected glasses make it possible not to look away from the artwork. This is a significant advantage.
I have always found the current arrangements extremely frustrating:

• Reading the explanations on an application forces you to look at your smartphone rather than at the artwork.
• Listening to a description with an audio guide can be confusing because you don’t always know what the explanation is about.

On the other hand, connected glasses could, in the long term, make it possible:

• to collect data on the visitor experience (where did the visitor go and how, how long did he stay, what digital content was consumed)
• to collect data on the very act of consuming the artwork (how did the eyes look at the painting? On which parts did the gaze stop? How long was the artwork contemplated)?

Imagine what the combination of these two assets could mean for the museum experience of tomorrow:

• an in-depth understanding of how works of art are consumed
• explanations taking into account the consumption pattern
• descriptions in augmented reality, superimposed on the gaze, allowing the visitor to be accompanied in his ” enjoyment ” of the work
• explanations adapted to each person (imagine how augmented reality could be used for children to captivate them and immerse them in the works)

These perspectives are innovative and promise to offer a credible alternative to the traditional guided tour. The museums would also be able to increase the loyalty of their public by making the visit more enjoyable and didactic by offering them an evolving and personalised experience.

Sources

Bhattacharjee, A., & Pal, S. (2018, July). Attention of viewers while viewing paintings depends on different types of exhibition light: A quantitative approach with eye-tracking method. In International Conference on Applied Human Factors and Ergonomics (pp. 328-336). Springer, Cham.

Massaro, D., Savazzi, F., Di Dio, C., Freedberg, D., Gallese, V., Gilli, G., & Marchetti, A. (2012). When art moves the eyes: a behavioral and eye-tracking study. PloS one, 7(5), e37285.

Quian Quiroga, R., & Pedreira, C. (2011). How do we see art: an eye-tracker study. Frontiers in human neuroscience, 5, 98.

Villani, D., Morganti, F., Cipresso, P., Ruggi, S., Riva, G., & Gilli, G. (2015). Visual exploration patterns of human figures in action: an eye tracker study with art paintings. Frontiers in psychology, 6, 1636.

By Lorène Fauvelle