Facing up to robots

This work has captured the public imagination and helped to raise awareness of computer science research to the public and students in schools. The research has also sparked public debate over the uses of artificial intelligence, and raised awareness of the issues involved via widespread media coverage and through targeted smartphone apps.

Our Impact

Impact on Public Engagement

This research led directly to a series of thought-provoking and informative ‘public engagement’ activities designed to inspire the next generation of science researchers as well as inform the general public. A range of approaches have been deployed, from traditional to innovative face to face events, to the use of apps and social media.

Multimedia and Apps

McOwan’s research into face recognition informed the production of a short film ‘Why faces are special’ (Black and McOwan, 2012).  Designed to be accessible to a wide audience, the film was selected as one of the finalist 55 from 1450 films submitted to the festival CERN CineGlobe film festival 2012. Available on YouTube, the film has been watched 2,227 times (January 2013).

A special edition of cs4fn (www.cs4fn.org), McOwans project to promote computer science research in schools, describing both the facial perception elements and the affect recognition system was developed as part of the cs4fn project as support for the Royal Society Summer Exhibition (http://www.cs4fn.org/faces/). The cs4fn website gains 14 million hits, 750,000 visitors per year. Industry has supported this engagement work including the largest grants given by Google’s CS4HS programme totalling 108K in the period 2008 to 2012 

Face to face events

Embedding engagement in real and constructured social environments 

Pogoing robots at the ICA (Institute of Contemporary Arts) in London, July 2006. This unique event, in partnership with digital artists’ soda.co.uk, involved three large pogo dancing robots as an integral part of the audience at a live musical gig. The robots were trained to respond, dance, depending on their musical tastes. McOwan was present to discuss issues around computational modelling of neuroscience, robot embodiment and emotion, with the diverse audience of concert goers and the media. He was responsible for the implementation of the live stage visuals showing the audience how the robots were processing the music and deciding to dance.

Guerrilla Science Blade Runner film recreation event (July 2010). This interactive dialogue with the public on computer image perception and social robots took place over 6 days on a reconstructed set from the film Blade Runner in London’s Canary Wharf and engaging with 7,000 people over the 6 days. The novel juxtaposition of current research with popular science fiction, mediated by a team of appropriately costumed researchers proved to be provocative and lead to over 1000 people taking our test and engaging in open discussions.

Underpinning research

This work combines contributions at QMUL from the Cognitive Science group and the Computer Vision group. Computational modelling of human facial perception, in particular the examination of the face space hypothesis was undertaken as a continuation of a long standing collaboration with Psychology at UCL predominantly through the EPSRC dynamic faces project. This research developed novel methods for the creation and manipulation of photorealistic avatars. The research focus was on the development and utilisation of new tools for the extraction of facial motion and mapping expressions between faces. One aim was dynamic 3D motion capture without using existing noisy time of flight technologies or restrictive structured light approaches. Faces vary in colour as well as image brightness but the natural colour signal is not used effectively in image motion or stereo algorithms We developed a new approach to image motion analysis that characterised the bright-dark, yellow-blue and red-green opponent channels of the human colour system as chromatic derivatives. We incorporated chromatic derivatives into our existing spatio-temporal brightness derivative method for motion and binocular disparity calculation and demonstrated improved performance. 

The prime motivation of the computer vision work was to build computer vision tools that could be used to develop new methods for studying the perception of facial motion. A major aim was to generate a photorealistic average avatar with which to separate out the motion of the face from its form. This was achieved using 2D image-based performance-driven animation. We constructed a photorealistic avatar using Principle Components Analysis (PCA) over vectors encoding the differences between single frames of movie sequence and a reference frame. This can deliver an expression space for a given person.

We examined the psychological validity of a PCA-based expression space. By adapting to facial images at the ends of a particular dimension of facial variation (e.g. the first principal component) we could shift the appearance of expressions away from the adapting expression but did not shift perception of faces arrayed along a second orthogonal direction. This showed adaptation within expression space and that images which were statistically orthogonal were also perceptually orthogonal. The idea that faces are represented as relative to a mean face, which has become the standard view in face perception, raises questions about over which set of faces is the mean constructed. We built PCA spaces across individuals rather than across expressions to investigate “family resemblance” between different classes.

We used a novel technique of mapping a vector representing a deviation of a male face from the male mean into a female face space. This resulted in a female “sibling”. We showed that the “sibling pairs” looked more alike than a random pairing indicating “family resemblance” may be encoded by similar vectors referenced to the average of classes of faces. The same technology can be used to visualise our prejudices. We found that average Conservative and Labour MP’s faces were indistinguishable. However average faces rated as strongly labour or strongly conservative did look distinctively different and were correctly matched to their stereotypical category by participants in a follow-up experiment.

Insights from this work for example dynamic areas of particular importance in processing fed into the development of robust facial expression and affective intent prediction technologies which formed QMUL’s contribution to the EU funded IP Lirec (Living with Robots and Interactive Companions) which examines the requirements for socially meaningful long term interactions between humans and robots in real world social scenarios.The research explores for example ‘affect sensitivity’ identifying the affective states of humans and linked non-verbal behaviours. It also identifies limitations and challenges arising from the design of an affect recognition framework in a real world scenario where an iCat robot plays chess with children.

Both of these projects had, by design, specific public engagement strategies embedded from the start, which were further and successfully amplified through the EPSRC PPE project Computer Science for Fun (based at QMUL) and McOwans QApps project www.qappsonline.com. Curzon and McOwan’s cs4fn project provides a successful strategic framework to do high quality public engagement through writing accessible articles about research to create engaging stories, while QApps provides a portal for promoting research based smartphone apps.