Eye Gaze Correction for Video Conference

The Invention

An algorithm has been developed that can detect the eyes of a video photographed subject. The eyes can then be artificially placed so that they are looking at the camera irrespective of the actual direction the eyes may be directed.

The Need

Video conferencing is a popular method of communication, whether through IP or using cellular phone technology. In either case, in order for the subject to seem to look at his interlocutor, he must be looking into the web camera. However, the camera is usually positioned above or to the side of the screen which the subject is looking at, resulting in an oblique eye angle and the disconcerting effect of talking to someone who is not looking at you. There is therefore a need for a tool which can manipulate the eyes so that no matter in which direction the subject is looking, the interlocutor will see his eyes directed at the camera. This would lead to a better video chat experience.

The Advantage

Other techniques rely on special additional infrared cameras to detect eye position before manipulation. They also attempt to artificially reposition the head which requires heavy CPU processing. Our algorithm relies only on the existing simple web camera and therefore does not require additional hardware. Furthermore, rather than attempting to reposition the user's eyes, our algorithm utilizes a pre-stored database of synthetic eyes which are seamlessly integrated to replace the subjects eyes.

Potential Applications

The software can be bundled with web cams or implemented onto thin clients to invisibly modify the images during video conferences. It has been tested with a variety of faces, eye angles and eye openings and has been found to give excellent results in all cases.

Project Status

Proof of concept demo software has been prepared.

Patent

Patent Pending

Project manager

Oren Calfon
VP Business Development, ICT

Project researchers

Lior Wolf
T.A.U Tel Aviv University, Exact Sciences
School of Computer Science