Eye-Tracking Emerges: Look Over Here, Your Computer Knows Exactly What You Want

- Feb 25, 2019 -

We’ve all been there. Racking our brain for the name of someone we just met and can picture in our mind. New kinds of human-computer interactions like eye-tracking could mean the end of these frustrating needle in the haystack searches.

They also herald a significantly changed experience for employees going about daily tasks. Many experts believe eye-tracking is part of the next revolution in human-computer interactions, and I watched an example in this VIDEO interview at SAP TechEd.


“Eye-tracking is among the technologies we’re evaluating to revolutionize the enterprise experience,” said Jan Schaffner, head of the SAP Innovation Center Network at that time. “The focus is on people, not the computer. We can use eye tracking to make SAP workplaces more efficient and productive.”

Gaze-based searches replace keywords

By tracking someone’s eye movements, this software can predict in the moment what they’re looking for next, automatically surfacing the most relevant items and images on the screen. Vision technologies are included in what IDC researchers called “affective computing” that’s predicted to rise by 25 percent in real-world applications by 2022. That’s the same year Gartner predicted 70 percent of enterprises will experiment with immersive technologies that incorporate expanded sensory channels, allowing systems to detect emotions based on facial expressions.

The demo I saw showed how quickly someone could search through a database of 5,000 faces to find a person in less than two minutes – just by their eye patterns. It’s part of a co-innovation project between Germany-based enterprise software startup 4tiitoo and the SAP Innovation Center Network.

“When you search for anything, your eyes are constantly scanning, while your brain subconsciously filters out what’s relevant,” said Stephan Odoerfer, founder and managing director at 4tiitoo. “Our software platform NUIA can detect those patterns based on AI, even if we can’t necessarily put them into words. The eye tracker sees where you’re looking on the screen – a person’s hair color, gender, age, any characteristics – and constantly calculates your interest levels. As it senses what’s most important, it automatically scrolls and rearranges search results in synch with your eye movements.”

Practical benefits of mouse-less future

Eye-tracking represents a new frontier in the future of search, encompassing ecommerce at-large, as well as company emails and other documents. There are also much larger potential benefits for companies using SAP S/4HANA for a variety of applications including accounting, planning and procurement, as well as CAD for construction and product lifecycle management.

“Many people in various departments tend to complete the same forms over and over, inputting different information,” said Schaffner. “While it’s impossible to automate all repetitive tasks, we can embed technologies like eye-tracking to assist humans. Rote activities become easier, reducing errors and allowing employees to spend more time on higher-level responsibilities.”

Odoerfer agreed on the pluses of hands-free computer interactions. “We’re looking ahead to a mouse-less future for standard SAP input screens,” he said. “All someone will need to do is look at particular fields, and our software automatically activates the field so they can type where they want. If someone wanted to use the mouse, the software would still save steps by moving the mouse cursor to the projected field, button or link. This makes daily work more efficient.”

The use cases in call centers, factories and elsewhere are unlimited. Consider workers in manufacturing facilities who check bottles for imperfections before filling them with liquids. It’s a demanding manual task that requires exchanging people every 15 minutes.

“Eye-tracking could help workers detect defects faster and measure their own stress levels before fatigue causes mistakes. The software could also collect data on defective bottles in SAP S/4HANA systems, improving operations for transparency company-wide,” said Schaffner.

We are just at the beginning of what promises to be a revolutionizing moment for human-computer interactions that’s certain to bring completely new experiences to every organization.

This blog also appeared in the SAP News Center.

Follow me on TwitterSAP Business Trends, or Facebook. Read all of my Forbes articles here.