Apple has been granted a patent (via AppleInsider)
that describes a method for tracking eye movement and using it to
control UI elements on computing devices. The gaze-based control system
also includes methods designed to counteract visual effects that can
occur when a person stares at a fixed space for a long stretch of time,
which can trick their eyes into thinking some UI elements have
disappeared completely.The patent overall describes ways in which future Mac or iOS devices could use their built-in camera equipment to track user eye movement, and then to relate those movements to onscreen elements, like a cursor. The cursor would then follow a user’s gaze, in certain instances, and translate that to on-screen activities. The main purpose of the patent isn’t to describe the specifics of such a system, however – it’s instead meant to flesh out tech aimed directly at countering the so-called Troller Effect, which can result in certain UI elements (like the cursor) fading or disappearing if a user stares at another fixed point for too long.
Vision-based interface systems could have big advantages in terms of adding accessibility improvements to iOS and Mac devices – interfaces controlled by the eyes might make it easier to use computers for those without full use of their hands and fingers, for instance. It could also be an innovative feature for supplementary control while hands are occupied with other on-screen functions, or with accessories. A final use case might see Apple using it in some capacity with a future version of the Apple TV; the Mac maker did acquire PrimeSense last year, which helped build the Kinect for Microsoft, so it has likely come up with some kind of camera-controlled living room R&D.