Apple’s multi-touch technology can go a long way towards basic operations on an iPhone or MacBook trackpad. However, a new Apple patent filing posted on Thursday shows that Apple intends to create a complete UI with the aid of secondary sensor data.

Previously, an Apple patent filing shows how Apple intended to create a multi-touch gesture dictionary based on "chords," various combinations of finger touches on the touchscreen. However, this may not go far enough in a complete UI, according to a new patent sought by Apple.

Credit: Unwired.com

"While the fingertip chording and movement data generated by multi-touch input devices can provide a strong set of user control means, additional information from other sensing modalities when combined or fused with the chording and movement data can significantly enhance the interpretative abilities of the electronic device and/or significantly improve the ease of use as well as streamline input operations for the user.

"Therefore, embodiments of the invention propose the concept of MT data fusion, which is defined as the combination of data from one or more independent sensing modalities with chording and movement data from a MT sensor in order to improve the operation and use of an electronic device," the summary of the invention says.

There are several different sensing modes described inclusing some aspect of the user’s body state (for example, facial expressions), the state of the electronics, voice input and a possible camera to identify the user’s fingers with respect to the touch screen.

The filing is complex, but generally seeks to invoke both a measurement of the device’s surroundings to enable a complex interpretation of the handling of the device. Also discussed is the ability of the system to sense frustration on the part of the user, again via facial expressions, and adapt to errant behavior of the user.

This patent filing suggests that Apple is on a long term effort to leave the traditional interfaces of PCs and Macs behind and envisions a future of personal, portable, touch screen devices, tied wirelessly to a data clound. Geordi LaForge would be very proud.