Time-of-flight cameras detect driver’s gestures, aid in autonomy

  • 04-Sep-2014 08:53 EDT
aetrMelexisTof.jpg

Melexis and SoftKinetic have teamed up to create time of flight cameras that can detect driver gestures.


Two major design trends, the push to provide more exotic human-machine interfaces (HMIs) and the march towards autonomous driving, are sparking interest in cameras not now used in vehicles. Time-of-flight (ToF) cameras provide gesture recognition for HMIs and tell autonomous controllers whether drivers are watching the road with their hands on the wheel.

ToF cameras work much like sonar or radar, sending out a light beam and measuring how long it takes for the light to return. That lets the camera build a 3D model. Sensors are capable of understanding subtle human gestures as well as the shape, size, and behavior of objects and people inside the car. The technology is getting increased interest from automotive developers, particularly those creating HMIs.

“Comfort will be the entry point,” said Gaetan Koers, Melexis’ Product Line Manager. “If the driver’s or passenger’s hand moves toward the HVAC or radio control, the system will provide feedback. Gestures like turning a knob can be used to adjust the temperature or radio volume.”

Melexis recently unveiled an automotive grade MLX75023 ToF sensor that’s bundled with software from SoftKinetic, a company that specializes in 3D vision and gesture recognition solutions.

“SoftKinetic developed a library that’s tuned to the automotive context, which is quite different than other consumer markets,” Koers said.

The linkup is yet another step for the merger of consumer and automotive technologies. ToF cameras for consumer applications are made by Texas Instruments and STMicroelectronics, which could easily focus on automotive uses. Along with reliability issues, usage in bright sunlight is a primary design concern for transportation cameras.

“The challenging part is to work in high levels of sunlight,” Koers said. “We’re at 120 kilolux, which is the amount of light you get at noontime in the Sahara.”

However, that’s only one of the challenges that must be met. Gesture recognition is still in its early stages. Much like voice recognition at a similar period, problems raise questions about gesture recognition’s role as an automotive technology.

“The jury is still out,” said Ian Riches, Global Automotive Practice Director at Strategy Analytics. “The second-generation Kinect sensor supplied with Microsoft’s Xbox One console uses a time-of-flight camera for its range imaging, but overall impressions seemed mixed as to the usefulness and accuracy of this particular implementation. Strategy Analytics does not see ToF entering automotive until the long-term, 2020 at the earliest.”

If that timetable plays out, the growing role of autonomous systems could be another driver for usage. In the initial phases of autonomous driving systems, vehicle control modules will have to determine whether the driver is distracted or actively involved in driving. Many systems are expected to activate advanced driver assistance systems more quickly when they see that the driver isn’t paying attention.

“Before the vehicle takes over something like braking, you may want to know if the driver is looking at the road or at the passenger or the radio,” Koers said.

He has a more bullish outlook on ToF’s adoption than Strategy Analytics’ Riches.

“We expect projects to reach production next year,” Koers said. “This is mostly driven by European OEMs. A lot of European carmakers feel cameras are limiting their ability to roll out more advanced ADAS systems and more advanced cockpit designs.”

Share
HTML for Linking to Page
Page URL
Grade
Rate It
4.00 Avg. Rating

Read More Articles On

2017-01-11
An ultralight door architecture nets a 42.5% weight savings compared to a current production door, and that's enough to put this lightweight concept, developed via collaboration, in an enviable position.
2016-11-28
If there’s any doubt that connectivity is the next wave for advanced features and functions, it should dissipate after CES 2017. A multitude of advances in over the air updates and security will be shown in Las Vegas in January, setting the stage for much of the auto industry’s technology rollouts throughout the year.
2016-11-28
Human-machine interfaces are changing rapidly and consolidating as radio head units handle more functions and connect to diverse systems inside and outside the vehicle.
2017-01-03
Development of advanced automotive technologies is increasingly being driven by suppliers, rather than OEMs. And megadeals worth at least $500 million that are behind a growing number of next-gen technologies may reach the highest level since before the Great Recession. That’s the view from Las Vegas, as the 2017 CES (formerly the Consumer Electronics Show) opens to over 160,000 attendees.

Related Items

Training / Education
2010-03-15
Technical Paper / Journal Article
2010-10-25