Design a Low-Cost Gesture-Sensing System for Automotive and Industrial Applications

The human machine interface (HMI) is a key component in almost all industrial and automotive applications. As interactions become more complex, traditional switch designs are not suitable for many environments: they are counterintuitive and distract the operator from other tasks such as controlling a machine or driving a vehicle.

While there are many options for gesture-based control, designers are challenged with balancing performance and accuracy with cost and simplicity, as well as power consumption.

This article will discuss the components needed for a gesture control system based on infrared sensing technology. It will then introduce a low-cost sensor evaluation board and its associated software to aid in rapid development and reduced time-to-market.

Contactless sensing applications

The first everyday application of contactless sensing was to detect the approach of an object and trigger an appropriate response. Automatic doors, soap dispensers, porch lights, and hand dryers all use contactless sensors as inputs.

The proximity sensors in these systems act as switches to provide a digital on/off output, while gesture sensing allows a user to employ three-dimensional movements to control a system in an intuitive manner.

Gesture control also allows a user to make changes within a large active field, allowing gross movements to replace finer ones. This capability is a key feature in video gaming systems that employ gesture sensing technology to detect complex sports related movements, or track several players.

Infrared gesture sensing technology

Video game systems use lasers and specialized cameras to sense user activity, but they are too complicated and expensive for many applications, such as an automobile. Here, changing a radio channel, for example, requires only a simple left or right movement of a hand (Figure 1). Similarly, adjusting the volume can be accomplished with an up or down swipe, or a push in or out motion.

Figure 1: A gesture sensing system uses intuitive movements to simplify machine control. (Image source: BBC/BMW)

Due to its ability to detect a variety of simple gestures at a low cost, infrared (IR) sensing technology is a good match for many industrial, consumer, and automotive applications (Figure 2). The hardware consists of two IR emitting diodes (IREDs) located a specified distance apart on the board, with a detector midway between them.

Figure 2: A simple IR gesture sensing system using two IR diodes and a sensing device located between them. (Image source: Vishay Semiconductor)

The radiation pattern of each IR emitter is highly directional. As a hand passes over an emitter, the reflected radiation measured by the detecting sensor will exhibit a corresponding spike. As the operator moves the hand from left to right, the signal from the left (black) emitter will increase and decrease before the right (green) emitter, and vice-versa for a right-to-left motion.

The sensor sends the data to a microcontroller that contains software to analyze the variation in signal strength over time to determine if a swipe gesture was made, and if so, in which direction.

Gesture sensing design example

A typical gesture sensing application circuit adds a third IR emitter to the basic system, enabling it to detect both up/down and left/right gestures (Figure 3).

Figure 3: A gesture sensing design with three emitters can detect both left/right and up/down gestures. (Image source: Vishay Semiconductor)

The heart of the system is Vishay Semiconductor’s VCNL4035X01 IR and ambient light sensor, which operates on a single 2.5 to 3.6 volt supply. The application-specific integrated circuit (ASIC) includes drivers for three external IR emitters, an internal photodiode to receive the reflected outputs, and circuitry to process the photodiode signal. An industry standard I²C serial interface communicates with an external microcontroller that analyzes the data to decode the gesture.

Figure 4: The VCNL4035 can drive up to three external IR emitters and includes an internal ambient light sensor. (Image source: Vishay Semiconductor)

To save power and reduce processing overhead, the VCNL4035 has a programmable interrupt pin that can wake up the microcontroller when a gesture event or ambient light change occurs, eliminating the need for continuous polling.

The VCNL4035X01 features a miniature 4.0 x 2.36 millimeter leadless package (LLP) with a height of 0.75 millimeters, enabling it to fit into space constrained smartphones, digital cameras, tablet PCs, and similar applications.

Figure 5: Normalized spectral response of the sensors in the VCNL4035 IR photodiode (a) and ambient light sensor (b). The response of the human eye is shown for comparison. (Image source: Vishay Semiconductor)

To determine the direction of motion, the software must be able to differentiate between the IRED outputs to be able to compare them. In gesture sensing mode, the VCNL4035 drives the IREDs sequentially in quick succession, and then flags the microcontroller to read the internal registers containing the three 16-bit photodiode outputs. The current drive strength, the drive time, and the time between successive sets of measurements are all user selectable.

The ambient light sensor (ALS) receives visible light and converts it to a 16-bit ADC value. The VCNL4035’s ALS has peak sensitivity at 540 nm and a bandwidth of 430 nm to 610 nm, closely matching that of the human eye.

IRED requirements

The output of the IREDs should match the sensitivity response of the VCNL4035’s photodiode to ensure the best performance. With a peak wavelength (λ_P) of 940 nm, Vishay Semiconductor’s VSMY2940RG series of emitters are well suited for this purpose. They are based on gallium aluminum arsenide (GaAlAs) surface emitter chip technology. They provide a typical radiant intensity of 880 milliwatts per steradian (mW/sr) at 1 amp forward current in pulsed operation (t_p = 100 μS).

Additionally, the devices emit radiation in a very directional pattern, essentially within ±10° beam angle (Figure 6).

Figure 6: Vishay’s VSMY2940RG IR emitter has a very constrained relative radiant intensity vs. angular displacement curve, making it suitable for gesture sensing applications. (Image source: Vishay Semiconductor)

Gesture sensing design considerations

The designer of a gesture sensing system has a number of tradeoffs to consider before determining the optimum design. One is detection range versus current consumption. Another is IRED placement.

Increasing the distance at which a gesture can be detected implies raising the output power of the IREDs, which is a function of their forward current. As a result, power consumption will increase, which is undesirable in battery-powered devices. The higher power also increases the amount of heat that must be dissipated, potentially increasing the size of the design.

In the design discussed earlier, an IRED drive current of 200 mA gives a typical radiant intensity of 200 mW/sr, allowing for the detection of hand gestures up to 40 cm from the sensor board.

For IRED placement, the number of external IREDS and their positioning depend upon the needs of the specific application. The distance between the IREDs and the sensor may be just a few millimeters for a simple proximity application, or to detect a small gesture such as finger movement.

A larger separation between the sensor and the IREDs offers advantages in determining the direction of a hand swipe at a longer distance. Empirical testing can arrive at the optimum distance.

Gesture detection software

There are several methods to determine the intended gesture using the outputs from the VCNL4035. One simple method is to set a minimum threshold for detection of an object. A rising edge that exceeds the threshold indicates the arrival of a hand over an IRED, and a falling edge indicates that the hand has left that IRED’s detection area.

With two IREDs, the order of events indicates whether a left or right swipe has occurred. This method analyzes the result of each measurement cycle in isolation and requires few processor resources.

A more computationally intense approach analyzes data from multiple sets (frames) of measurements. The algorithm calculates two quantities for each frame: the standard deviation of each signal compared to its respective sample set, and the time delay between the left-right signals. By comparing the results to user-defined thresholds, the algorithm can tell what type of gesture has occurred.

The standard deviation s, is a measure of the spread of the data within the frame being analyzed. It is calculated using the formula:

Where x̄ is the mean of the current frame, and n is the number of samples being analyzed.

A high standard deviation implies a large signal change, suggesting the movement of the hand across or towards a sensor.

On the other hand, a low standard deviation implies there is little or no change in the signal where either there is no hand in the sensor’s detection area, or the hand is not making a movement. A sufficiently large time delay between the signals signifies a swipe gesture has been made. This calculation can also estimate the delay with a cross-correlation algorithm that calculates the overlap between the two signals.

Get started quickly with a sensor starter kit

Vishay’s VCNL4020 Sensor Starter Kit is an easy way to get started with an IR gesture sensing design. The kit includes a USB dongle, a plug-in VCNL4020 gesture demo board, and a mini-CD containing the USB driver and software.

The VCNL4020 itself is an integrated proximity and ambient light sensor with an internal IR emitter. On the demo board, the VCNL4020 also drives an external IRED to provide increased spatial separation if needed.

Figure 7: Vishay’s VCNL4020 Sensor Starter Kit provides the hardware and software needed to begin gesture-sensing development. It is Vishay’s default starting point and connects to a variety of expansion boards for different Vishay sensors. (Image source: Vishay Semiconductor)

The USB dongle enables I²C-to-USB communication between the demo board and a host PC. It contains Cypress Semiconductor’s CY768013A, an ASIC that combines an enhanced 8051 microcontroller with an integrated single-chip USB 2.0 transceiver. It also includes a 3.3 volt regulator, plus Microchip Technology’s MCP3421, an 18-bit, single-input sigma-delta analog-to-digital converter (ADC) to provide an additional analog input.

The USB dongle serves as the base for other VCNL sensor demo boards, all of which are free from Vishay, including the VCNL4035 demo board. This board is especially intuitive in that it includes an array of LEDs that indicates the direction of a gesture.

The development software on the CD included with the VCNL4035 board implements the standard deviation and cross-correlation detection algorithms discussed earlier.

The software allows the user to change key parameters of both the data acquisition and the detection algorithm. Numerous parameters can be adjusted to fine-tune the system performance, including:

• Data measurement rate
• Measurement sample time
• Up/down detection threshold
• Left/right detection threshold
• Threshold slope
• Standard deviation threshold

Figure 8: Gesture sensing screen using a LabVIEW GUI for the VCNL4035 showing the results of a left-to-right motion. (Image Source: Vishay Semiconductor)

The starter kit includes a graphical user interface (GUI) that runs on National Instruments’ popular LabVIEW programming platform. The gesture-sensing screen displaying the results of a left-to-right movement is shown (Figure 8, again). The left (red) IRED indicates first, followed by the middle (green), then the right (blue).

For developers new to the LabVIEW environment, Digilent’s LabVIEW Home Bundle offers developers the chance to get started at minimal cost.

Conclusion

An infrared gesture sensing system can meet many HMI challenges. Its combination of low-cost hardware and sophisticated software can detect many hand motions common to industrial, automotive, and consumer applications.

Vishay offers a variety of optoelectronic components and starter kits that help developers get started quickly on an HMI gesture sensing design. The inclusion of a LabVIEW GUI also simplifies development and helps to better manage the design trade-offs involved.