Fundamentals of Distance Measurement and Gesture Recognition Using ToF Sensors

Many applications require sensing the presence or distance to physical objects without touching them. This need for proximity sensing has led to many competing solutions, including optical time-of-flight (ToF) sensors. While these are accurate, they have been costly and complex to implement; but recent solutions have made the technology a lot simpler to use.

The list of products that use proximity sensing includes camera autofocus systems, robots and drones, a variety of bathroom fixtures, and portal access sensors for automatic doors. These are but a few examples, and the list is growing. The competing proximity sensing technologies start with simple infrared and ultrasonic sensors, gradually working up to more complex sensor systems such as stereoscopic video cameras with neural networks.

All of these technologies have limitations, and while artificial intelligence may be needed to implement complex tasks such as object recognition and tracking for autonomous vehicles, the technology is overkill for dispensing paper towels and soap. Designers, facing ever decreasing budget and design windows, need to minimize cost, space and design time.

One of the alternatives to consider for proximity sensing are ToF sensors. These measure distance to a target object by timing the round trip of photons as they travel from the sensor to the object and back. To date, it’s been difficult to implement ToF designs quickly and at low cost, but new generations of highly integrated and inexpensive ToF sensors now make it possible to bring highly accurate touchless sensing to lower cost designs.

This article will discuss the evolution and use of distance measuring technology, including ToF sensors, in a variety of applications for both distance sensing and gesture recognition. It will then describe how ToF sensor technology operates before introducing some of the latest solutions and how to get started with them.

Early proximity sensors

The Polaroid SX-70 instant film camera, introduced in 1972, incorporated many innovative technologies including a three mirror optical design that folded flat, Fresnel lenses, a flat 6 volt battery built into the instant film package, and a ten shot flash bar. However, one of the most influential technologies that Polaroid introduced in the SX-70 camera was the Sonar Autofocus System, which first appeared in the Polaroid SONAR OneStep camera announced in 1978 (Figure 1). The SONAR Autofocus System employed an innovative ultrasonic transducer to both transmit the ultrasonic ranging pulse and to receive the reflected ultrasound energy.

Figure 1: The Polaroid SONAR OneStep SX-70 camera incorporated an ultrasonic transducer (the large gold circle on the top of the camera) for autofocus ranging. (Image source: Wikipedia)

The SONAR’s ultrasonic sensor was so successful that Polaroid build a business around the sensor alone, and echoes of the Polaroid ultrasound sensor autofocus sensor can be found today. For example, the inexpensive SparkFun SEN-13959 HRC-SR04 ultrasonic ranging module is a distance sensor with separate transmit and receive transducers (Figure 2). The sensor is designed to be directly driven by an Arduino development board. The sensor’s range extends from 2 to 400 centimeters (cm), and it’s specified to be able to make non-contact proximity measurements with a minimum resolution of 3 millimeters (mm).

Figure 2: SparkFun’s SEN-13959 ultrasonic ranging module uses reflected 40 kHz pulses to sense distance. (Image source: SparkFun)

To measure distance with this module, the Arduino board (or another controller) sends a 10 microsecond (µs) pulse to the board’s Trig pin, which triggers a series of eight short ultrasound pulses that are emitted by the ultrasonic transmitter. The sound pulses hit the target and reflect back, traveling at 343 meters (m) per second (in a typical environment at 20˚C). The distance to the target is the time duration between the transmission and receipt of the ultrasonic pulses, multiplied by 343 meters per second and divided by two (to account for the round trip).

Ultrasonic pulses reflect well off of hard surfaces and less well when bouncing off of soft surfaces such as drapes, carpets, clothing, and pets. Measurement accuracy will depend on the method used for timing the pulses. There’s no such timing control capability in the SparkFun SEN-13959 module. It depends on the host CPU for precise timing. In addition, the accuracy and stability of the distance measurements from an ultrasonic sensor vary with air temperature (which alters the speed of sound in air) and air movement (which carries away much of the reflected ultrasound energy and attenuates the return signal).

Infrared (IR) LEDs have also been used for proximity detection and distance measurements. For example, the Sharp Microelectronics GP2Y0A41SK0F distance measuring sensor unit can sense objects at distances from 4 cm to 30 cm based on the intensity of the reflected infrared light sent from an IR LED (Figure 3). The sensor outputs an analog voltage ranging from just over 3 volts (corresponding to a proximity of 3 cm) to roughly 0.3 volts (corresponding to a proximity of 40 cm) to indicate object distance. The host controller is responsible for converting this analog voltage to a digital representation.

Figure 3: The Sharp GP2Y0A41SK0F infrared distance measuring sensor unit can detect objects over a range of 3 cm to 40 cm. (Image source: Sharp Microelectronics)

However, because object distance is based on the amount of reflected IR energy, the accuracy of this type of IR sensor is subject to error due to variables such as object reflectivity and the intensity of the ambient light.

Another way to measure an object’s distance with infrared light is to time the flight of photons as they travel from the sensor’s infrared emitter to the object, reflect, and return to the sensor. This sort of proximity measurement sensor combines the ToF characteristics of the ultrasound sensor with the relative imperturbability of photon speed, which is not affected by moving air, ambient light, or reflectivity.

Until recently, it’s been difficult to time the flight of photons traveling over small distances because light travels at 299,792,458 meters per second, or as a rule of thumb, about one foot per nanosecond. As a result, ToF sensors need very accurate, sub-nanosecond timing in order to detect distances of a few millimeters, a few centimeters, or even inches.

However, thanks to the video game industry, ToF sensor technology has gotten a lot less expensive. Perhaps the best known use of ToF sensors to date is Microsoft’s® Kinect game controller (Figure 4). The first generation of the Microsoft Kinect was introduced as a peripheral for the company’s Xbox 360 in late 2010. This controller became very popular with robot builders because it was able to generate three-dimensional maps of the robot’s immediate environment using ToF distance sensing.

Figure 4: The Microsoft Kinect controller for the company’s Xbox 360 video game console employs ToF sensing to create a three-dimensional map of its surroundings. (Image source: Wikipedia)

The Kinect controller’s sensing technology has been miniaturized and simplified to create practical distance measurement sensors that are appropriate for many embedded applications.

VCSELs and SPADs

For example, STMicroelectronics now has a multigenerational line of miniature ToF sensors for measuring proximity. These sensors are based on some extremely modern foundational technologies including infrared vertical cavity, surface emitting lasers (VCSELs), and arrays of single photon avalanche photodiodes (SPADs).

Three of the sensors in this ToF sensor line are the VL53L0CX, the VL53L1CX, and the VL6180X. Although they all measure distance, each of these three sensors has different capabilities.

The first generation VL6180X ToF sensor has one range mode and measures proximity from a few millimeters to 100 millimeters (Figure 5). This sensor measures 4.8 x 2.8 x 1.0 mm and has a 42-degree field of view. It also has a built-in ambient light sensor that allows it to compensate for changes to the ambient lighting.

Figure 5: STMicroelectronics’ first generation VL6180X sensor has a maximum range of 100 millimeters. (Image source: STMicroelectronics)

The second generation VL53L0CX ToF sensor has an indoor range of 50 to 1200 mm working against a white target (Figure 6). It measures 4.4 x 2.4 x 1.0 mm and has a 25-degree field of view. Due to ambient light, the maximum range shrinks to 600 to 800 mm when the sensor is used outdoors.

Figure 6: STMicroelectronics’ second generation VL53L0CX sensor has a maximum range of 1200 mm. (Image source: STMicroelectronics)

The third generation VL53L1CX ToF sensor has three distance modes (Figure 7). Maximum distance in the short, medium, and long range modes is 1360, 2900, and 3600 mm for a white target with no ambient light. With strong ambient light, maximum distance in the short, medium, and long range modes is 1350, 760, and 730 mm. Counterintuitively, the short distance mode has the longest measurement reach if the ambient light is strong.

The VL53L1CX measures 4.9 x 2.5 x 1.56 mm and has a 27-degree maximum field of view. (This sensor’s field of view is programmable and can be made narrower, as explained below.)

Figure 7: STMicroelectronics’ third generation VL53L1CX sensor has a maximum range of nearly 4 m. (Image source: STMicroelectronics)

All three of these ToF sensors report proximity measurements with 1 mm resolution to a host processor over a digital I²C interface that also serves as the sensors’ control port. Because these sensors all employ an I²C interface, they’re extremely simple to connect to a host processor (Figure 8).

Figure 8: Like earlier sensors in the family, the STMicroelectronics’ third generation VL53L1CX sensor connects to the host processor using a simple I²C connection. (Image source: STMicroelectronics)

Note the very specific bypassing requirement for the AVDDVCSEL and AVDD power supply lines. The 100 nanofarad and 4.7 microfarad bypass capacitors need to be located as close to the sensor as possible to prevent power supply noise from entering the sensor and degrading accuracy.

These ToF sensors are all essentially one dimensional. They report proximity for objects within their fields of view. If multiple objects are in the field of view, these sensors report the distance to the nearest object. A single sensor cannot detect the direction of a one-handed gesture, but it can be used to detect four simple gestures including:

1. Single tap (hand moving down to “tap” the sensor)
2. Double tap
3. Single swipe (hand moving across and through the sensor’s field of view)
4. Double swipe

It is possible to get gesture and motion information from one of these ToF sensors by using one, two, or more sensors to detect gestures and motion in multiple dimensions. Paired ToF sensors can be used to discern left to right and right to left hand movement as well.

In addition, it’s possible to get more information from the third generation VL53L1CX proximity sensor by selectively narrowing its field of view. This is done by switching off individual SPADs in the sensor’s array using commands sent to the sensor over the I²C interface. The VL53L1CX proximity sensor’s SPAD array consists of 256 photodiodes in a 16 x 16 array. Any square or rectangular portion of the array can be made active through software commands that specify the two corners of a box surrounding the SPADs in the array that should be made active. Reducing the number of active elements reduces the sensor’s field of view and narrows the sensor’s region of interest. The only requirement is that at least 16 SPADs must be active, as a 4 x 4 photodiode array, but larger arrays are permitted as well.

Designing with ToF sensors

To kickstart a design, the VL53L1CX proximity sensor has an accompanying evaluation kit, the P-NUCLEO-53L1A1. This includes the STM32F401RE Nucleo evaluation board, which is based on an STMicroelectronics STM32 microcontroller, and the X-NUCLEO-53L1A1 expansion board that mounts on the microcontroller board and accepts two VL53L1X breakout boards (which are also included in the kit) (Figure 9).

Figure 9: The sensor breakout board included in the STMicroelectronics P-NUCLEO-53L1A1 evaluation kit includes one V53L1X ToF proximity sensor mounted directly on the board. The board accepts two more V53L1X sensors on pluggable breakout boards. (Image source: STMicroelectronics)

The P-NUCLEO-53L1A1 evaluation kit also includes system software and source code examples for a fast development start. STMicroelectronics also offers TOF ranging and gesture detection expansion modules for its STM32Cube software development package. These expansion modules are specific to the individual sensors and can be downloaded for free directly from STMicroelectronics.

Because they’re fairly small, these STMicroelectronics ToF sensors can fit almost anywhere a designer might envision. Here are a few application examples to juice the imagination:

• General proximity sensors for robots
• Touchless paper towel and soap dispensers
• Touchless toilet and urinal flushers
• Touchless sink faucets
• Wall following and object avoidance sensors for robotic vacuum cleaners
• Low cost operator presence detectors for laptop computers and monitors
• Simple presence and gesture detection for retail kiosks
• Physical inventory management for vending machines
• Coin counting for vending machines
• Smart shelves that automatically manage inventory for cashier-less stores
• Ground proximity detection for drones
• Ceiling proximity detection for indoor drones

Unlike proximity detectors based on two-dimensional ToF sensors or stereo cameras and neural networks, the cost for these integrated STMicroelectronics ToF proximity sensors is relatively low, making it possible to incorporate them into a variety of end products designed to sell over a wide price range.

Conclusion

There are many technologies available for proximity sensing including optics and ultrasonics, and there are many good solutions based on these technologies. However, one of the newest of these proximity sensing technologies is ToF, or Time-of-Flight. These measure distance to a target using the round-trip time required for a photon to leave the sensor, travel to the target, and then reflect back to the sensor.

The emergence of integrated sensors that incorporate infrared emitters and receivers and the required circuitry for the subnanosecond timing of the photonic travel times, make the use of this technology cost effective. Associated development kits also allow for experimentation and faster prototyping.