Fall Detection Using mmWave

Table of Contents

Introduction

TI mmWave Sensors can be used to detect rapid height changes in tracked people in real environments and raise an alert upon sensing a fall event.

In many circumstances, it can be valuable to detect the stance of a person with mmWave sensors. For example, in an elderly care scenario, the mmWave sensor can detect the stance, as well as detect dangerous events like falls, without compromising the privacy of the individual. While the radar sensor provides 3D position, velocity, and acceleration of tracked objects/people, it is not precise enough to be used to determine the identity or take an image of the person. The goal of this experiment is to demonstrate how this information can be used to detect falls, while distinguishing them from other common events, such as sitting down, or lying down. This experiment shows how a simple algorithm for height estimation can detect falls, how the algorithm can be tuned, and potential future work to enhance the algorithm’s performance.

Setup

The following software and hardware setup was used to complete the experiment:

Hardware Setup

Software Setup

🛑 Software Versions
This experiment’s software compatibility has been affected by updates made to the mmWave Radar Toolbox. In order to use this experiment as described, please use the binary and visualizer from the Radar Toolbox Version 1.2 or later

Fall Detection Algorithm Description

The fall detection algorithm can be split into two parts, height estimation and fall detection. The height estimation algorithm uses the tallest points in the point cloud to estimate target height. It then implements an alpha filter over time to smooth the estimate. For a more detailed explanation, refer to the 3D People Tracking Demo Implementation Guide.

The fall detection algorithm monitors the estimated height of each target for sudden changes. If the estimated height changes by more than the pre-set percentage in a limited amount of time, then a fall is detected. The pre-set percentage is dictated by the slider in the Industrial Visualizer GUI. The amount of time is set in the visualizer code to 2.5 seconds.

When the sensitivity is set lower, the sensor can even be used to distinguish between a controlled descent to the ground and an uncontrolled fall event. The default value used by the Industrial Visualizer software defines a fall as a reduction in height by 60% over the 2.5 second period. This value can be set as high as 80% (more sensitive) or as low as 40% (less sensitive). For example, if the sensitivity is set to 80%, then a fall event for target of 2 meters would require the height of the target to decrease to 1.6 m.

📝 Expected Performance
Since the fall detection algorithm compares the height over a period of 2.5 seconds, there may appear to be a delay between the falling and the detection. This is to be expected if the user falls suddenly.

Areas of Interest for Development

The software and setup provided by TI are meant to function as a starting point for development. There are a number of ways developers can distinguish their own algorithms and tuning from the generic offering. Some areas of interest include adding more indicators for a fall action, such as a change in the track velocity, or even referring to the range and elevation data in the radar cube as another measurement of height information, rather than relying on the point cloud. The detection mechanism could also be made more robust through the use of machine learning to distinguish between falling, sitting and lying down.

Results in a Variety of Environments

A depricated version of the fall detection software was tested in a variety of environments to illustrate the robustness of the fall detection demo in different settings. Both the algorithm and visualizer have been upgraded since these experiments, so results from Radar Toolbox 1.2 will not look the same as below. However, the robustness illustrated still applies to the algorithm used by Radar Toolbox 1.2, so the results are shown below.

Procedure

For each mounting height, the following steps were followed:

  1. Fall detection demo is configured and begins running
  2. One person enters the scene and moves in front of the sensor
  3. The person walks around the scene to verify accurate tracking.
  4. The person simulates fall events in various methods.
  5. The fall events were repeated in various rooms and with typical objects like shower curtains obstructing camera view.

During the course of the experiment in each scene, the fall detection software was left running.

Scene 1: Living Room

A typical living room environment was tested with sitting and falling around a couch. The room is about 5m long, with the boundary box being set at the perimeter of the room.

When transitioning from standing to sitting, a change in height is recognized, but it is not fast enough to trigger a fall event.

When a person falls from standing, the change in height is rapid enough to be seen as different than sitting and triggers a fall event.

Multiple height adjustments can still be detected and trigger fall events. Transitioning from sitting to standing shows an increase in height, but a following fall event is still detected.

Scene 2: Bathroom

The mmWave radar is an ideal technology for fall detection in a bathroom because it maintains privacy, unlike a camera system. Fall events are also quite likely to occur in bathrooms due to slick floors or wet surfaces. Because of the relatively small size of many bathrooms, and reflective tile or vinyl flooring, mmWave sensors can be prone to picking up reflected signals and ghost images, which could lead to false tracking and fall events. But by matching the boundary box settings to the size of the bathroom, the tracking algorithm operating on the mmWave sensor can intelligently ignore these false signals and accurately detect falls in the small room.

In the video clip above, the tracking works very well in the small bathroom with tile floors. The tracker can distinguish between a sitting motion and a fall event on the floor.

As long as the person remains inside the boundary box, the tracker maintains location and height estimation even when the person is behind a thin barrier, such as a shower curtain. The above clip shows the tracker can still detect a fall event even when a traditional camera would not be able to see through the opaque barrier.

Scene 3: Outdoors

The outdoor environment was only used to demonstrate the range capabilities of the fall detection. The IWR6843ISK was used for this test, as it has better range capabilities. But the limited vertical resolution of the antenna pattern begins to become problematic at longer ranges, as the minimum change in estimated height comes close to the total height of the person.

Conclusion

The IWR6843 3D tracking can be used to reliably track changes in absolute and relative height of a human. These height changes can be used to detect falls while filtering out sitting or lying down. More advanced algorithms could be used to extend this work using more input data, or even machine learning. The fall detection software is also robust enough to be used in a number of household, medical and commercial spaces where fall events need to be monitored.