Vital Signs User Guide

Table of Contents

Overview

This lab demonstrates the use of TI mmWave sensors to monitor breathing rate and heart rate using the IWRL6432 sensor module.

Detection and tracking algorithms run onboard the IWRL6432 mmWave sensor and are used to localize people and track their movement. Vital sign measurement algorithms use the position data of the tracking to measure the vital sign info of the tracked person.

This demo uses algorithms from the motion and presence detection demo to perform detection and tracking. In this demonstration, localization and tracking is performed upon any moving object in the scene and the people or objects will continue being tracked until they leave the scene. This will continue tracking people as they sit and remain stationary.

The vital sign measurements will only be accurate when the tracked person stops moving for at least 30 seconds. The person should be seated with the sensor pointed toward their chest, up to a 3m distance.

With the motion and presence detection and vital sign software running on the IWRL6432 chip, the mmWave sensor module outputs a data stream consisting of 3 dimensional point cloud information and a list of tracked objects which can be visualized using the PC based visualizer included in the toolbox.

Quickstart

Prerequisites

⚠️ WARNING
Run Out of Box Demo
Before continuing with this lab, users should first run the out of box demo for the EVM. This will enable users to gain familiarity with the sensor’s capabilities as well as the various tools used across all labs in the Radar Toolbox.

⚠️ WARNING
Sufficient PC CPU and GPU resources required
The visualizer requires sufficient GPU and CPU resources to process the incoming UART data every frame and run the demo smoothly. Users with insufficient PC resources may notice occasional visualizer lag or missed frames in the log file. The demo should still run on most PCs regardless of hardware resources, but performance might be slightly different.

1. Hardware and Software Requirements

Hardware

Software

2. Flash the EVM

1. Set the device in flashing mode
2. Flash the prebuilt binary which corresponded to your evm from the prebuilt_binaries folder of this demo

3. Physical Setup

1. Setup the device for functional mode

Setup Requirements:

• Elevate EVM: 1 - 1.5m high
• Down tilt: ~0-15 degree

4. Run the Lab

To run the lab, launch and configure the visualizer which displays the detection and tracked object data received via UART. The visualizer that should be used for this lab can be found at ‘\tools\visualizers\Applications_Visualizer\Industrial_Visualizer' of the toolbox. Choose the ’IWRL6432 Vital Signs’ option for config type when connecting to the COM ports.

Please ensure you use the default chirp for your device:

• Vital_Signs_With_Tracking_BOOST.cfg
• Vital_Signs_With_Tracking_AOP.cfg

⚠️ WARNING
Device must be restarted before sending a new configuration.

Modifying Configuration File

The configuration files included a set of commands which are used to specify the scene boundaries (i.e. area of interest) in relation to the sensor position and may need to be modified accordingly. These commands and their respective parameters are listed below. All values are in world coordinates, so they are all relative to the position of the sensor listed in sensorPosition.

• staticBoundaryBox [-X] [X] [NearY] [FarY] [-Z] [Z]
  • This sets boundaries where static points can be used by the tracker and tracks are allowed to become static. Each value denotes an edge of the 3D cube. Currently, it is recommend to keep NearY greater than or equal to 2.
• boundaryBox [-X] [X] [NearY] [FarY] [-Z] [Z]
  • This sets boundaries where tracks can exists. Only points inside the box will be used by the tracker. Each value denotes an edge of the 3D cube.
• presenceBoundaryBox [-X] [X] [NearY] [FarY] [-Z] [Z]
  • The demo supports early presence detection if a pre-configured number of points are detected in the boundary box defined by presenceBoundaryBox. The points also have to meet a pre-defined velocity threshold which is modeled after typical human walking motion. Early presence detection could be useful in applications such as smart lighting, where it may be desirable to turn on the lights as soon as human presence is detected in the area, even before tracks could be allocated.
• sensorPosition [Z] [AzimuthTilt] [ElevationTilt]
  • Z - height of sensor.
  • AzimuthTilt - horizontal rotation of sensor with respect to the back wall, as represented by FarY.
  • ElevationTilt - vertical rotation of the sensor with respect the the ground.

UART Output Data Format

The demo outputs the point cloud and tracking information using a TLV(type-length-value) encoding scheme with little endian byte order. For every frame, a packet is sent consisting of a fixed sized Frame Header and then a variable number of TLVs depending on what was detected in that scene. The TLVs can be of types representing the 3D point cloud, target list object, and associated points. Please see Understanding UART Data Output for more information.

Frame Header

View the frame header structure here: Understanding UART Data Output: Frame Header.

TLVs

The TLVs can be of type POINT CLOUD, TARGET LIST, TARGET INDEX or PRESENCE INDICATION.

TLV Header

View the TLV header structure here: Understanding UART Data Output: TLV Header.

Point Cloud TLV

Size: sizeof (tlvHeaderStruct) + sizeof(pointUnit) + sizeof (pointStruct) x numberOfPoints

Each Point Cloud TLV consists of an array of points. Each point is defined in 8 bytes. The pointUnit struct is used to decompress each point to five floats (20 bytes).

Point Unit Structure

pointUnit = struct(...
    'elevationUnit',        {'float', 4}, ... % Multiply each point by this value - used for compression
    'azimuthUnit',          {'float', 4}, ... % Multiply each point by this value - used for compression
    'dopplerUnit',          {'float', 4}, ... % Multiply each point by this value - used for compression
    'rangeUnit',            {'float', 4}, ... % Multiply each point by this value - used for compression
    'snrUnit',              {'float', 4});    % Multiply each point by this value - used for compression

Point Cloud Structure

pointStruct = struct(...
    'elevation',        {'int8_t', 1}, ... % Elevation in radians
    'azimuth',          {'int8_t', 1}, ... % Azimuth, in radians
    'doppler',          {'int16_t', 2}, ... % Doppler, in m/s
    'range',            {'uint16_t', 2}, ... % Range, in meters
    'snr',              {'uint16_t', 2});    % SNR, ratio

Target List TLV

Size: sizeof (tlvHeaderStruct) + sizeof (trackerProc_Target) x numberOfTargets

The Target List TLV consists of an array of targets. Each target object is defined as given below.

targetStruct3D = struct(...
    'tid',             {'uint32', 4}, ... % Track ID
    'posX',            {'float', 4}, ... % Target position in X dimension, m
    'posY',            {'float', 4}, ... % Target position in Y dimension, m
    'posZ',            {'float', 4}, ... % Target position in Z dimension, m
    'velX',            {'float', 4}, ... % Target velocity in X dimension, m/s
    'velY',            {'float', 4}, ... % Target velocity in Y dimension, m/s
    'velZ',            {'float', 4}, ... % Target velocity in Z dimension, m/s
    'accX',            {'float', 4}, ... % Target acceleration in X dimension, m/s2
    'accY',            {'float', 4}, ... % Target acceleration in Y dimension, m/s
    'accZ',            {'float', 4}, ... % Target acceleration in Z dimension, m/s
    'ec[16]',          {'float', 16x4}, ... % Tracking error covariance matrix, [4x4] in range/azimuth/elevation/doppler coordinates
    'g',               {'float', 4}, ... % Gating function gain
    'confidenceLevel'  {'float', 4}, ... % Confidence Level

Target Index TLV

Size: sizeof (tlvHeaderStruct) + sizeof(uint8) x numberOfPoints (NOTE: here the number of points are for frame n-1)

The Target Index TLV consists of an array of target IDs. A targetID at index i is the target to which point i of the previous frame’s point cloud was associated. Valid IDs range from 0-249.

targetIndex = struct(...
'targetID',         {'uint8', 1});    % Track ID

Other Target ID values:

Presence Indication TLV

Size: sizeof (tlvHeaderStruct) + sizeof(uint32)

The Presence Indication TLV consists of a single uint32 value to provide a binary indication of presence in the presence boundary box. A value of 1 represents presence detected and 0 represents no presence detected.

Vital Sign Info TLV

Size: 136 Bytes

VS_Feature = struct(...
    'id',                                       {'uint16', 2},...     % Target ID used for XYZ location
    'rangebin',                                 {'uint16', 2},...     % range bin for XYZ location
    'breathingDeviation',                       {'float', 4},...      % deviation of breathing measurement over time
    'heart_rate',                               {'float', 4},...      % Heart Rate Measurement
    'breathing_rate',                           {'float', 4},...      % Breath Rate Measurement
    'VitalSigns_Heart_CircularBuffer[15]',      {'float', 4x15}),...  %Buffer of heartrate waveform
    'VitalSigns_Breath_CircularBuffer[15]',     {'float', 4x15}),...  %Buffer of breathrate waveform

📝 NOTE - Presence Detection and Smoothing

Visualizer

Patient status is determined in the python source code for the visualizer based on the breathing deviation value passed through the Vital Signs Info TLV.

The breathing deviation is calculated by looking at the variation of the last 40 values in the breathing waveform. It is recalculated every 15 frames when the vital signs processing is performed.

First the visualizer looks to see if a track is present. If no track is present, no data is displayed. If a track is present and the breathing deviation is zero, no patient data is displayed. However, the range bin where the person’s track was last detected will still be displayed for debugging purposes.

If the breathing deviation value falls below 0.02 “holding breath” is detected and the breath rate value is no longer displayed.

Additionally, the visualizer smooths the heart rate results by calculating the median of the latest 10 results.

Need More Help?

• Search for your issue or post a new question on the mmWave E2E forum