Vital Signs With People Tracking User Guide

Table of Contents

Overview

This lab demonstrates the use of TI mmWave sensors to track and count moving and stationary people while monitoring breathing rate and heart rate using the IWR6843AOPEVM or IWR6843ISK sensor module.

Detection and tracking algorithms run onboard the IWR6843 mmWave sensor and are used to localize people and track their movement. Vital sign measurement algorithms use the position data of the tracking to measure the vital sign info of the tracked person.

🛑 Before Continuing!
No Source Code Provided
This lab is provided as a binary file without source code. This can be used to test the application in various different use cases. For more information on the source code and implementation, please reach out to your local TI sales representative.

This demo uses algorithms from the 3D People Tracking demo to perform detection and tracking. In this demonstration, localization and tracking is performed upon any moving object in the scene and the people or objects will continue being tracked until they leave the scene. This will continue tracking people as they sit or lie down and remain stationary.

The vital sign measurements will only be accurate when the tracked person stops moving for at least 20 seconds. The person should be seated or lying down with the sensor pointed toward their chest, up to a 5m distance.

With the 3D people tracking and vital sign software running on the IWR6843 chip, the mmWave sensor module outputs a data stream consisting of 3 dimensional point cloud information and a list of tracked objects which can be visualized using the PC based visualizer included in the toolbox.

This user guide covers the procedures to Flash, Run, and Compile the 3D people tracking with vital signs demo. This demo uses algorithms from the 3D People Tracking demo. For details regarding the demo software algorithms and implementation of the detection and tracking layers from the 3D people tracking demo, please refer to the following documents available in the People Tracking Lab docs directory. Details specific to the vital sign measurement will be discussed later in this document.

• 3D People Tracking Demo Software Implementation Guide
• Group Tracker Algorithm Implementation Guide

🛑 Before Continuing!
AoP ES2.0 EVM, Rev F or later only
The IWR6843 AoP version of this lab is only compatible with ES2.0 silicon and the corresponding EVM. Please ensure your EVM is the same as in the below image.

Quickstart

Prerequisites

⚠️ WARNING
Run Out of Box Demo
Before continuing with this lab, users should first run the out of box demo for the EVM. This will enable users to gain familiarity with the sensor’s capabilities as well as the various tools used across all labs in the Radar Toolbox.

🛑 Before Continuing!
If using MMWAVEICBOOST latest XDS110 firmware required
The XDS110 Firmware runs on the microcontroller onboard the MMWAVEICBOOST Carrier Card which provides the JTAG Emulation and serial port communication over the XDS110 USB port. We have observed packet loss on the serial port (COM ports) with some versions of the XDS110 firmware which can cause the demo to fail. The latest XDS110 firmware is installed with the latest version of Code Composer Studio. CCS version 10.1 or later required.

⚠️ WARNING
Sufficient PC CPU and GPU resources required
The visualizer requires sufficient GPU and CPU resources to process the incoming UART data every frame and run the demo smoothly. Users with insufficient PC resources may notice occasional visualizer lag or missed frames in the log file. The demo should still run on most PCs regardless of hardware resources, but performance might be slightly different.

1. Hardware and Software Requirements

Hardware

Software

2. Flash the EVM

1. Set the device in flashing mode:

   a. If using ISK standalone module, follow the instructions for setting Modular EVM to flashing mode

   b. If using ISK with ICBOOST, follow the instructions for Hardware Setup of ICB for Flashing Mode

2. Follow the instruction to Flash the mmWave Device Following the instructions in the guide above, flash the prebuilt binary which corresponded to your evm from the prebuilt_binaries folder of this demo

3. Physical Setup

1. Setup the device for functional mode

   a. If using EVM standalone, follow the instructions for setting Modular EVM to functional mode

   b. If using the ICBOOST, follow the instructions for Hardware Setup of ICB for Functional Mode

Setup Requirements:

• Elevate EVM: 1 - 1.5m high
• Down tilt: ~0-15 degree

4. Run the Lab

To run the lab, launch and configure the visualizer which displays the detection and tracked object data received via UART. See the instructions in the visualizer folder. The visualizer that should be used for this lab can be found at ‘\tools\Visualizers\Industrial_Visualizer' of the toolbox. Choose the ’Vital Signs with People Tracking’ option for config type when connecting to the COM ports.

Please ensure you use the default chirp for your device:

• AOP: vital_signs_AOP_xm.cfg
• ISK: vital_signs_ISK_xm.cfg

⚠️ WARNING
Device must be restarted before sending a new configuration.

Developer’s Guide

This lab is provided as a binary file without source code. This can be used to test the application in various different use cases. For more information on the source code and implementation, please reach out to your local TI sales representative.

Modifying Configuration File

The configuration files included a set of commands which are used to specify the scene boundaries (i.e. area of interest) in relation to the sensor position and may need to be modified accordingly. These commands and their respective parameters are listed below. All values are in world coordinates, so they are all relative to the position of the sensor listed in sensorPosition.

• staticBoundaryBox [-X] [X] [NearY] [FarY] [-Z] [Z]
  • This sets boundaries where static points can be used by the tracker and tracks are allowed to become static. Each value denotes an edge of the 3D cube. Currently, it is recommend to keep NearY greater than or equal to 2.
• boundaryBox [-X] [X] [NearY] [FarY] [-Z] [Z]
  • This sets boundaries where tracks can exists. Only points inside the box will be used by the tracker. Each value denotes an edge of the 3D cube.
• presenceBoundaryBox [-X] [X] [NearY] [FarY] [-Z] [Z]
  • The demo supports early presence detection if a pre-configured number of points are detected in the boundary box defined by presenceBoundaryBox. The points also have to meet a pre-defined velocity threshold which is modeled after typical human walking motion. Early presence detection could be useful in applications such as smart lighting, where it may be desirable to turn on the lights as soon as human presence is detected in the area, even before tracks could be allocated.
• sensorPosition [Z] [AzimuthTilt] [ElevationTilt]
  • Z - height of sensor.
  • AzimuthTilt - horizontal rotation of sensor with respect to the back wall, as represented by FarY.
  • ElevationTilt - vertical rotation of the sensor with respect the the ground.

Customization

For detailed description of the configuration parameters and recommendations related to tuning the parameters, please refer to the following documents available in the docs folder of the People Tracking lab directory

• 3D People Tracking Demo Detection Layer Tuning Guide
• 3D People Tracking Demo Tracker Layer Tuning Guide

The vital sign measurement accepts two additional commands beyond what is included in the above documents. These configuration inputs allow the algorithm to select a fixed or variable measurement range, and set the update rate for vital measurements. The below commands should be included in the configuration file.

vitalsign 15 300
VSRangeIdxCfg 0 21

UART Output Data Format

The demo outputs the point cloud and tracking information using a TLV(type-length-value) encoding scheme with little endian byte order. For every frame, a packet is sent consisting of a fixed sized Frame Header and then a variable number of TLVs depending on what was detected in that scene. The TLVs can be of types representing the 3D point cloud, target list object, and associated points. Please see Understanding UART Data Output for more information.

Frame Header

View the frame header structure here: Understanding UART Data Output: Frame Header.

TLVs

The TLVs can be of type POINT CLOUD, TARGET LIST, TARGET INDEX or PRESENCE INDICATION.

TLV Header

View the TLV header structure here: Understanding UART Data Output: TLV Header.

Point Cloud TLV

Size: sizeof (tlvHeaderStruct) + sizeof(pointUnit) + sizeof (pointStruct) x numberOfPoints

Each Point Cloud TLV consists of an array of points. Each point is defined in 8 bytes. The pointUnit struct is used to decompress each point to five floats (20 bytes).

Point Unit Structure

pointUnit = struct(...
    'elevationUnit',        {'float', 4}, ... % Multiply each point by this value - used for compression
    'azimuthUnit',          {'float', 4}, ... % Multiply each point by this value - used for compression
    'dopplerUnit',          {'float', 4}, ... % Multiply each point by this value - used for compression
    'rangeUnit',            {'float', 4}, ... % Multiply each point by this value - used for compression
    'snrUnit',              {'float', 4});    % Multiply each point by this value - used for compression

Point Cloud Structure

pointStruct = struct(...
    'elevation',        {'int8_t', 1}, ... % Elevation in radians
    'azimuth',          {'int8_t', 1}, ... % Azimuth, in radians
    'doppler',          {'int16_t', 2}, ... % Doppler, in m/s
    'range',            {'uint16_t', 2}, ... % Range, in meters
    'snr',              {'uint16_t', 2});    % SNR, ratio

Target List TLV

Size: sizeof (tlvHeaderStruct) + sizeof (trackerProc_Target) x numberOfTargets

The Target List TLV consists of an array of targets. Each target object is defined as given below.

targetStruct3D = struct(...
    'tid',             {'uint32', 4}, ... % Track ID
    'posX',            {'float', 4}, ... % Target position in X dimension, m
    'posY',            {'float', 4}, ... % Target position in Y dimension, m
    'posZ',            {'float', 4}, ... % Target position in Z dimension, m
    'velX',            {'float', 4}, ... % Target velocity in X dimension, m/s
    'velY',            {'float', 4}, ... % Target velocity in Y dimension, m/s
    'velZ',            {'float', 4}, ... % Target velocity in Z dimension, m/s
    'accX',            {'float', 4}, ... % Target acceleration in X dimension, m/s2
    'accY',            {'float', 4}, ... % Target acceleration in Y dimension, m/s
    'accZ',            {'float', 4}, ... % Target acceleration in Z dimension, m/s
    'ec[16]',          {'float', 16x4}, ... % Tracking error covariance matrix, [4x4] in range/azimuth/elevation/doppler coordinates
    'g',               {'float', 4}, ... % Gating function gain
    'confidenceLevel'  {'float', 4}, ... % Confidence Level

Target Index TLV

Size: sizeof (tlvHeaderStruct) + sizeof(uint8) x numberOfPoints (NOTE: here the number of points are for frame n-1)

The Target Index TLV consists of an array of target IDs. A targetID at index i is the target to which point i of the previous frame’s point cloud was associated. Valid IDs range from 0-249.

targetIndex = struct(...
'targetID',         {'uint8', 1});    % Track ID

Other Target ID values:

Presence Indication TLV

Size: sizeof (tlvHeaderStruct) + sizeof(uint32)

The Presence Indication TLV consists of a single uint32 value to provide a binary indication of presence in the presence boundary box. A value of 1 represents presence detected and 0 represents no presence detected.

Vital Sign Info TLV

Size: 136 Bytes

VS_Feature = struct(...
    'id',                                       {'uint16', 2},...     % Target ID used for XYZ location
    'rangebin',                                 {'uint16', 2},...     % range bin for XYZ location
    'breathingDeviation',                       {'float', 4},...      % deviation of breathing measurement over time
    'heart_rate',                               {'float', 4},...      % Heart Rate Measurement
    'breathing_rate',                           {'float', 4},...      % Breath Rate Measurement
    'VitalSigns_Heart_CircularBuffer[15]',      {'float', 4x15}),...  %Buffer of heartrate waveform
    'VitalSigns_Breath_CircularBuffer[15]',     {'float', 4x15}),...  %Buffer of breathrate waveform

📝 NOTE - Presence Detection and Smoothing

Visualizer

Patient status is determined in the python source code for the visualizer based on the breathing deviation value passed through the Vital Signs Info TLV.

The breathing deviation is calculated by looking at the variation of the last 40 values in the breathing waveform. It is recalculated every 15 frames when the vital signs processing is performed.

First the visualizer looks to see if a track is present. If no track is present, no data is displayed. If a track is present and the breathing deviation is zero, no patient data is displayed. However, the range bin where the person’s track was last detected will still be displayed for debugging purposes.

If the breathing deviation value falls below 0.02 “holding breath” is detected and the breath rate value is no longer displayed.

Additionally, the visualizer smooths the heart rate results by calculating the median of the latest 10 results.

Need More Help?

• Search for your issue or post a new question on the mmWave E2E forum

• For details regarding the demo software algorithm and implementation, please refer to the following documents available in the docs folder of the People Tracking lab directory

• 3D People Tracking Demo Software Implementation Guide

• Group Tracker Algorithm Implementation Guide