Video Doorbell Users Guide

Table of Contents

Introduction

Outdoor video surveillance systems are used to detect and identify people in a variety of different environments. Identifying people in a video frame can be extremely accurate in a controlled setting. But, many of the locations that video surveillance systems get installed in are noisy environments. Additionally, capturing and recording camera data can be a very power-intensive process, which drains battery life for battery-powered systems.

Using radar to detect movement and alert the video surveillance system increases battery life by keeping the camera off in times when it is unnecessary. Radar outperforms similar presence detection technologies such as PIR, ultrasonic, or just the camera operating alone, by reducing the number of false positive detections. This leads to longer battery life for camera systems.

More than just saving battery power, the radar also adds functionality to the system. Its precise distance measurements keep the camera from unnecessarily recording neighbors as they pass on the sidewalk. Its long range detects people at 15+ meters. It can detect people and objects in conditions that are challenging for cameras such as darkness, glare, smoke, fog, and rain.

Quickstart

Prerequisites

Prerequisite 1 - XDS Firmware

If using an IWRL6432BOOST EVM, the latest XDS110 firmware is required. The XDS110 Firmware runs on the microcontroller onboard the IWRL6432BOOST, which provides the JTAG Emulation and serial port communication over the XDS110 USB port. Packet loss has been observed on the serial port (COM ports) with some versions of the XDS110 firmware which can cause the demo to fail.

The latest XDS110 firmware is installed with the latest version of Code Composer Studio. CCS version 12.4 or later required.

---

Prerequisite 2 - PC CPU and GPU Resources

The Industrial Visualizer requires sufficient GPU and CPU resources to process the incoming UART data every frame and run the demo smoothly. Users with insufficient PC resources may notice occasional visualizer lag or missed frames in the log file. The demo should still run on most PCs regardless of hardware resources, but performance might vary.

---

Requirements

Hardware Requirements

Item	Details
Device	IWRL6432BOOST or IWRL6432AOPEVM
Computer	PC with Windows 10. If a laptop is used, please use the ‘High Performance’ power plan in Windows.
Micro USB Cable

Software Requirements

Tool	Version	Download Link
TI mmWave L SDK	5.4.0.x	TI mmWave SDK 5.4.0.x
Uniflash	Latest	Uniflash tool is used for flashing TI mmWave Radar devices. Download offline tool or use the Cloud version
Code Composer Studio	CCS 12.4 or later	Code Composer Studio
TI Radar Toolbox	Latest	Radar toolbox should be downloaded to access binaries and source code. Download Instructions in the readme file.

1. Configure the EVM for Flashing Mode

Follow the instructions for Hardware Setup of Flashing Mode

2. Flash the EVM using Uniflash

Flash the binary listed below using UniFlash. Use the AOP version for the IWRL6432AOPEVM, and the non-AOP version for the IWRL6432BOOST EVM. Both can be found at <RADAR_TOOLBOX_INSTALL_DIR>\source\ti\examples\Video_Doorbell\prebuilt_binaries\. Follow the instructions for using UniFlash

• video_doorbell_demo_aop.Release.appimage OR video_doorbell_demo.Release.appimage

3. Configure the EVM for Functional Mode

Follow the instructions for Hardware Setup of of Functional Mode

4. Open the Radar Toolbox Industrial Visualizer

Navigate to the Radar Toolbox download location and find the Industrial Visualizer directory at <RADAR_TOOLBOX_INSTALL_DIR>\tools\visualizers\Applications_Visualizer\Industrial_Visualizer\. Then, run the visualizer either directly from source with python gui_main.py, or as an executable by running the Industrial_Visualizer.exe

📝 NOTE
In general, running the visualizer from the Python source runs faster and results in fewer errors over long periods of time. If running Python directly from source, use Python version 3.7.3, and run the setUpEnvironment.bat script first to ensure all the correct packages and versions are installed.

📝 NOTE
If running the visualizer from the Python source, users may need to uncomment line 5 of gui_main.py, which gives the gui_main.py access to the other files in the directory.

5. Connect the Device

1. Look for the “XDS110 Class Application/User UART” COM port in your Device Manager, and select this port in the GUI. If you do not see this, you need to install the XDS110 drivers listed in the Requirements section above.
   
   

2. Make sure the correct device is selected and choose the COM port found in the previous step, and then select “Connect”.
   

7. Run the Demo

In the Industrial Visualizer, select the “Video Doorbell” demo. Click on “Select Configuration” button and select the mode_1_profile_aop.cfg (or the mode_1_profile_boost.cfg for IWRL6432BOOST). Select Start without Send Configuration, and the sensor should start immediately. The device should begin running with the pre-programmed configuration files in the multi_pass_mode_profiles/ folder in the Video Doorbell source code.

Doorbell-Specific Features

While the video doorbell demo was built on top of the motion and presence demo for MMWAVE-L-SDK 5.4.0.x, a number of features were added to it to increase detection range within a wide field of view, decrease the overall power consumption and reduce the number of false positive detections. For a thorough explanation of the motion & presence detection chain, consult the documentation in the docs/ folder of the MMWAVE-L-SDK 5.4.0.x. The below section describes the modifications made to the motion & presence detection chain to create the video doorbell detection chain. Video doorbell features that involved changes to the data processing units can be found in x432 datapath documentation.

Mode switching

Mode switching - Feature Description

The video doorbell demos now include a mode-switching feature that allows users to cycle between lower power and higher performance modes depending on whether presence is detected. The intended usage is that the radar would be in the first pass low-power mode until it detects presence, which would be confirmed by the second and third pass modes, ultimately turning on the video camera once presence is confirmed in the highest performance third-pass mode. This feature is built into the video doorbell demo, which has a pre-built binary, and a build option in the source code. It can be run with video doorbell demo in the industrial visualizer.

Mode switching - Creating configurations

If the profiles used in multi-pass mode need to be dramatically modified, TI recommends starting development by testing new profiles in single pass mode by defining the STAY_IN_FIRST_PASS_MODE symbol in motion_detect.h. This is meant to simplify development.

⚠️ WARNING: Configuration Files with Video Doorbell Demo
The video doorbell demo does NOT take a .cfg file via the CLI. Rather, the .cfg files are programmed into the video doorbell software in the .c code found in the multi_pass_mode_profiles/ folder. The configuration files found in the toolbox for the demo can be used by copy+pasting them into the .c code.

Mode switching - Measuring Power

Switching rapidly between multiple modes can cause unexpected behavior when operating the INA228 current measurement amplifiers on the IWRL6432AOPEVM and IWRL6432BOOST. TI recommends users estimate power consumption in multi-pass mode by measuring the power consumption for both profiles in a single-pass mode, then computing the average amount of time spent in either mode when operating in multi-pass mode.

Mode switching - CLI structure

The profileSwitchCfg CLI argument is given below. It has been modified from the MMWAVE-L-SDK CLI to support a third mode.

profileSwitchCfg <switchCfgEnable> <numFramesForMode1toMode2> <numFramesForMode2toMode3> <numFramesForMode3toMode1> <numFramesForMode2toMode1>

Mode switching - CLI Argument Descriptions

switchCfgEnable : Set to 1 to enable mode switching

numFramesForMode1toMode2 : Number of frames needed (in a row) with a detection in order to transition from mode 1 to mode 2

numFramesForMode2toMode3 : Number of frames needed (in a row) with a detection in order to transition from mode 2 to mode 3

numFramesForMode3toMode1 : Number of frames needed (in a row) without a detection in order to transition from mode 3 to mode 1

numFramesForMode2toMode1 : Number of frames needed (in a row) without a detection in order to transition from mode 2 to mode 1

Advanced Clutter Removal

Advanced Clutter Removal - Feature Description

The video doorbell demos include two more secondary options for clutter removal : Zero Doppler Difference Frames, and a Minimum Velocity to Include.

The Zero Doppler Difference Frame option allows the video doorbell demo to detect slow-moving objects without running the entire minor motion chain by subtracting the 0-doppler bin from the previous frame from the zero doppler bin in the current frame for each range bin and each antenna. This is recommended for low-power applications that would benefit from minimizing the time between chirps.

The Minimum Velocity to Include is an optional argument that can be used when the clutter removal method is set to 1 (Remove Zero Doppler Bin), but not when it is set to 2 (Zero Doppler Difference Frames). Users can specify a minimum velocity to include when they want to exclude low-velocity points, as well as the points whose velocities are measured to be 0 m/s. This can be useful to avoid slow-moving objects and spectral leakage from the 0-doppler bin.

The clutter removal feature in the motion and presence demo removes all detections from stationary objects in the scene. When there are few chirps and a large maximum velocity, this removes all detections whose absolute velocities are less than one-half the velocity resolution of the chirp configuration.

However, if the velocity resolution increases, the low-velocity moements can result in unwanted detections. The graphic below illustrates this. Therefore, the video doorbell adds an optional second argument to the clutterRemoval CLI command that lets the user set a minimum velocity for detections.

Advanced Clutter Removal - CLI structure

The clutterRemoval CLI argument is given below. It has been modified from the MMWAVE-L-SDK CLI to support the difference frame method.

clutterRemoval <0 = Disabled, 1 = Remove Zero Doppler Bin, 2 = Zero Doppler Difference Frames> <minVelocityToInclude (Optional)>

Advanced Clutter Removal - CLI Argument Descriptions

The first parameter in this CLI line is the clutter removal method. Selecting 0 will disable the clutter removal altogether so the radar can detect stationary objects. Selecting 1 will run clutter removal by removing the zero-doppler bin from the detection matrix, which will ensure that no stationary objects are detected. It will be the most effective way to remove radar returns from stationary objects, but it will also struggle to detect moving objects whose speeds are slower than half of the velocity resolution. Selecting 2 will run clutter removal via difference frames, which subtracts the 0-doppler bin of the previous frame from the current frame to reduce the amount of power that stationary objects can return to the radar. This method will be more prone to noise than method #1 (removing 0-doppler bin altogether), but will not struggle to detect slow-moving objects as method #1 will.

The second optional parameter in this CLI line sets the minimum absolute velocity (in meters/second) to ignore if enabled. This is only enabled if the clutter removal method is 1. The minimum velocity to include only runs on the major motion radar cube, and not the minor motion radar cube.

SNR vs Range and Angle - CLI structure

The rangeSNRCompensation filtering can be implemented with the following CLI command:

rangeSNRCompensation <enabled> <compensationRange> <compensationSNR> <minimumCompensationDistance> <maximumCompensationDistance> <MinimumCompensationAngle> <MaximumCompensationAngle> <MinimumSecondaryCompensationAngle> <MaximumSecondaryCompensationAngle> <SecondaryCompensationSNRDelta>

SNR vs Range and Angle - CLI Argument Descriptions

enabled : Set to 1 to enable the rangeSNRCompensation. Set to 0 to disable.

compensationRange : The range from the radar in meters at which points with SNR values above the detection threshold (specified by compensationSNR) should be retained. Points with SNR values below this threshold at this range are discarded.

compensationSNR : The minimum SNR value needed for a point to be considered a detection at the compensationRange. Points with SNR values below the compensationSNR at the compensationRange are discarded.

minimumCompensationDistance (optional) : Range in meters at which the compensation value saturates at the SNR corresponding to the SNR at minimumCompensationDistance away

maximumCompensationDistance (optional) : Range in meters at which the compensation value saturates at the SNR corresponding to the SNR at maximumCompensationDistance away

minimumCompensationAngle (optional) : Minimum angle in degrees where the compensation is applied. If not populated, the compensations is applied throughout the entire field of view. If populated, points whose azimuth angle is less than this value will be detected with the cfarCfg’s threshold value or the secondary compensation threshold (if enabled).

maximumCompensationAngle (optional) : Maximum angle in degrees where the compensation is applied. If not populated, the compensations is applied throughout the entire field of view. If populated, points whose azimuth angle is greater than this value will be detected with the cfarCfg’s threshold value or the secondary compensation threshold (if enabled).

secondaryMinimumCompensationAngle (optional) : Minimum angle in degrees where the 2nd pass compensation is applied. If not populated, no second pass compensation occurs. If populated, points whose azimuth angle is less than this value will be detected with the cfarCfg’s threshold value.

secondaryMaximumCompensationAngle (optional) : Maximum angle in degrees where the 2nd pass compensation is applied. If not populated, no second pass compensation occurs. If populated, points whose azimuth angle is greater than this value will be detected with the cfarCfg’s threshold value.

SecondaryCompensationSNRDelta (optional) : The difference between the primary and secondary SNR compensations in dB.

📝 NOTE
For the range compensation to work correctly, the SNR corresponding to the maximumCompensationDistance set in the 5th argument of the rangeSNRCompensation (minus the secondaryMaximumCompensationAngle if applicable) must be less than or equal to the thresholdScale set in the cfarCfg line. Since the rangeSNRCompensation code filters the CFAR, if no points are detected in the CFAR, no points can be detected in the rangeSNRCompensation. To ensure that this problem does not arise, the user can set the maximumCompensationDistance (rangeSNRCompensation[5]) equal to the compensationRange (rangeSNRCompensation[2]), and set the compensationSNR (rangeSNRCompensation[3]) in the rangeSNRCompensation equal to the thresholdScale in the cfarCfg (cfarCfg[6]). Below shows an example where the compensationSNR in the rangeSNRCompensation line is equal to 8, and the secondaryMaximumCompensationAngle is equal to 2. Their difference of 6 is equal to the detectionSNR in the cfarCfg line. The compensationRange is also equal to the maximumCompensationDistance, meaning that there is no attempt to compensate the SNR below the value of 8 dB at 12 meters at within [-30,30] degrees, and 6 dB at 12 meters within [-50, 50] degrees.
rangeSNRCompensation 1 12 8 5 12 -30 30 -50 50 2
cfarCfg 2 4 3 2 0 6 0 0.5 0 1 1 1

Modified Zone Boundaries

Modified Zone Boundaries - Feature Description

The motion and presence state machine included in the MMWAVE-L-SDK has been extended in the doorbell demo to allow for radial “arc” shaped boundary boxes. This enables system alerts when a person is within a specific radius of the video doorbell.

Modified Zone Boundaries - CLI structure

An mpdBoundaryArc can be implemented with the following CLI command:

mpdBoundaryArc <zoneIdx> <minimumRadius> <maximumRadius> <minimumTheta> <maximumTheta> <minimumZ> <maximumZ> <maxSNRForMajor> <numPointsForMajor>

Modified Zone Boundaries - CLI Argument Descriptions

zoneIdx : Number used to order the zones, must be unique from other zones, including mpdBoundaryBoxes

minimumRadius : Minimum radius in meters setting the zone boundary arc

maximumRadius : Maximum radius in meters setting the zone boundary arc

minimumTheta : Minimum azimuth angle in degrees setting the zone boundary arc

maximumTheta : Maximum azimuth angle in degrees setting the zone boundary arc

minimumZ : Minimum Z coordinate in meters setting the zone boundary arc

maximumZ : Maximum Z coordinate in meters setting the zone boundary arc

maxSNRForMajor (optional) : Maximum SNR of the points in the cluster needed to transition this zone from empty to major. Works in place of the majorStateCfg.

numPointsForMajor (optional) : Number of points in the cluster needed to transition this zone from empty to major. Works in place of the majorStateCfg.

📝 NOTE
The mpdBoundaryArc logic runs on a cylindrical coordinate system, meaning that the Z coordinate is not included in the calculation of the radius and azimuth angle.

Zone-specific detection thresholds - CLI structure

mpdBoundaryBox <zoneIdx> <minimumX> <maximumX> <minimumY> <maximumY> <minimumZ> <maximumZ> <maxSNRForMajor> <numPointsForMajor>

Zone-specific detection thresholds - CLI Argument Descriptions

zoneIdx : Number used to order the zones, must be unique from other zones, including mpdBoundaryArcs

minimumX : Minimum X coordinate in meters setting the zone boundary box

maximumX : Maximum X coordinate in meters setting the zone boundary box

minimumY : Minimum Y coordinate in meters setting the zone boundary box

maximumY : Maximum Y coordinate in meters setting the zone boundary box

minimumZ : Minimum Z coordinate in meters setting the zone boundary box

maximumZ : Maximum Z coordinate in meters setting the zone boundary box

maxSNRForMajor (Optional) : Maximum SNR of the points in the cluster needed to transition this zone from empty to major. Works in place of the majorStateCfg.

numPointsForMajor (Optional) : Number of points in the cluster needed to transition this zone from empty to major. Works in place of the majorStateCfg.

Single Point Detection

Single Point Detection - Feature Description

Single points can now be used for detecting objects on the edges of the field of view. When people approach the radar at extreme angles, typically larger than 60 degrees, it becomes harder to create multiple points for a single detection. Therefore, the user can now dictate that single points with high SNR and high angles can be used to enable detection. This skips the clustering step altogether.

⚠️ WARNING: singlePointDetectionOnFoVEdge and Boundary Boxes
The singlePointDetectionOnFoVEdge command is only designed to work with the mpdBoundaryArc zones. It will fail if used with mpdBoundaryBoxes.

Single Point Detection - CLI structure

The singlePointDetectionOnFoVEdge can be implemented with the following CLI command:

singlePointDetectionOnFoVEdge <minRange> <maxRange> <startingAngle> <detectionSNR>

Single Point Detection - CLI Argument Descriptions

minRange : The minimum range in meters that sets the boundary for this detection criteria.

maxRange : The maximum range in meters that sets the boundary for this detection criteria.

startingAngle : The angle in degrees that sets the boundary for this detection criteria on both the positive and negative sides of the field of view (abs(detection_angle) > abs(startingAngle)).

detectionSNR : The SNR needed for a single point to trigger a detection at angles whose absolute values are greater than +/- startingAngle degrees.

Power Measurement Interval

Power Measurement Interval - Feature Description

Low power consumption is a critical part of a Video Doorbell to enable wireless products. Unfortunately, this ultra-low power consumption may be hard to see when evaluating the IWRL6432 because effective evaluation may require the users to stream large amounts of TLV data over the UART for the visualizer. For example, streaming out the point cloud TLV, along with the range-azimuth heatmap, or other features enabled by the guiMonitor CLI line may be hundreds of bytes long, forcing the IWRL6432 to stay powered-on for longer than it would in operation. When the device is deployed in a product, it is much more likely that only time the UART would turn on would be when presence is detected, or at least not every frame.

To enable users to evaluate the power consumption of the device when the UART is less active, the powerMeasurementInterval CLI line allows the user to specify how often the radar should broadcast TLV data over the UART.

Power Measurement Interval - CLI structure

The powerMeasurementInterval can be implemented with the following CLI command:

powerMeasurementInterval <UARTInterval>

Power Measurement Interval - CLI Argument Descriptions

UARTInterval : The number of seconds between UART messages.

📝 NOTE
TI recommends setting this value close to 30 seconds for the most accurate ambient power measurement. It takes about 25.3 seconds for the INA228 current measurement amplifiers on the device to clear their buffers. So, the best strategy is to send the UART data in one frame, then run the device for about 30 seconds. After the 30 seconds have passed, the power can be measured, then sent off via the UART, restarting the cycle.

Point Bounds

Point Bounds - Feature Description

To enable users to filter out points whose Z-coordinates are too high or too low, TI offers the pointBounds CLI command. Radar returns outside the pointBounds CLI command are not detected or included in the point cloud list. This feature is meant to target objects whose Z coordinates are too high (say for example - swinging trees outdoors), or too low (robot vacuums, pets, small animals).

Point Bounds - CLI structure

pointBounds <minimumX> <maximumX> <minimumY> <maximumY> <minimumZ> <maximumZ>

Point Bounds - CLI Argument Descriptions

minimumX : Minimum X coordinate for detected points (exclude points whose X coordinate is below this value)

maximumX : Maximum X coordinate for detected points (exclude points whose X coordinate is above this value)

minimumY : Minimum Y coordinate for detected points (exclude points whose Y coordinate is below this value)

maximumY : Maximum Y coordinate for detected points (exclude points whose Z coordinate is above this value)

minimumZ : Minimum Z coordinate for detected points (exclude points whose Z coordinate is below this value)

maximumZ : Maximum Z coordinate for detected points (exclude points whose Z coordinate is above this value)

Improved Angle Estimation

To improve angle estimation for incoming objects and people, especially at the edges of the field of view, TI offers the AOA SVC DPU, which is documented in greater detail in the AOA_SVC_DPU_Implementation_Details.pdf.

Results

Configuration	Range @ 0°	Range (m) @ +/- 30°	Range (m) @ +/- 60°	Range (m) @ +/- 75°	Average Power Consumption (mW)
Prebuilt Release Binary 3-Pass Mode	>15	13	9	7	4.4

📝 NOTE
Range can be increased or decreased depending on a variety of factors, including antenna design, board design and radome. Results shown above are based off the doorbell detection chain, which includes false error rejection as well.

📝 NOTE
Power results posted in the table above show the best potential results, which require limiting the UART output to the bare minimum and removing much of the unnecesary processing and peripherals found on the IWRL6432 device. Measured power using the .cfgs files will be higher without these changes.