AWR6843 CPD With Classification Users Guide
Table of Contents
Overview
This lab demonstrates the use of TI 60GHz mmWave sensors for Vehicle Occupancy Detection (VOD) and Child Presence Detection (CPD) for up to 3 rows of seats. This lab outputs a 3D point cloud over UART, which is then utilized by an Occupancy Detection State Machine in the accompanying MATLAB visualizer.
This lab is compatible with xWR6843 devices, and is targeted for the xWR6843ISK-ODS and xWR6843AOP evaluation modules (EVMs). This lab can also run on xWR6843ISK module. With a narrow FOV in elevation, xWR6843ISK can be considered for front mounting instead of overhead mounting.
⚠️ Data Quality on AOP EVMs
Due to differences in the FOV between the ISK-ODS and AOP EVMs, performance may be slightly more optimal on the ISK-ODS EVM.
Requirements
Hardware Requirements
Item | Details |
---|---|
xWR6843 Evaluation Board | xWR6843ISK-ODS ES2.0 Evaluation Board, Or xWR6843AOP ES2.0 Evaluation Board Or xWR6843ISK-ODS ES2.0 Evaluation Board |
MMWAVEICBOOST Carrier Board | OPTIONAL: MMWAVEICBOOST Carrier Board for CCS based development and debugging |
Computer | PC with Windows 10. If a laptop is used, please use the ‘High Performance’ power plan in Windows. |
Micro USB Cable | Due to the high mounting height of the EVM, a 15ft+ cable or USB extension cable is recommended. |
Power Supply | 5V, >3.0A with 2.1-mm barrel jack (center positive). The power supply can be wall adapter style or a battery pack with a USB to barrel jack cable. Due to the high mounting height, a barrel jack extension cable is recommended to extend the 5v power supply cable. |
Note: Both AWR6843 and IWR6843 are supported and can be used interchangeably. Please consult the respective datasheet for details on the differences between the devices.
Software Requirements
Tool | Version | Download Link |
---|---|---|
TI mmWave SDK | 3.5.0.4 | TI mmWave SDK 3.5.0.4 and all the related tools are required to be installed as specified in the mmWave SDK release notes |
MATLAB Runtime | 2019a (9.6) | Exact version required. https://www.mathworks.com/products/compiler/matlab-runtime.html |
UniFlash | Latest | UniFlash tool is used for flashing TI mmWave Radar devices. Download offline tool or use the Cloud version |
Getting familiar with the device
🛑 Run Out of Box Demo
Before continuing with this lab, users should first run the out of box demo for the EVM. This will enable users to gain familiarity with the sensor’s capabilities as well as the various tools used across all labs in the Radar Toolbox.
Calibration
In order to get the best detection performance, users should run the Out of Box Demo Range Bias and Rx Channel Gain/Phase Measurement and Compensation procedure to get the calibration coefficients for their Antenna Module and replace the default coefficients in the desired configuration file which can be found in <PACKAGE_LOCATION>\tools\visualizers\InCabin_CPD_with_Classification_GUI\config_file\
with the values returned by the calibration procedure.
The following command in the chirp configuration file needs to be updated with the calibrated values.
compRangeBiasAndRxChanPhase 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0
(This is the default command)
The updated command will look similar to the following (but with different values)
compRangeBiasAndRxChanPhase 0.1044657 -0.11493 0.79419 -0.13123 -0.83478 -0.08484 -0.77560 0.04535 0.72226 0.09491 0.81415 -0.41129 -0.69214 -0.27069 -0.64630 0.20770 0.63818 0.67627 0.69116 -0.90146 -0.43289 -0.81061 -0.48209 0.72995 0.50214
To run the calibration procedure, please follow the instructions provided in the Out of Box demo HTML documentation given in the calibration section of the file listed below. <MMWAVE_SDK3_INSTALL_DIR>\packages\ti\demo\xwr68xx\mmw\docs\doxygen\html\index.html
Quickstart
1. Configure the EVM for Flashing Mode
Instructions for EVM setup for Flashing can be found in the EVM Setup Guide
2. Flash the EVM using UniFlash
Flash the binary listed below using UniFlash. Follow the instructions for using UniFlash <PACKAGE_LOCATION>\source\ti\examples\InCabin_Sensing\AWR6843_CPD_with_Classification \prebuilt_binaries\occupancy_detection_3d_68xx.bin
3. Configure the EVM for Functional Mode
Instructions for EVM setup for functional mode can be found in the EVM Setup Guide
4. Mount the EVM and Create Test Environment
Mount the sensor at the desired location securely so that the sensor is stable during runtime. Update the desired configuration file so that it reflects the device’s location. More information on mounting can be found in the section below.
5. Launch the Visualizer
⚠️ MATLAB Runtime Version R2019a (9.6)
Exact version R2019a (9.6) required.
- Launch the visualizer executable located at
<PACKAGE_LOCATION>\tools\visualizers\InCabin_CPD_w_Classification_GUI\
- Double click to launch AWR6843_CPD_w_Classification_visualizer.exe.
- A black console window will appear. After 30-60sec, a configuration window will appear as shown below.
6. Config Visualizer
Select the EVM Control and Data COM ports using the respective dropdowns.
You should see the Chirp configuration file automatically selected in the Configuration dialogue. But if the configuration file is not the correct one, browse to select a file provided in the config_file directory:
<PACKAGE_LOCATION>\tools\visualizers\InCabin_CPD_w_Classification_GUI\config_file
Select Real Demo mode and press Go! to start the Visualizer.
Select Index for recording file if users plan to record the point cloud for future playback
The Visualizer window will appear as shown below and after several seconds, the visualizer should start showing the point cloud and occupancy decisions in their respective panels:
7. Understand the Visualizer
The Visualizer window is divided into five panels as shown in the annotated picture above:
Statistics: This section prints real-time information from the target: Frame number, both processed and (target), and a Zone table that displays the occupancy state, number of detections in the zone, and average SNR of the detections. When the Count button is clicked, the Points column becomes the “Frame” count (total number of frames that have been counted), and the SNR column becomes “Count” - the number of frames with positive occupancy state.
Chirp Configuration: This panels shows the static chirp configuration loaded at startup time.
Control: This panel provides an Exit button, a Record button (saves frame by frame point cloud and occupancy data). In addition, there are two sub-panels, which independently allows either the Occupancy Display to be paused, or selection/pausing of a display in the Point Cloud panel.
Occupancy: This panel pictorially displays the output of the state machine, which are the occupancy decisions. The occupancy display assumes 5 seat for 2 row car and 8 seat for 3 row car.
Point Cloud: This panel can display one of several displays: “2D Point Cloud” shows and overhead view of the detected points, along with the zone boundaries when that zone decision is positive. The zone boundary turns red when there is excessive movement in the zone. The state machine will flag all zones “overloaded” when any one zone is flagged. This is because excessive energy in one zone may cause excessive energy in neighboring zones as well. In this condition, the state machine will freeze processing (it will hold the current empty or occupied state) until the overload condition abates, or for a minimum of 2 frames. This “2D Point Cloud” is capable of display the occupancy decision of any customer added zone, such as footwell zone, or intruder detection zone.
8. Re-playing previously saved output
As described in the previous section, the AWR6843_CPD_w_Classification_visualizer can save the EVM UART output (point cloud information) in files named fHistRT_xxxx
.mat where each file can contain up to 1000 frames of information. These saved fHist files can be re-played offline by running AWR6843_CPD_w_Classification_visualizer.exe
as shown below:
Navigate to
<PACKAGE_LOCATION>\tools\visualizers\InCabin_CPD_w_Classification_GUI\
Copy the previously saved fHistRT_xxxx.mat files to the directory above
Launch the same visualizer AWR6843_CPD_w_Classification_visualizer.exe, select the mode as Playback instead of Real Demo and select the preferred file index for the recorded file, and and press Go! to start the playback as shown in the figure below. In playback mode, the port numbers are no longer relevant, but a matching chirp configuration is still needed.
Algorithm Overview
This demo can be viewed as two pieces: the device running the radar code and the host running the visualizer. The device code is based on the Overhead 3D People Tracking Demo, and additional information related to that part of the algorithm can be viewed in the 3D People Tracking Demo Implementation Guide
In order to increase performance of detection of nearly-static objects such as people sitting still in the vehicle, this lab uses multiple frames to generate range-angle heatmap, rather than a single frame. By using a sliding window of multiple frames, there is a finer granularity of detection with respect to time. The number of frames in this window is configured by the predefined symbol HEATMAP_SLIDING_WINDOW (= 4 by default) which is added to both the MSS and DSS projects
The high level flow for this lab is described here:
Range processing is generic (nothing new here), and is controlled by the HWA (Hardware Accelerator) to produce the radar cube. All frame rate algorithms are performed by the DSP. Input configuration and output scaling of the point cloud is performed by the MSS ARM. The zone mapping, occupancy detection state machine, and the classification (optional) is running on host inside visualizer.
Doppler Binning
The first step of frame processing is range-azimuth-elevation 3D heatmap generation. Previously after clutter removal, for each range bin, all the chirps are used for covariance matrix generation during Capon (also called MVDR) spectrum calculation. In order to improve 3-row coverage, one of the idea is to average several neighboring chirps to improve SNR, which is similar to apply a low pass filter. Later on, we proposed a more configurable solution called “Doppler binning”, e.g. after clutter removal, Doppler FFT is applied, and only a few bins (configurable) of Doppler output are selected for angle spectrum generation. The procedure of picking few lower Doppler bins and remove higher Doppler bins is equivalent to implement a low pass filter in frequency domain.
Note that Doppler FFT is only meaningful for evenly spaced chirps. However, the collected chirps in the radar cube comes from HEATMAP_SLIDING_WINDOW (4) frames, and only chirps from the same frame are evenly spaced in time. Therefore, we apply Doppler binning for the chirps that belongs to one frame, and repeated this procedure for 4 (HEATMAP_SLIDING_WINDOW) times. For example as listed in the table below, if we have 54 chirps in one frame, and total 216 chirps to process. We will apply 64 point Doppler FFT for every 54 chirps, pick up 10 bins out of each Doppler output, and total 40 bins to feed into next stage in covariance matrix generation and Capon beamforming. As a comparison, the original algorithm uses all 216 chirps in the next stage.
Doppler Binning Disable | Doppler Binning Enable | |
---|---|---|
Time Domain chirps to process | 54 * HEATMAP_SLIDING_WINDOW (4) = 216 | 54 * HEATMAP_SLIDING_WINDOW (4) = 216 |
Each Doppler FFT size | N/A | 64 |
Each Doppler FFT output bins | N/A | 10 |
Number of Doppler FFT to run per antenna | N/A | HEATMAP_SLIDING_WINDOW (4) |
Total Doppler FFT output bins per antenna | N/A | 40 |
Total number of vectors for covariance matrix generation | 216 | 40 |
Range-Azimuth-Elevation Heatmap Generation
After Doppler binning, the selected Doppler bins will be used for covariance matrix calculation and azimuth-elevation angle spectrum generation. The calculation performed is the standard Capon (MVDR) algorithm used in people tracking demo. The 3D range-azimuth-elevation heatmap is treated as a 2D heatmap with azimuth and elevation dimension reduced to one dimension (angleInd = azimInd + elevInd * azimDim).
CFAR Detection
CFAR is performed next. The variant used is a two-pass CASO (cell averaging smaller of) with one significant enhancement: An additional “near range” search window is now used in the first (range) pass. These new parameters are included in the CFAR configuration command, dynamicRACfarCfg. This results in four window averages being calculated, a “near” and “far” average on both sides of the cell under test. The minimum of these four averages is found, thresholded, and if less than the peak power, the cell is kept as an initial detection, which is then confirmed by the second pass.
So, with the default configuration, the “far range” and “near range” windows appear as follows, overlapping by 1 cell:
The outputs of the range-angle CFAR are:
- Range and angle detections, specified as indexes into the range-angle heatmap.
- Noise. This is the weakest cell found in the search window.
- SNR estimate. This is the peak value of the search window divided by the Noise value.
Avoiding False Detections Due to Static Leakage
Static signals result in an leakage after clutter removal due to temperature variation from chirp to chirp. The leakage level is typically about 55dB below the static signal level. If the static signal is strong enough, the leakage can appear above the thermal noise floor and causes false detection. To reduce false detection, two logic are added to the CFAR module:
- Adjust CFAR threshold K0 to be angle dependent: introduce an angle related ratio to increase CFAR threshold around bore sight.
- Adjust CFAR threshold K0 to be range dependent: introduce an range related ratio to increase CFAR threshold at near range.
The reason behind is that the static signal is strongest at bore sight and near range.
High Accuracy Angle Estimation
Following CFAR, a zoom-in azimuth-elevation heatmap is generated for ranges-angle bins where the initial detections were found. This is an “angle only” processing, and estimates the fine azimuth and elevation angle of each range-angle detection found by CFAR. It is configured with the dynamic2DAngleCfg CLI command. The one difference from people tracking demo processing is the Doppler indexes are not scanned, because we have not processed any Doppler data. The final point cloud is complete, and is sent to the MSS for scaling and output.
Output Point Cloud Format
The output of DSP’s frame processing is the point cloud: (values per each detected point)
- Range, azimuth and elevation angles (spherical coordinates)
- Noise. This is the weakest cell found in the CFAR search window.
- SNR estimate from CFAR.
- Doppler (set to zero)
Zone Mapping
The point cloud is sent to host through UART. Inside visualizer, the point cloud is first transformed into car coordination. Then mapped to the zones defined through a number of cuboids. A common practice is to define each seating area as one zone for localization/seatbelt reminder application. In addition, users can also define footwear region and intruder warning region through zone definition.
Occupancy Detection State Machine
After zone mapping, the Occupancy Detection State Machine is run to determine the occupancy status for each zone using a number of factors that can be configured by the occStateMach
CLI command.
The general idea is that a zone can be detected as occupied only if the total number and quality (average SNR) of the detected point cloud within the zone are higher than the pre-determined thresholds. The occStateMach
CLI holds the condition (threshold) to enter the occupancy state, the condition (threshold) to stay in the occupancy state, and the condition(threshold) to leave the occupancy state.
Users can define same or different zoneType for each zone through zoneNeighDef
CLI. By introduce zoneType,
the system can have multiple sets of occupancy detection thresholds. For example, second-row middle seat often experience stronger SNR and higher rate of false detection. Therefore, it makes sense to use a higher occStateMach
threshold for the middle seat; the first row footwear may experience weaker SNR and higher rate of miss detection. Therefore, users can consider to use a lower occStateMach
threshold for the first row footwear. We have also introduce the logic through zoneNeighDef
CLI to reduce miss detection when the movement of a certain seat generates point cloud at neighboring seat.
Classification
In this demo, we also added an optional feature for baby/adult classification. When a seat is occupied, the classification logic can be applied to separate the baby from adult. The classification logic can be enabled through classParam CLI, and is based on accumulated SNR and volume of the total detected points. The SNR and volume threshold needs tuning for different sensor mounting position. Two example configuration files are provided as an example. They are
- config_file\vod_6843_aop_overhead_2row_classification.cfg
- config_file\vod_6843_ODS_overhead_2row_classification.cfg
The classification results is displayed as text added to the occupancy display as shown in the figure below.
3-row enhancement
In this demo, we have implemented the Doppler binning feature to enhance the 3-row coverage. By removal the higher bins of Doppler output, the SNR is improved. An example below demonstrated how the Doppler binning feature improve the 3-row coverage.
3-row coverage enhancement | ||
---|---|---|
Baby is placed in the 1st-row passenger side, rear-facing AWR6843ISK-ODS is mounted in the center right behind the 2nd-row Performance without Doppler binning dopplerBinSelCfg -1 0 64 0 4 | ||
Baby is placed in the 1st-row passenger side, rear-facing Same sensor mounted in the same location Doppler binning feature is added to improve the SNR dopplerBinSelCfg -1 1 64 0 4 |
With improved SNR, users will likely observe more points cloud out of the zone due to multipath, which can potentially cause false detection in occupancy decision. To help with this situation, we recommend users to set higher thresholds in state machine for the seat zones that closer to the sensor (2nd-row and 3rd-row in the above example), and set lower thresholds for far away zones (1st-row in the above example). An example of configuration for 3-row detection can be found at \config_file\vod_6843_ods_overhead_3row_optimized.cfg
.
EVM Mounting and Coordinate Transforms
The detected point cloud is all relative to the sensor. In order to make occupancy decision for each seat in the car, we need to transform the point cloud from sensor coordinates to the car coordinates. There are different position and different mounting angle that can be considered for mounting the device. To give the full flexibility, “sensorPosition” CLI is used to indicate the mounting offset in (x, y, z) and mounting rotation angle in y-z plane, x-y plane and x-z plane. Based on these offsets and tilting angles, the visualizer will transform the point cloud from the sensor coordinate to the car coordinate. The car coordinate is plotted in the figure below.
Since the point cloud will be transform to this car coordinates, the zone definition for each seat need to follow this car coordinates as well.
Next we give one example:
- when the sensor is mounted around the center of the roof facing downward, then sensor position should be programmed as: sensorPosition 0 1.2 1.1 90 0 0
This indicates the sensor is mounted at (x = 0, y = 1.2m, z = 1.1m) and rotated 90 degree clockwise in y-z domain to facing the floor. Please referred to the figure below to help understanding.
Users can find more mounting examples in the demo setup guide at
Antenna Configuration
The lab supports different antenna layouts. Users can follow the demo setup guide to learn how to support different antenna pattern by programming CLI commands antGeometry0, antGeometry1 and antPhaseRot.
In this package, we provide example configurations for all three TI xWR6843 EVM modules:
AWR6843AOP:
The AOP Antenna is shown in the following image along with the virtual antenna array. The provided configuration file \config_file\vod_6843_aop_overhead_2row.cfg provides the CLI commands necessary to support this antenna.
AWR6843ISK-ODS:
The ODS Antenna is shown in the following image along with the virtual antenna array. The provided configuration file config_file\vod_6843_ods_overhead_2row.cfg provides the CLI commands necessary to support this antenna.
AWR6843ISK:
The ISK Antenna is shown in the following image along with the virtual antenna array. The provided configuration file config_file\vod_6843_isk_frontMount_2row.cfg provides the CLI commands necessary to support this antenna. Note that ISK antenna has narrower field of view (FOV) in elevation direction, therefore the configuration example we provided above assumes front mounting.
Demo Specific Configuration File Parameters
In addition to the standard mmWave chirp configuration CLI commands (please refer to the SDK’s user guide for these commands), there are some additional commands that are specific to this demo:
Command | Parameters (in command line order) |
---|---|
dynamicRACfarCfg | subframe number. Only -1 is supported in this demo |
range bin start index (first range bin processed by CFAR) | |
range bin end skip (ending range bins not processed by CFAR) | |
angle start index (first angle bin processed by CFAR) | |
angle end skip (ending angle bins not processed by CFAR) | |
search window size - “far” range | |
search window size - angle | |
search window size - “near” range | |
guard window size - “far” range | |
guard window size - angle | |
guard window size - “near” range | |
“K0” cross range detection threshold applied to first search pass | |
“K0” cross angle detection threshold used in the second search pass | |
“sidelobe” threshold used in the second search pass | |
second search pass enable flag | |
RefRangeBinIdx: 15 is recommended. The CFAR threshold “K0” will be adjusted for shorter range bins based on K0*(refRangeBinIdx/rangeBinIdx)^2. | |
dynamicRangeAngleCfg | subframe number. Only -1 is supported in this demo |
angle search step (inter-bin resolution) 7 degree creates a azimuth-elevation heatmap of size 17x17 for each range bin. Smaller values create larger heatmaps, increasing memory and processing requirements. | |
MVDR “alpha” (diagonal loading factor) 0.03 is recommended. | |
range-az-elev detection method. Only 2 is supported in this demo. | |
Doppler estimation detection method. Only 0 is supported in this demo (Doppler is not calculated). | |
dynamic2DAngleCfg | subframe number. Only -1 is supported in this demo |
zoom-in Factor for finer angle estimation: 5 is recommended | |
Number of coarse neighboring angle bins of zoom-in. Only 1 is supported | |
Number of samples on each side to expand the peak of the zoomed-in azimuth-elevation heatmap, 1 is recommended. | |
Linear SNR threshold for the peak to enable peak expansion in the zoomed-in heatmap. | |
localMaxCheck: 0: No local maximum check. 1: If the coarse peak is not local maximum in the elevation domain, exclude it from the detection. 2: If the coarse peak is not local maximum in both elevation and azimuth domains, exclude it from the detection. | |
antGeometry0 | azimuth offset of all available virtual antennas (see device and SDK user guides) |
antGeometry1 | elevation offset of all available virtual antennas (see device and SDK user guides) |
antPhaseRot | phase rotation needed for all available virtual antennas (see device and SDK user guides) |
fovCfg | subframe number. Only -1 is supported in this demo |
azimuth field of view in degrees, 64 is recommended, which covers the fov from (-64, 64) in azimuth direction. | |
elevation field of view in degrees, 64 is recommended, which covers fov from (-64, 64) in elevation direction. | |
dopplerBinSelCfg | subframe number. Only -1 is supported in this demo |
Doppler Binning feature enable/disable | |
Doppler FFT size per frame | |
Starting Doppler bins, recommend set to 0 | |
Ending Doppler bins, recommend set to 4, the symmetric negative Doppler bins will be automatically selected as well. | |
sensorPosition | offset in x direction, in meter |
offset in y direction, in meter | |
offset in z direction, in meter | |
clockwise rotation angle in y-z plane, in degree | |
clockwise rotation angle in x-y plane, in degree | |
clockwise rotation angle in x-z plane, in degree | |
numZones | number of zones to define, min = 5, max = 8. The first 5 zones must correspond to the zone numbers shown in the Occupancy Display. If you only want 2 operational zones, define the others as NULL zones. (see cuboidDef below). |
totNumRows | number of rows in the car for occupancy detection. |
occStateMach | zone type: the state machine allow different thresholds for different zone now, use zone type to separate them. In our example, we use a different set of thresholds for middle seat. |
threshold 1: number of detected points in a zone to enter the occupied state | |
threshold 1: average SNR of detected points in a zone to enter the occupied state | |
threshold 2: number of detected points in a zone to enter the occupied state | |
threshold 2: average SNR of detected points in a zone to enter the occupied state | |
numEntryThreshold: every of continuous frames that the zone is passing the occupancy condition. | |
number of detected points in a zone to remain in the occupied state | |
average SNR of detected points in a zone to remain in the occupied state | |
number of frames with less than ceiling points (see next parameter) to hold occupied state before dropping | |
ceiling points (max points) in a zone during “hold” frames. | |
average SNR in a zone to declare overload condition (excessive movement). Causes state machine to freeze all zones in that row. | |
interiorBounds | min X (azimuth) -X towards passenger side, +X towards driver side (zone1) |
max X (azimuth) | |
min Y (depth) 0: brake pedal, +Y towards the rear | |
max Y (depth) | |
cuboidDef | parent zone number (1 based) |
cuboid number within the zone (also 1 based). min = 1, max = 3 per zone | |
min X (azimuth) | |
max X (azimuth) | |
min Y (depth) | |
max Y (depth) | |
min Z (height) 0.0 is the car floor | |
max Z (height) should not be higher than the car roof. | |
(Note: to define a NULL zone, define it with a single cuboidDef command setting all X,Y,Z values to 0.0) | |
zoneNeighDef | zone ID |
zone type: defined to allow different set of threshold in occStateMach. In our example, we use different zone type of middle seat than the rest of the seats. | |
numNeigh: number of neighbor. When adding a neighbor to a zone, the condition for the zone to enter the occupied state is enhanced. I.e., the condition 2 (small number of point cloud with larger average SNR), the average SNR of the zone also has to be larger than the average SNR of its defined neighbors. This is majorly added to reduce false detection in the middle seat. | |
neighbors: define the zone ID for each neighbor. | |
classParam | mode: set mode = 0 to disable classification, and set mode = 1 to enable baby/adult classification. |
numFrameAvg: number of frames to average for classification | |
snrTh_1, snrTh_2, …snrTh_numZones: Total SNR threshold per zone: 10*log10(sum of linear SNR over all points per frame, average over numFrameAvg) | |
volVarTh_1, volVarTh_2, …volVarTh_numZones: Volume variance threshold per zone: the variance of volume is calculated with all the points detected over numFrameAvg. |
Zone Definition
You may define five to eight zones in the chirp configuration file for target and GUI processing. The Occupancy Display in the GUI supports a fixed five zones, and so the first five defined zones must match these positions. Note: If you do not wish to use some of these zones, simply define these zones as “NULL” (explained below). For example, you may not want to monitor zones 1 and 2 (the front row). Set these to NULL.
Since our point cloud includes range, azimuth and elevation, we need a way to define our zones as volumes. We do this by defining a number of cuboids, which are simply rectangular volumes, having depth (y direction), width (x direction), and height (z direction). These cuboids will approximate the space where we want to look for occupants.
In this demo, we may define 1, 2, or 3 cuboids per zone. A detected point in the point cloud residing inside any of these cuboids will be included in the calculations for determining occupancy. If your zone is a cargo area, you may need to define only one cuboid to approximate the space. For normal vehicle seats, we need to check the foot well areas for children and pets. So we define three cuboids:
- Cuboid 1: Head and chest area
- Cuboid 2: Lap area
- Cuboid 3: Foot well
This is illustrated here:
Zones may overlap or be disjoint. The only check for zone inclusion is that a detected point is within the boundaries of at least one cuboid. It is therefore not a good idea to define Zone 1 and Zone 2 (for example) with cuboids that overlap, as this will reduce performance. Also, there is no performance difference (in terms of MIPs) if cuboids are large or small. The boundary checks are the same in either case. It is a good idea to leave a space in between zones, since detection points from a zone will spread out from the occupant’s actual position, more as the amount of occupant movement increases.
Cuboids are defined by the cuboidDef CLI command in a chirp configuration file. Here is a sample definition of five zones:
% zone 1 (driver) cuboids
1 1 0.15 0.70 0.6 1.2 0.85 1.1
cuboidDef 1 2 0.2 0.70 0.3 0.9 0.5 0.85
cuboidDef 1 3 0.2 0.70 0.0 0.7 0.0 0.5
cuboidDef % zone 2 (front passenger) cuboids
2 1 -0.70 -0.15 0.6 1.2 0.85 1.1
cuboidDef 2 2 -0.70 -0.2 0.3 0.9 0.5 0.85
cuboidDef 2 3 -0.70 -0.2 0.0 0.7 0.0 0.5
cuboidDef % zone 3 (2nd row driver side) cuboids
3 1 0.30 0.70 1.5 2.2 0.80 1.1
cuboidDef 3 2 0.30 0.70 1.2 2.0 0.3 0.80
cuboidDef 3 3 0.30 0.70 0.9 1.4 0.0 0.3
cuboidDef % zone 4 (2nd row middle) cuboids
4 1 -0.10 0.10 1.5 2.2 0.80 1.1
cuboidDef 4 2 -0.10 0.10 1.2 2.0 0.3 0.80
cuboidDef 4 3 -0.10 0.10 0.9 1.4 0.0 0.3
cuboidDef % zone 5 (2nd row passenger side) cuboids
5 1 -0.70 -0.30 1.5 2.2 0.80 1.1
cuboidDef 5 2 -0.70 -0.30 1.2 2.0 0.3 0.80
cuboidDef 5 3 -0.70 -0.30 0.9 1.4 0.0 0.3 cuboidDef
Example Zone/Cuboid CLI Definitions
The first pair of arguments define the parent zone, and the cuboid number within the zone, the second pair is the minimum and maximum X (width), the third pair is the min/max Y (depth), and finally the min/max Z (height). Notice that the driver side X values are positive while the passenger side are negative. All Z values are positive since the floor is 0.0meters.
To define a NULL zone, simply define a zone with a single cuboid, and set all parameters to zero. This makes it impossible for the zone assignment function to map any points to the zone, and the zone will stay silent.
% zone 4 (NULL zone)
4 1 0.0 0.0 0.0 0.0 0.0 0.0 cuboidDef
Example NULL Zone Definition
To help with cuboid placement, the “3D Zones Row 1[2]” displays may be enabled as shown in the image below. This display will show the real-time point cloud in 3D space as they relate to the cuboid positions.
Tuning
The key items to focus on for tuning are as follows:
- Proper mounting - Ensure that all FOV and resolution constaints from the device are accounted for.
- Matching Antenna Pattern - Make sure antGeometry0, antGeometry1 and antPhaseRot settings in the configuration file matches with the antenna pattern on the device/EVM.
- Point Cloud Tuning - Users can adjust CFAR detection threshold “K0 cross range” and “K0 cross angle” in dynamicRACfarCfg to trade off between number of detected points vs false detection ratio. Lower “K0” results in richer point cloud but higher false detection ratio. Users can also lower RefRangeBinIdx in dynamicRACfarCfg to get richer point cloud in closer range bins. Users can also lower “Linear SNR threshold” in dynamic2DAngleCfg to lower the point cloud extension threshold to include more neighboring points next to the detected peak.
- Accurate zone definition - Ensuring the zones align with the physical space of the vehicle. This is described in further detail in the section zone definition.
- occStateMach - All the parameters in the occStateMach affect how the occupancy state machine moves between states. They may need to be adjusted depending on the number of points and their SNR that is observed in the specific environment. This parameter is further defined in the parameter list.
- Define different zones - Customer can easily modify or add zones to extend the solution to different applications. Examples provided in the package are listed below:
- An example configuration to cover intruder detection: config_file\vod_6843_aop_overhead_2row_intruder.cfg.
- An example configuration to cover 3 row total 6-seat detection on a bus: config_file\vod_6843_aop_overhead_3row_bus.cfg.
- An example configuration to cover 7-seat detection on a 3-row mini-van/SUV: config_file\vod_6843_ods_overhead_3row_optimized.cfg.
Developer’s Guide
1. Import Lab Project
To import the source code into your CCS workspace, a CCS project is provided in the lab at the path given below.
Start CCS and configure your workspace as desired.
Import the projects specified in the following folder to CCS.
<PACKAGE_LOCATION>\examples\incabin_sensing\3D_Occupany_Detection\src\68xx\
Verify that the import occurred without error by checking that
occupancy_detection_3d_68xx_mss
andoccupancy_detection_3d_68xx_dss
show up in the project explorer
⚠️ Error during Import to IDE
If an error occurs, check that the software dependencies listed above have been installed. Errors will occur if necessary files are not installed in the correct location for importing.
2. Build the Lab
Select the occupancy_detection_3d_68xx_dss project in the Project Explorer so that it is highlighted. Right click on the project and select Rebuild Project. The project will then build.
On successful build, the following should appear in
<PROJECT_WORKSPACE_DIR>\occupancy_detection_3d_68xx_dss\default
- occupancy_detection_3d_68xx_dss.xe674 (this is the C674 DSP binary used for CCS debug mode)
Select the occupancy_detection_3d_68xx_mss project in the Project Explorer so that it is highlighted. Right click on the project and select Rebuild Project. The project will then build.
On successful build, the following should appear in
<PROJECT_WORKSPACE_DIR>\occupancy_detection_3d_68xx_mss\default
- occupancy_detection_3d_68xx_mss.xer4f (this is the Cortex R4F binary used for CCS debug mode)
- occupancy_detection_3d_68xx.bin (this is the flashable binary used for deployment mode)
⚠️ Building vs Rebuiling
Selecting Rebuild instead of Build ensures that the project is always re-compiled. This is especially important in case the previous build failed with errors.
⚠️ Build Fails with Errors
If the build fails with errors, please ensure that all the software requirements are installed as listed above and in the mmWave SDK release notes.
Note
As mentioned in the Quickstart section, pre-built binary files are provided in the pre-compiled directory of the lab.
3. Execute the Lab
There are two ways to execute the compiled code on the EVM:
- Deployment mode: the EVM boots from flash and starts running the bin image at power-up
- Using UniFlash, flash the occupancy_detection_3d_68xx.bin found at
<PROJECT_WORKSPACE_DIR>\occupancy_detection_3d_68xx_mss\default
- The same procedure for flashing can be used as detailed in the Flash the EVM section.
- Using UniFlash, flash the occupancy_detection_3d_68xx.bin found at
- Debug mode: This mode is used for downloading and running the executable from CCS. This mode enables JTAG connection with CCS while the lab is running and is useful during development and debugging.
- Follow the CCS Debug Mode Guide
After executing the lab using either method, launch the visualizer and proceed as described under Quickstart
Known Issues and Limitations
- When the number of range bins used for Dynamic CFAR is less than 29, false detections begin to occur.
- To avoid this, keep the CFAR leftSkipSize and rightSkipSize small enough to ensure that at least 29 range bins are used.
- The angle dependent CFAR ratio is hardcoded in the code, assuming a size of 17x17 in coarse angle spectrum during range-azimuth-elevation heatmap generation.
- Please keep fovCfg and dynamicRangeAngleCfg as default setting.
Need More Help?
- Search for your issue or post a new question on the mmWave E2E forums
- See the SDK for more documentation on various algorithms used in this demo. Start at
<MMWAVE_SDK3_INSTALL_DIR>\docs\mmwave_sdk_module_documentation.html