3.11.3. Use Camera¶

The Camera Evaluation section briefly shows how to capture video using either a USB camera or a CSI-2 image sensor. This section will go further to explain how to configure the camera/sensor to capture videos at a specific format, resolution or frame rate. It will also be explained in this section how to enable a new CSI-2 sensor other than the default supported sensor in the SDK.

3.11.3.1. Query Camera Capabilities¶

Before configuring the camera/sensor, it is necessary to find out the supported formats, resolutions and frame rates.

3.11.3.1.1. Query a USB Camera¶

The v4l2-ctl utility can be used to query a USB camera. Below commands show an example of querying a USB camera.

Find the video device number of the camera:

# v4l2-ctl --list-devices
Video Capture 4 (usb-xhci-hcd.1.auto-1.2):
        /dev/video1
        /dev/video2
        /dev/media1

Display the supported formats of the camera (using video1 listed above):

# v4l2-ctl --list-formats -d /dev/video1
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'YUYV' (YUYV 4:2:2)
        [1]: 'MJPG' (Motion-JPEG, compressed)

Display the supported resolutions (frame sizes) and frame rates for all formats:

# v4l2-ctl --list-formats-ext -d /dev/video1
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'YUYV' (YUYV 4:2:2)
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
                        Interval: Discrete 0.042s (24.000 fps)
        ...

        [1]: 'MJPG' (Motion-JPEG, compressed)
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
                        Interval: Discrete 0.042s (24.000 fps)
                        Interval: Discrete 0.050s (20.000 fps)

3.11.3.1.2. Query a CSI-2 Image Sensor¶

For CSI-2 image sensors, instead of querying the video device, we need to query the sensor’s device node. The reason is that the CSIRX subsystem (video device) has no limitations on frame sizes. It is the image sensor (device node) that has limitations. The media-ctl utility will be used with the v4l2-ctl utility to query the sensor. A few steps are needed to find out the supported resolutions and frame rates, with examples given below.

Find the media device number corresponding to the image sensor (note media0 below):

# v4l2-ctl --list-devices
j721e-csi2rx (platform:30102000.ticsi2rx):
        ...
        /dev/media0

Find the device node name corresponding to the image sensor (using OV5640 as an example) (note /dev/v4l-subdev2 and pad0 below):

#  media-ctl -d /dev/media0 -p
Media controller API version 5.10.109
...
- entity 13: ov5640 4-003c (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev2
        pad0: Source
                [fmt:UYVY8_2X8/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                -> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]
...

List the media bus codes and find the one for the desired format:

# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-mbus-codes pad=0
ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=0)
        0x4001: MEDIA_BUS_FMT_JPEG_1X8
        0x2006: MEDIA_BUS_FMT_UYVY8_2X8
...

Display the supported resolutions for the desired format (using the pad number and the media bus code obtained earlier):

# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-framesizes pad=0,code=0x2006
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0)
        Size Range: 176x144 - 176x144
        Size Range: 320x240 - 320x240
        Size Range: 640x480 - 640x480
        Size Range: 720x480 - 720x480
        Size Range: 720x576 - 720x576
        Size Range: 1024x768 - 1024x768
        Size Range: 1280x720 - 1280x720
        Size Range: 1920x1080 - 1920x1080
        Size Range: 2592x1944 - 2592x1944

Display the supported frame rates for a specific format and frame size:

# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-frameintervals pad=0,width=640,height=480,code=0x2006
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (pad=0)
        Interval: 0.067s (15.000 fps)
        Interval: 0.033s (30.000 fps)
        Interval: 0.017s (60.000 fps)

3.11.3.2. Configure Camera Parameters¶

After the capabilities of a camera/sensor are known, we can configure the camera/sensor to capture with specific parameters such as format, resolution or frame rate.

For a USB camera, the parameters can be specified in the gst-launch command. For example, the following command captures video in raw format, with frame size 1024x576, at frame rate 15 fps:

# gst-launch-1.0 v4l2src device="/dev/video1" ! video/x-raw, width=1024, height=576, framerate=15/1 ! kmssink driver-name=tidss

Another command shown below captures video in MPEG format, with frame size 1920x1080, at frame rate 30 fps:

# gst-launch-1.0 v4l2src device="/dev/video1" ! image/jpeg, width=1920, height=1080, framerate=30/1 ! jpegparse ! jpegdec ! kmssink driver-name=tidss

For a CSI-2 sensor, the capturing parameters have to be configured using media-ctl before capturing.

For example, the following commands set the OV5640 image sensor to UYVY format, 1024x768 frame size, 30 fps:

# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-frameintervals pad=0,width=1024,height=768,code=0x2006
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (pad=0)
    Interval: 0.067s (15.000 fps)
    Interval: 0.033s (30.000 fps)
# media-ctl --set-v4l2 "'ov5640 4-003c':0 [fmt:UYVY8_2X8/1024x768@1/30 field:none]"

Refer to the supported formats and the corresponding codes displayed by command v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-mbus-codes.

Use media-ctl -p to verify the format, frame size and frame rate are configured properly:

# media-ctl -d /dev/media0 -p
Media controller API version 5.10.109
...
- entity 13: ov5640 4-003c (1 pad, 1 link)
        type V4L2 subdev subtype Sensor flags 0
        device node name /dev/v4l-subdev2
    pad0: Source
            [fmt:UYVY8_2X8/1024x768@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
            -> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]

Then run GStreamer to capture video with the configured parameters:

# gst-launch-1.0 v4l2src device="/dev/video0" ! video/x-raw , width=1024, height=768, framerate=30/1 ! kmssink driver-name=tidss

3.11.3.3. Enable A New CSI-2 Sensor¶

The Processor SDK supports one or more camera sensors by default. Both the driver modules and the device tree overlays for the supported sensors are pre-built and included in the SDK. To enable a new CSI-2 sensor, the following listed steps are needed to build the driver and device tree on the host and copy to the target. The OV9281 sensor is used as an example in the given steps. Some steps can be skipped if the driver is already included in the SDK.

Add the sensor driver source code to drivers/media/i2c (path is relative to <sdk root>/board-support/linux-<version>):
drivers/media/i2c/ov9281.c

Add the sensor in drivers/media/i2c/Kconfig. For example:

config VIDEO_OV9281
        tristate "OmniVision OV9281 sensor support"
        depends on GPIOLIB && VIDEO_V4L2 && I2C
        select MEDIA_CONTROLLER
        select VIDEO_V4L2_SUBDEV_API
        select V4L2_FWNODE
        help
          This is a Video4Linux2 sensor driver for the OmniVision
          OV9281 camera sensor.

Add the sensor driver to drivers/media/i2c/Makefile:
obj-$(CONFIG_VIDEO_OV9281) += ov9281.o
Enable the sensor in ti_config_fragments/v8_audio_display.cfg:
CONFIG_VIDEO_OV9281=m
Create a device tree overlay for the sensor and add to arch/arm64/boot/dts/ti:
arch/arm64/boot/dts/ti/k3-am625-sk-csi2-ov9281.dts
Add the device tree overlay to arch/arm64/boot/dts/ti/Makefile:
dtb-$(CONFIG_ARCH_K3) += k3-am625-sk-csi2-ov9281.dtbo

Generate the defconfig and rebuild kernel image, modules and device tree, according to instructions given in section Build Kernel:

$ ti_config_fragments/defconfig_builder.sh -t ti_sdk_arm64_release
$ make ti_sdk_arm64_release_defconfig
$ make Image modules dtbs

Install the newly built kernel image, modules and device tree overlay binary:

$ sudo cp arch/arm64/boot/Image <path to root of file system>/boot
$ sudo cp arch/arm64/boot/dts/ti/k3-am625-sk-csi2-ov9281.dtbo <path to root of file system>/boot
$ sudo make INSTALL_MOD_PATH=<path to root of file system> modules_install

Boot the EVM with the updated kernel and root filesystem. Apply the device tree overlay at U-Boot prompt:
```
=>setenv name_overlays k3-am625-sk-csi2-ov9281.dtbo
=>saveenv
=>boot
```

Verify the sensor is added using “v4l2-ctl” and “media-ctl” utilities as described earlier.

#media-ctl -d /dev/media0 -p
...
- entity 13: ov9281 4-0060 (1 pad, 1 link)
      type V4L2 subdev subtype Sensor flags 0
      device node name /dev/v4l-subdev2
 pad0: Source
         [fmt:SGRBG8_1X8/1280x800 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range
          crop.bounds:(0,0)/1280x800
          crop:(0,0)/1280x800]
         -> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]
...

	Linux Academy for AM62X