Use Camera#
This module will help you get started on camera development.
The Camera Evaluation section briefly shows how to capture video using either a USB camera or a CSI-2 camera. This section will go further to explain how to configure the camera to capture videos at a specific format, resolution or frame rate. It will also be explained in this section how to enable a new CSI-2 sensor other than the default supported sensors in the SDK.
Query Camera Capabilities#
Before configuring the camera, it is necessary to find out the supported formats, resolutions and frame rates.
Query a USB Camera#
The v4l2-ctl utility can be used to query a USB camera. Below commands show an
example of querying a USB camera.
Find the video device number of the camera:
# v4l2-ctl --list-devices Video Capture 4 (usb-xhci-hcd.1.auto-1.2): /dev/video1 /dev/video2 /dev/media1Display the supported formats of the camera (using video1 listed above):
# v4l2-ctl --list-formats -d /dev/video1 ioctl: VIDIOC_ENUM_FMT Type: Video Capture [0]: 'YUYV' (YUYV 4:2:2) [1]: 'MJPG' (Motion-JPEG, compressed)Display the supported resolutions (frame sizes) and frame rates for all formats:
# v4l2-ctl --list-formats-ext -d /dev/video1 ioctl: VIDIOC_ENUM_FMT Type: Video Capture [0]: 'YUYV' (YUYV 4:2:2) Size: Discrete 640x480 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.042s (24.000 fps) ... [1]: 'MJPG' (Motion-JPEG, compressed) Size: Discrete 640x480 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.042s (24.000 fps) Interval: Discrete 0.050s (20.000 fps)
Query a CSI-2 Camera#
For CSI-2 cameras, instead of querying the video device, we need to query the camera
sensor’s device node. The reason is that the CSIRX subsystem (video device) has no
limitations on frame sizes. It is the image sensor (device node) that has limitations.
The media-ctl utility will be used with the v4l2-ctl utility to query the sensor.
A few steps are needed to find out the supported resolutions and frame rates, with
examples given below.
Find the media device number corresponding to the image sensor (note media0 below):
# v4l2-ctl --list-devices j721e-csi2rx (platform:30102000.ticsi2rx): ... /dev/media0Find the device node name corresponding to the image sensor (using IMX219 as an example) (note /dev/v4l-subdev2 and pad0 below):
# media-ctl -d /dev/media0 -p Media controller API version 5.10.168 ... - entity 15: imx219 4-0010 (1 pad, 1 link, 0 route) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev2 pad0: Source [stream:0 fmt:SRGGB8_1X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080] -> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE] ...List the media bus codes and find the one for the desired format:
# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-mbus-codes pad=0 ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=0,stream=0) 0x300f: MEDIA_BUS_FMT_SRGGB10_1X10 0x3014: MEDIA_BUS_FMT_SRGGB8_1X8Display the supported resolutions for the desired format (using the pad number and the media bus code obtained earlier):
# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-framesizes pad=0,code=0x300f ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0,stream=0) Size Range: 3280x2464 - 3280x2464 Size Range: 1920x1080 - 1920x1080 Size Range: 1640x1232 - 1640x1232 Size Range: 640x480 - 640x480
Configure Camera Parameters#
After the capabilities of a camera/sensor are known, we can configure the camera/sensor to capture with specific parameters such as format, resolution or frame rate.
For a USB camera, the parameters can be specified in the gst-launch command.
For example, the following command captures video in raw format, with
frame size 1024x576, at frame rate 15 fps:
# gst-launch-1.0 v4l2src device="/dev/video1" ! video/x-raw, width=1024, height=576, framerate=15/1 ! kmssink driver-name=tidss
Another command shown below captures video in MPEG format, with frame size 1920x1080, at frame rate 30 fps:
# gst-launch-1.0 v4l2src device="/dev/video1" ! image/jpeg, width=1920, height=1080, framerate=30/1 ! jpegparse ! jpegdec ! kmssink driver-name=tidss
For a CSI-2 sensor, the capturing parameters have to be configured using media-ctl before capturing.
For example, the following commands set the IMX219 image sensor and other entities in the media pipe to 10-bit RGGB raw format, 1920x1080 frame size:
# media-ctl -V '"imx219 4-0010":0 [fmt:SRGGB10_1X10/1920x1080 field:none]' # media-ctl -V '"imx219 4-0010":0 [fmt:SRGGB10_1X10/1920x1080 field:none]' # media-ctl -V '"cdns_csi2rx.30101000.csi-bridge":0 [fmt:SRGGB10_1X10/1920x1080 field:none]' # media-ctl -V '"30102000.ticsi2rx":0 [fmt:SRGGB10_1X10/1920x1080 field:none]'
Refer to the supported formats and the corresponding codes displayed by command v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-mbus-codes.
Use media-ctl -p to verify the format and frame size are configured properly:
# media-ctl -d /dev/media0 -p ... - entity 15: imx219 4-0010 (1 pad, 1 link, 0 route) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev2 pad0: Source [stream:0 fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080] -> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]
Then run GStreamer to capture video with the configured parameters:
# gst-launch-1.0 -v v4l2src device=/dev/video3 io-mode=dmabuf-import ! \ video/x-bayer, width=1920, height=1080, framerate=30/1, format=rggb10 ! \ tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_SONY_IMX219_RPI" \ dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b.bin format-msb=9 ! \ video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! kmssink driver-name=tidss
Enable A New CSI-2 Sensor#
The Processor SDK supports a few sensors out-of-box. Both the driver modules and the device tree overlays for the supported sensors are pre-built and included in the SDK. To enable a new CSI-2 sensor, the following listed steps are needed. Some steps can be skipped if the driver is already included in the SDK.
Add the sensor driver source code to drivers/media/i2c/<sensor_name>.c (path is relative to <sdk root>/board-support/ti-linux-kernel).
Add the sensor configuration in drivers/media/i2c/Kconfig. For example:
config VIDEO_<SENSOR_NAME> tristate "<sensor_name> support" depends on VIDEO_DEV && I2C depends on ACPI || COMPILE_TEST select MEDIA_CONTROLLER select VIDEO_V4L2_SUBDEV_API select V4L2_FWNODE help This is a Video4Linux2 sensor driver for the <sensor_name>Enable the compilation of the sensor driver in drivers/media/i2c/Makefile:
obj-$(CONFIG_VIDEO_<SENSOR_NAME>) += <sensor_name>.o
Enable the kernel module for the sensor in arch/arm64/configs/defconfig:
CONFIG_VIDEO_<SENSOR_NAME>=m
Create a device tree overlay for the sensor and add it to arch/arm64/boot/dts/ti:
arch/arm64/boot/dts/ti/<device_tree_overlay_name>.dtso
Add the device tree overlay to arch/arm64/boot/dts/ti/Makefile:
dtb-$(CONFIG_ARCH_K3) += <device_tree_overlay_name>.dtso
Rebuild and install Linux kernel according to the instructions given in section Build Kernel.
Boot the EVM with the updated kernel. Apply the device tree overlay according to instructions given in the Camera Evaluation section.
Verify the sensor is probed according to Camera Evaluation.
For sensors without in-built ISP, the integration of the ISP on the SoC (such as AM62A) into the image processing pipeline is also needed. Please refer to the AM6xA ISP Tuning Application Note for more details.