3.9.3. Use Camera#

The Camera evaluation section briefly shows how to capture video using either a USB camera or a CSI-2 image sensor. This section will go further to explain how to configure the camera/sensor to capture videos at a specific format, resolution or frame rate.

The following Linux user space applications/utilities are commonly used to access and configure the camera/sensor:

  • v4l2-ctl

  • media-ctl

3.9.3.1. Query Camera Capabilities#

Before configuring the camera/sensor, it is necessary to find out the supported formats, resolutions and frame rates.

3.9.3.1.1. Query a USB Camera#

The v4l2-ctl utility can be used to query a USB camera. Below commands show an example of querying a USB camera.

  1. Find the video device number of the camera:

    # v4l2-ctl --list-devices
    Video Capture 4 (usb-xhci-hcd.1.auto-1.2):
            /dev/video1
            /dev/video2
            /dev/media1
    
  2. Display the supported formats of the camera (using video1 listed above):

    # v4l2-ctl --list-formats -d /dev/video1
    ioctl: VIDIOC_ENUM_FMT
            Type: Video Capture
    
            [0]: 'YUYV' (YUYV 4:2:2)
            [1]: 'MJPG' (Motion-JPEG, compressed)
    
  3. Display the supported resolutions (frame sizes) and frame rates for all formats:

    # v4l2-ctl --list-formats-ext -d /dev/video1
    ioctl: VIDIOC_ENUM_FMT
            Type: Video Capture
    
            [0]: 'YUYV' (YUYV 4:2:2)
                    Size: Discrete 640x480
                            Interval: Discrete 0.033s (30.000 fps)
                            Interval: Discrete 0.042s (24.000 fps)
            ...
    
            [1]: 'MJPG' (Motion-JPEG, compressed)
                    Size: Discrete 640x480
                            Interval: Discrete 0.033s (30.000 fps)
                            Interval: Discrete 0.042s (24.000 fps)
                            Interval: Discrete 0.050s (20.000 fps)
    

3.9.3.1.2. Query a CSI-2 Image Sensor#

For CSI-2 image sensors, instead of querying the video device, we need to query the sensor’s device node. The reason is that the CSIRX subsystem (video device) has no limitations on frame sizes. It is the image sensor (device node) that has limitations. A few steps are needed to find out the supported resolutions and frame rates, with examples give below.

  1. Find the media device number corresponding to the image sensor (note media0 below):

    # v4l2-ctl --list-devices
    j721e-csi2rx (platform:30102000.ticsi2rx):
            /dev/video0
            /dev/video1
            /dev/video2
            /dev/video3
            /dev/media0
    
  2. Find the device node name corresponding to the image sensor (using ov5640 as an example) (note /dev/v4l-subdev2 and pad0 below):

    #  media-ctl -d /dev/media0 -p
    Media controller API version 5.10.109
    ...
    - entity 13: ov5640 4-003c (1 pad, 1 link)
                type V4L2 subdev subtype Sensor flags 0
                device node name /dev/v4l-subdev2
            pad0: Source
                    [fmt:UYVY8_2X8/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                    -> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]
    ...
    
  3. List the media bus codes and find the one for the desired format (note 0x2006 for UYVY format below):

    # v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-mbus-codes pad=0
    ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=0)
            0x4001: MEDIA_BUS_FMT_JPEG_1X8
            0x2006: MEDIA_BUS_FMT_UYVY8_2X8
    ...
    
  4. Display the supported resolutions for the desired format (using the pad number and the media bus code obtained earlier):

    # v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-framesizes pad=0,code=0x2006
    ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0)
            Size Range: 176x144 - 176x144
            Size Range: 320x240 - 320x240
            Size Range: 640x480 - 640x480
            Size Range: 720x480 - 720x480
            Size Range: 720x576 - 720x576
            Size Range: 1024x768 - 1024x768
            Size Range: 1280x720 - 1280x720
            Size Range: 1920x1080 - 1920x1080
            Size Range: 2592x1944 - 2592x1944
    
  5. Display the supported frame rates for a specific format and frame size:

    # v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-frameintervals pad=0,width=640,height=480,code=0x2006
    ioctl: VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (pad=0)
            Interval: 0.067s (15.000 fps)
            Interval: 0.033s (30.000 fps)
            Interval: 0.017s (60.000 fps)
    

3.9.3.2. Configure Camera Parameters#

After the capabilities of a camera/sensor are known, we can configure the camera/sensor to capture with specific parameters such as format, resolution or frame rate.

For a USB camera, the parameters can be specified in the gst-launch command. For example, the following command captures video in raw format, with frame size 1024x576, at frame rate 15 fps:

# gst-launch-1.0 v4l2src device="/dev/video1" ! video/x-raw, width=1024, height=576, framerate=15/1 ! waylandsink

Another command shown below captures video in MPEG format, with frame size 1920x1080, at frame rate 30 fps:

# gst-launch-1.0 v4l2src device="/dev/video1" ! image/jpeg, width=1920, height=1080, framerate=30/1 ! jpegparse ! jpegdec ! waylandsink

For a CSI-2 sensor, the capturing parameters have to be configured using media-ctl before capturing. For example, the following commands set the OV5640 image sensor to UYVY format, 1024x768 frame size, 30 fps:

# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-frameintervals pad=0,width=1024,height=768,code=0x2006
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (pad=0)
    Interval: 0.067s (15.000 fps)
    Interval: 0.033s (30.000 fps)
# media-ctl --set-v4l2 "'ov5640 4-003c':0 [fmt:UYVY8_2X8/1024x768@1/30 field:none]"

Refer to the supported formats and the corresponding codes displayed by command v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-mbus-codes.

Use media-ctl -p to verify the format, frame size and frame rate are configured properly:

# media-ctl -d /dev/media0 -p
Media controller API version 5.10.109
...
- entity 13: ov5640 4-003c (1 pad, 1 link)
        type V4L2 subdev subtype Sensor flags 0
        device node name /dev/v4l-subdev2
    pad0: Source
            [fmt:UYVY8_2X8/1024x768@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
            -> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]

Then run Gstreamer to capture video with the configured parameters:

# gst-launch-1.0 v4l2src device="/dev/video0" ! video/x-raw , width=1024, height=768, framerate=30/1 ! waylandsink