Camera

1 Camera Introduction

In modern society, with the improvement of chip computing power and AI empowerment, the camera — as a sensor that has almost become indispensable for mid-to-high-end embedded devices — is undergoing continuous innovation and performance optimization.

The following is a structural diagram of a CMOS sensor camera, roughly divided into the lens, the image sensor, and the image processing unit. Below we will explain based on the author's understanding.

1.1 Lens Introduction

Let's first introduce the lens. As shown above, it is composed of multiple lens elements. Theoretically, only one lens is needed to form an image, but there are many problems such as chromatic aberration and astigmatism. Combining multiple lenses can effectively correct the aberration. In addition, by changing the positions of multiple lenses, zoom can be achieved. Of course, most importantly, the autofocus of most cameras is also implemented by changing the position of the lens group.

Cameras that support zoom and autofocus generally have a micro motor to change the lens position. The common camera motor type is VCM (Voice Coil Motor). The structural diagram is as follows:

When the coil is energized, it generates a magnetic field. By adjusting the magnitude of the current flowing through the magnetic field, the lens can be moved back and forth. This structure is called VCM (Voice Coil Motor).

1.2 CMOS Sensor Introduction

Next, a brief talk about the most important part of the camera — its image sensor, which is commonly known as CMOS. The role of CMOS is to absorb photoelectrons so that each pixel can accumulate a different amount of charge. The collected charge is converted by the amplifier below the pixel into a voltage signal of corresponding intensity. As an analog quantity, the voltage signal is converted to a digital signal by an ADC. Taking a 12-bit ADC as an example, each pixel is divided into 4096 levels according to brightness. At this point, the raw file of a black-and-white picture can be obtained.

By adding a color filter in front of each pixel — for example, a green filter only allows green light to pass through — the light intensity value at that point is the level value of the green light intensity. The entire digital grid described by the brightness levels of the three primary colors RGB is the camera's raw file.

At this point, friends will understand why, when we take a photo with a phone at night, we have to wait a few seconds after pressing the shutter to complete the shot, while during the day it is very fast — because the CMOS unit needs to collect more photoelectrons, otherwise the result would be a completely black picture. Friends with photography background also understand the origin of the three elements of photography (aperture, shutter, ISO) — they all control the amount of light entering through different means!

1.3 Image Processing Unit Introduction

The image processing unit (ISP) processes the raw file. Based on the value of each pixel, it estimates the true color of each pixel through an algorithm (for example, if it finds that the red pixel (R) value on the left is 50 and the red pixel value on the right is 55, it will reasonably infer: "Hmm, the red value at this position is about 52"). After algorithmic processing, a color picture is born. If you need to output an image in a format such as JPG, the processor performs white balance correction, sharpening, denoising, and compression on the raw file to finally generate the image we need.

2 Location of the Camera Interface on the Board

The board reserves one MIPI camera interface, which is currently adapted for two cameras: GC8034 from Galaxycore Microelectronics and OV5695 from OmniVision Technologies.

Specific parameters can be viewed in the following table:

Parameter	GC8034	OV5695
Manufacturer	China Galaxycore (Galaxycore)	USA OmniVision (OmniVision)
Resolution	3264H x 2448V (about 8MP)	2592H x 1944V (about 5MP)
Optical size	1/4 inch	1/4 inch
Output format	RAW10 / RAW8	10-bit RGB RAW
Interface	MIPI (2 lanes or 4 lanes supported)	MIPI (1 lane or 2 lanes supported)
Frame rate	30 fps at full size (MIPI 4 lanes)	30 fps at full size (5MP)

3 Camera Testing

3.1 Device Tree Detailed Explanation

Tips

The file path below: out/kernel/src_tmp/linux-5.10/arch/arm64/boot/dts/rockchip

requires the kernel source to be compiled first.

Because the device tree configuration of the camera is relatively complex, according to the SDK's camera configuration, the data transmission process is roughly divided into the following stages:

Camera data transmission flow:

Step	Component/Stage	Function Description	Data Flow
1	Camera module	Image sensor captures light signals	Light signal → Electrical signal
2	I2C control configuration	Configure camera parameters	Control signal transmission
3	Image data acquisition	Convert electrical to digital signal	Analog signal → Digital signal
4	MIPI CSI-2/DVP interface transfer	High-speed data interface transfer	Digital image data transmission
5	CSI2 D-PHY physical layer processing	Physical layer signal processing and synchronization	Signal conditioning and clock recovery
6	ISP image signal processing	Image quality optimization processing	Raw data → Processed data
7	Denoising, color correction, etc.	Image enhancement and correction	Image quality improvement
8	Data output	Processed image data	Final image data
9	Store to memory	Data caching and storage	Memory write
10	Display	Image display output	Screen display

Because the device tree configuration of the camera is relatively complex, it will not be expanded in detail here.

In the i2c4 node of the board-level matching file rk3568-toybrick-x10.dtsi, the gc8034 and ov5695 are excerpted as examples:

&i2c4 {
    status = "okay";                              // 启用 I2C4 总线

    gc8034: gc8034@37 {                           // GC8034 摄像头模块，I2C 地址 0x37
        compatible = "galaxycore,gc8034";         // 兼容性字符串，用于驱动匹配
        reg = <0x37>;                             // I2C 设备地址
        clocks = <&cru CLK_CIF_OUT>;              // 时钟源：CRU 的 CIF 输出时钟
        clock-names = "xvclk";                    // 时钟名称：外部时钟
        power-domains = <&power RK3568_PD_VI>;    // 电源域：视频输入电源域
        pinctrl-names = "default";                // 引脚控制状态名称
        pinctrl-0 = <&cif_clk>;                   // 默认引脚配置：CIF 时钟引脚
        reset-gpios = <&gpio0 RK_PD6 GPIO_ACTIVE_LOW>;    // 复位 GPIO，低电平有效
        pwdn-gpios = <&gpio4 RK_PB4 GPIO_ACTIVE_LOW>;     // 电源控制 GPIO，低电平有效
        rockchip,grf = <&grf>;                    // 通用寄存器文件引用
        rockchip,camera-module-index = <0>;       // 摄像头模块索引号
        rockchip,camera-module-facing = "back";   // 摄像头朝向：后置
        rockchip,camera-module-name = "RK-CMK-8M-2-v1";   // 摄像头模块名称
        rockchip,camera-module-lens-name = "CK8401";      // 镜头名称
        port {                                    // 端口定义，用于连接其他设备
            gc8034_out: endpoint {                // 输出端点
                remote-endpoint = <&mipi_in_ucam1>;       // 连接至 MIPI CSI 的 ucam1 输入
                data-lanes = <1 2 3 4>;           // 使用 4 条数据线
            };
        };
    };
......

    ov5695: ov5695@36 {                           // OV5695 摄像头模块，I2C 地址 0x36
        status = "okay";                          // 启用该设备
        compatible = "ovti,ov5695";               // 兼容性字符串，OmniVision 驱动
        reg = <0x36>;                             // I2C 设备地址
        clocks = <&cru CLK_CIF_OUT>;              // 时钟源：CRU 的 CIF 输出时钟
        clock-names = "xvclk";                    // 外部时钟名称
        power-domains = <&power RK3568_PD_VI>;    // 电源域：视频输入电源域
        pinctrl-names = "default";                // 引脚控制状态名称
        pinctrl-0 = <&cif_clk>;                   // 默认引脚配置：CIF 时钟引脚
        reset-gpios = <&gpio0 RK_PD6 GPIO_ACTIVE_LOW>;    // 复位控制 GPIO，低电平有效
        pwdn-gpios = <&gpio4 RK_PB4 GPIO_ACTIVE_LOW>;     // 电源控制 GPIO，低电平有效
        rockchip,camera-module-index = <0>;       // 摄像头模块索引号
        rockchip,camera-module-facing = "back";   // 摄像头朝向：后置
        rockchip,camera-module-name = "TongJu";   // 模块厂商名称
        rockchip,camera-module-lens-name = "CHT842-MD";   // 镜头型号
        port {                                    // 端口配置
            ov5695_out: endpoint {                // 输出端点
                remote-endpoint = <&mipi_in_ucam2>;       // 连接至 MIPI CSI 的 ucam2 输入
                data-lanes = <1 2>;               // 使用 2 条数据线
            };
        };
    };
};

3.2 Camera Effect File Description

For cameras that output raw images, the image effect needs to be adjusted by the ISP to meet normal requirements. Currently the M4R1 effect file is adapted for GC8034 by default. If you need to adapt other raw cameras, you also need to modify the effect file on the board accordingly!

Below we use the adaptation of ov5695 as an example.

After installing the HDC tool, enter in the terminal:

# 赋予修改系统文件夹的权限
hdc shell mount -o remount,rw /vendor
# 推送效果文件
hdc file send "C:\Users\Administrator\Documents\ov5695_TongJu_CHT842-MD.json" /vendor/etc/iqfil.json
# 重启板子
hdc shell reboot

After rebooting the board, the ISP module on the board will read this JSON file so that the ov5695 outputs images normally.

Note

The ov5695 effect file is located at the path M4-R1\05-开发资料\01-OpenHarmory 开发资料\iqfile

3.3 Camera Test Demonstration

Taking the OV5695 adapted above as an example, connect the camera to the MIPI CAM interface while powered off. Be careful not to hot-plug!!! (The author burned out a board and a camera this way.)

Open the camera app that comes with the HarmonyOS system:

The image is successfully produced; the test is normal.