ARM and FPGA Communication

1 Hardware Interface Introduction

There are three interfaces for communication between FPGA and RK: IIC, FSPI, and PCIE. The schematic diagram is shown below:

2 IIC Communication

In IIC communication, FPGA acts as a slave, and its device address is 0x66.

Currently, three registers are defined in the code with addresses 0x00, 0x01, and 0x02. Among them, 0x02 is a read-only register.

For register 0x00, you can test whether the read and write data is correct. For example, if you write 1, reading it out should also be 1, indicating normal communication.

i2cdetect -r -y -a 0

You can see that there is an address 0x66 on the I2C bus.

i2cset -r -y -a 0 0x66 0x00 0x40

After executing this command, 0x40 will be written to register 0x00.

At this time, enter the command i2cdump -f -y -a 0 0x66

This command will list the values of all registers of 0x66. You can see that the value of register 0x00 is already 0x40 just written.

i2cget -f -y -a 0 0x66 0x00

Read the value of register 0x00, you can see that 0x40 is read out, which is consistent with the written value.

i2cset -f -y -a 0 0x66 0x01 0x01

At this time, the LED on the baseboard lights up.

Enter the following command:

i2cset -f -y -a 0 0x66 0x01 0x00

At this time, the LED on the baseboard goes out.

i2cset -f -y -a 0 0x66 0x01 0x03

At this time, both LEDs on the baseboard light up.

Because the lower 2 bits of this register are assigned to the LEDs in the code, the LEDs can only be lit when the lower 2 bits are 1. So far, IIC read and write communication is normal.

3 FSPI Communication

FSPI is used for interaction between FPGA and RK. FPGA will have a RAM to buffer data to achieve read-write loopback.

First load the driver:

insmod smdt_spi_controller.ko

Then enter the command: lsmod

If the words in the red box above appear, it means the driver is loaded successfully.

Switch the directory to the folder of the smdt_spi_rw executable program, and then enter the command:

./smdt_spi_rw -d /dev/spidev4.0 -s 50000000 -OH -m 3 -S 1024 -c 1

If Byte error rate is 0, it means the communication is normal and the data read and write is correct.

Among them, -s sets the communication frequency, 50MHZ.

-m indicates the selection mode: 1 is single-wire mode, 2 is two-wire mode, and 3 is four-wire mode.

-S indicates the selected transmission size, here is 1024. Since the RAM depth of FPGA is set to 2048, if the transmission exceeds 2048, a byte error will occur.

-c indicates the number of transmissions, 1 indicates transmission 1 time.

4 PCIE Communication

Related File Paths

smdt_fpga_dma_memcpy_demo project directory path:
01-Development Materials (Baidu Cloud Disk) -> 05-Development Materials -> 01-Linux System -> linux_demo -> smdt_fpga_dma_memcpy_demo

【Based on the above project path】
FPGA firmware file path: fpga_sfc -> dram_pcie_pg2l50h.sfc
smdt_fpga_dma_memcpy_demo executable file path: bin -> smdt_dma_memcpy_demo

The FPGA side needs to flash the PCIe project first, because RK will only identify the PCIe device once during power-on, so ensure that the FPGA has loaded the PCIe program before RK starts up, otherwise it will cause identification failure.

Connect the USB downloader of the FPGA.

Note

After inserting the USB into the USB port of the computer, the downloader lights up red, and after the board is powered on, the downloader lights up yellow.

Open the FPGA flashing software.

Power on the development board, and then click the icon pointed by the arrow in the software below.

When the device is identified, the following interface will pop up. Close the popup window.

Move the mouse to the chip, right-click, and then select the option pointed by the arrow below.

In the popup window, select the dram_pcie_pg2l50h.sfc we need, and then click [Open].

Note

Do not use Chinese characters in the sfc file storage path, otherwise an error will occur and it cannot be opened.

Right-click [Outer Flash], select the option [Program] pointed by the arrow below, and the program will be flashed.

Waiting for flashing to complete...

Console displays the following information indicating that flashing is complete.

4.2 Run PCIE Communication Demo

Restart the development board.
Enter the following command in the debugging tool to switch adb.

echo 1 > /sys/devices/platform/fe8a0000.usb2-phy/otg_mode
echo 2 > /sys/devices/platform/fe8a0000.usb2-phy/otg_mode
usbdevice start

The operation effect is as follows:

Open cmd to check if there is an adb device.

adb devices

Push smdt_dma_memcpy_demo to the board via adb push command.

Note

The case places smdt_dma_memcpy_demo in drive E and intends to transfer it to the data directory of the board.

adb push E:\smdt_dma_memcpy_demo /data/

Check if the file exists in the board and add executable permission. Enter the following commands in cmd.

adb shell
cd /data
ls
chmod +x ./smdt_dma_memcpy_demo
ls -al

Check if PCIE has established a link and if input_dev_demo is generated via commands.

lspci -vv
cat /proc/bus/input/devices | grep -i "Name=\"input_dev_demo\"" -A 8 -B 1

Enter lspci -vv, the output is as shown below. If it can display 0755:0755 and the lnkSta status is normal, Speed 5GT/S, Width x2, it means PCIe has established a link. You can also see the LED light on the board flashing quickly.

Note

From the figure below, it can also be seen that the PCIe BAR space is mapped to 0xf0200000 with a size of 64KByte.

Enter cat /proc/bus/input/devices | grep -i "Name=\"input_dev_demo\"" -A 8 -B 1, the output is as shown below. The input_dev_demo device in the case is event0.

Execute the following commands to run the PCIE communication Demo.

echo 1 > /sys/class/pci_bus/0002:21/device/0002:21:00.0/enable
./smdt_dma_memcpy_demo -a 0xf0200000 -s 20480 -c 1 -d /dev/input/event0

Info

If there is no error in both read and write, it means DMA interaction is normal.
-a indicates address
-s indicates the number of bytes transmitted
-c indicates the number of transmissions
-d indicates the device path
You can adjust s and c to change the data volume of DMA transmission.

4.3 Source Code Analysis

1️⃣ Open Device Node

2️⃣ Write Test Phase Code Analysis (ARM -> FPGA)

Info

Function: static int perform_write_test(DMAContext *ctx, uint16_t *write_buf);
Flow: ARM prepares data → DMA setting → Memory mapping → CPU copy → DMA transmission → Wait for completion

DMA Configuration

Establish DMA Channel

Establish Memory Mapping

CPU Data Copy and Performance Test

Start DMA
Start DMA through the ioctl function and transport data to FPGA DRAM via the PCIe bus.

Wait for Transmission Completion
The program waits for the input event reported by the driver.

Obtain DMA Transmission Performance Indicators
After the program receives the input event reported by the driver, it indicates that the DMA transmission is complete. The DMA data transport time is obtained through the ioctl function, and the DMA transmission rate is calculated.

Cleanup and Close
1. Clear memory mapping
2. Close DMA transmission and return status

3️⃣ Read Test Phase Code Analysis (FPGA -> ARM)

Info

Function: static int perform_read_test(DMAContext *ctx, uint16_t *write_buf, uint16_t *read_buf);
Flow: DMA setting → Start DMA transmission → Wait for completion → Memory mapping → CPU copy → Data verification

DMA Configuration

Establish DMA Channel

Start DMA
Start DMA through the ioctl function and transport FPGA DRAM data to the continuous memory space (located in DDR) applied by the driver via the PCIe bus.

Wait for FPGA Data Transmission

Establish Memory Mapping

Read Data
After the program receives the input event reported by the driver, it reads data from kernel space to user space.

Data Integrity Verification
Verify if the data is consistent with the written data.

Obtain DMA Performance Indicators
Obtain DMA data transport time through ioctl function and calculate DMA transmission rate.

Cleanup and Close
1. Clear memory mapping
2. Close DMA transmission and return status

4.4 ioctl Function

The specific implementation code can be seen in the smdt_pcie_dma_memcpy.c file in the kernel source code.
Part of the code is as follows:

Specific File Path

File Path: SDK/kernel-6.1/drivers/pci/pcie/smdt_pcie_dma_memcpy.c

4.5 FPGA Part Detail

For the content of the FPGA part in the PCIe communication test, please refer to: Based on PCIe DMA/PIO Control Experiment