FPGA DMA Read/Write Demo

1 Demo overview

Functional description: the ARM side uses the PCIe bus to read and write the FPGA's DRAM. The application triggers a DMA data transfer through the ioctl function, then waits for an input event from the driver as a response. When the application receives that event, the DMA data transfer is complete.

2 Test program flow

ARM side

  1. Write data to the contiguous memory space allocated by the dma_memcpy driver (located in DDR). Configure DMA parameters such as source address, destination address, and transfer size.

  2. Write operation: start DMA via the ioctl function to move data to the FPGA DRAM over the PCIe bus; after the program receives the input event reported by the driver, it obtains the DMA transfer elapsed time through the ioctl function and computes the DMA write rate.

  3. Read operation: start DMA via the ioctl function to move data from the FPGA DRAM to the contiguous memory space allocated by the dma_memcpy driver (located in DDR) over the PCIe bus; after the program receives the input event reported by the driver, it reads the data from kernel space to user space, verifies the data, and obtains the DMA transfer elapsed time through the ioctl function to compute the DMA read rate.

FPGA side

  1. Implement the PCIe Endpoint function.

  2. Handle PCIe BAR0 space read/write transactions initiated by the PCIe RC side.

  3. Buffer the PCIe BAR0 read/write data in the FPGA DRAM.

3 Procedure

Open a terminal and copy the executable smdt_dma_memcpy_demo from this demo's bin directory (05-Development Materials\Software Development Materials\linux_demo\smdt_fpga_dma_memcpy_demo\bin) to the board's filesystem (the source code is available under the src path).

#修改 Linux 内核日志的显示级别，内核的日志级别被设置为只显示紧急或更高级别的消息
echo 1 4 1 7 > /proc/sys/kernel/printk

In the terminal, run the following commands to switch to the directory containing the smdt_dma_memcpy_demo executable.

#切换到 smdt_dma_memcpy_demo 可执行程序所在目录
cd ‘可执行文件所在目录’

#查看 smdt_dma_memcpy_demo 是否在该目录下
ls

If the executable smdt_dma_memcpy_demo is in the current directory, change its permissions.

#修改文件权限
chmod 777 smdt_dma_memcpy_demo

#查询是否修改成功
ls -ld smdt_dma_memcpy_demo

After confirming the permission change succeeded, run ./smdt_dma_memcpy_demo -h to view the program's help information.

#查看帮助信息
./smdt_dma_memcpy_demo  -h
#-a：设置 PCIe 地址；
#-s：设置传输数据大小（单位：Byte）；
#-c：设置循环读写次数；
#-d：设置输入设备；
#-v：显示版本信息；
#-h：显示帮助内容。

The result is shown below:

FPGA_DMA_3

Run the following command to view the PCIe BAR space address. As shown below, the PCIe BAR space is mapped to address 0xf0200000 with a size of 64KB, and you can check the PCIe device connection status.

lspci -s 0002:21:00.0 -vv
FPGA_DMA_1

Run the following command to enable the PCIe device.

echo 1 > /sys/class/pci_bus/0002\:21/device/0002\:21\:00.0/enable

After the application sends a command through the ioctl function to start the DMA data transfer, it waits for the driver to report an input event; when the application layer receives the input event, the DMA transfer is complete. Run the following command to view the input event interface. As shown in the figure, the input event interface is /dev/input/event0.

evtest
FPGA_DMA_4

Run the following command to write randomly generated data to the FPGA's DRAM, then read the data back from the FPGA's DRAM. After the test, it receives an event notification through /dev/input/event0 and outputs the test results, including average read/write time, average transfer rate, and the number of read/write errors. The result is shown below.

./smdt_dma_memcpy_demo -a 0xf0200000 -s 65536 -c 100 -d /dev/input/event0
#这里的event0根据evetest查询到的结果更改。
FPGA_DMA_3