How to use DMA to write PCIe data
Hi. I am attempting to use DMA to receive PCIe data on NVIDIA AGX Xavier end point. I have two Jetson AGX Xaviers, one acting as root port, and the other acting as end point.
I can send data between the two using the method given in this guide:
https://docs.nvidia.com/jetson/archives/r35.2.1/DeveloperGuide/text/SD/Communications/PcieEndpointMode.html
However, I need max throughput for PCIe, which is why I need to use DMA. For that, I read the Xavier technical reference manual. In there, I found the following guidelines for programming DMA on page 110:
Based on this, I used the following sequence of data writes to program DMA channel 1 on end point (attempting to program channel 0 resulted in kernel panic).
busybox devmem 0x0262000C 32 0x3a040000 // source address
busybox devmem 0x02620010 32 0x80000000 // destination address
busybox devmem 0x02620018 32 0x03800000 // burst-size 16 words, no word wrap, no byte swap
busybox devmem 0x0262001C 32 0x2780007f // peripheral (PCIe) bus-width (32-bit wide), no byte-swap, no word wrap, and burst size 16 words
busybox devmem 0x02620020 32 0x00000400 // data length (0x400 words = 4kB data)
busybox devmem 0x02620008 32 0x0002C000 // No idea what this does. Just selected channel 1 here
busybox devmem 0x02620000 32 0x09208400 // program the control register: disable all interrupts (need to focus only on DMA, will integrate interrupts later), use fully coherent, MMIO to Mem mode.
busybox devmem 0x02620000 32 0x89208400 // enable the DMA transfer
Here, I set the destination address randomly, and the took the source address from the PCIE_C5_32BIT_DMA entry on page 70 of the technical reference manual (not sure whether this is correct or not).
When I send data from root port to end point, I dont see any data at address 0x80000000 (which is my destination address).
What am I doing wrong? I am new to this, and am not sure whether the above process is correct or not, so any help would be appreciated.
Regards,
Sana Ur Rehman