FPGA EP DMA to Xavier(RC), data is written to nowhere

Hello, nVidia Support,

I am using a Xilinx FPGA as EP and DMA data to Xavier as RC. The Xilinx FPGA and DMA function was verified on an Intel x86_64 CPU, so it is confirmed the FPGA functions well. The issue comes when we port the driver from x86_64 to ARM64 on Xavier AGX. When set-up the DMA, the DMA looks like is running, can receive interrupts, but when read data from the buffer where the DMA writes data to, the data are pre-filled patterns 0x5a5a. It looks like the DMA is writing data to somewhere else.

Here are the address I use for setup the DMA, I use the kernbuf Address and program to FPGA as DMA destination address. The userBuf Address is the virtual address that I read from. These address were obtained by calling a DMABufAlloc function that is from a driver from a third-party company.

buf_0 bufferSize : 0x100000

buf_0 userBuf Addr: 0x7f9edee000

buf_0 kernBuf Addr: 0x4fff00000

buf_0 kernBuf64 Addr: 0x4fff00000

My question is, for the kernBuf address that is to be used as the DMA destination address, does it fall within 0x8000_0000 to 0xFFF0_0000?

as per this post:

In this nvidia forum, post 2 states that “Bus address for PCIe is reserved from 0x8000_0000 to 0xFFF0_0000. So, whether the memory is allocated directly or mapped later on, iova address always comes from this region.”

PCIE DMA Problem between TX2 & FPGA - #2 by Phoenixlee

2). some forum posts state by disabling SMMU might solve this issue. Is this the only solution for this? Since the CPU designed the SMMU/IOMMU, there are good reasons to use this, and we would like to keep these functions. What are the solutions to resolve the above issue?

Thank you.

Mei Guodong