Xavier AGX PCIe configuration
We have a project that requires a high-speed 32-channel PCIe DAQ (analog to digital converter) and we are using the Xavier AGX dev kit. The manufacturer of the PCIe card (General Standards) is having trouble getting their driver to work properly with the AGX and sent me the following information. This is very time sensitive, so I appreciate a quick response:
(From the manufacturer)
I’ve identified the problem.
- The problem is that the host’s Plug-N-Play configuration of the GSC
card is ignoring certain configuration bits when making physical address
assignments. The 18AI32SSC1M, and virtually all GSC devices, are
configured to have their BAR0 and BAR2 regions assigned to 32-bit
addresses. This is specified with bits D1 and D2 within both of the BAR0
and BAR2 PCI registers. This can be verified by issuing the command
“lspci -vv” (two v’s) and looking at the “Region 0” and “Region 2” lines
towards the very bottom of the output (PCI device 5:02). Those same two
lines however show the physical addresses assigned to each region. What
I see are 37-bit addresses. While the 18AI32SSC1M can support 64-bit
addresses the board is configured for 32-bit addresses, which is what
the driver is designed to support. I initially verified this with a
different I/O card which was limited to 32-bit addresses. In this case
the board was still assigned addresses which the board did not support.
In the end, the driver used the lower 32-bits of the physical address to
try access the device. Given that the 32-bits of physical address were
not the correct address, the virtual addresses given to the driver to
access device registers were also wrong. When those virtual addresses
were use result was a system reboot.
NOTE: With my modified driver I was able to look at the BAR0 and BAR2
registers and verified that they were configured for 32-bit addressing.
- A similar issue appears with the Plug-N-Play initialization of the
BAR1 region. While this region is unused by the driver, it is not
initialized correctly. The “lspci -vv” output for “Region 1”, towards
the bottom of the output, shows how the region should have been
initialized. With my modified driver I was able to read the BAR1
register. Its assigned address was zero though the lspci output shows it
should be otherwise.
I recommend that NVIDIA be contacted regarding this issue. I’ll be glad
to participate in any such email exchange if you desire. If they ask,
the 18AI32SSC1M uses a PLX PCI9056 PCI Bus Interface chip.