Increasing PCIe non-prefetchable memory size on Orin Nano
Hi,
We are using jetson Orin Nano 4GB module with custom carrier board, interfacing FPGA through pcie, and we need to increase the non-prefetchable memory size from 128MB to 2GB+ on PCIe C1 RP.
I have changed the device tree settings from:
pcie@14100000 {
…
ranges = <0x81000000 0x00 0x30100000 0x00 0x30100000 0x0 0x00100000 /* downstream I/O (1MB) /
0x82000000 0x00 0x40000000 0x20 0xa8000000 0x0 0x08000000 / non-prefetchable memory (128MB) /
0xc3000000 0x20 0x80000000 0x20 0x80000000 0x0 0x28000000>; / prefetchable memory (640MB) */
…
to
ranges = <0x81000000 0x00 0x30100000 0x00 0x30100000 0x0 0x00100000 /* downstream I/O (1MB) /
0x82000000 0x00 0x40000000 0x25 0x40000000 0x0 0x90000000 / non-prefetchable memory (2304MB) /
0xc3000000 0x20 0x80000000 0x25 0xd0000000 0x0 0x20000000>; / prefetchable memory (512MB) */
And we have reflashed the board with the new device tree settings. Now the board can boot and the pcie device is correctly enumerated with the new addresses.
However when I try to access the new addresses with devmem2, it returns all F’s and I got some sort of CBB error from dmesg:
[ 251.747782] CPU:0, Error: cbb-fabric@0x13a00000, irq=22
[ 251.753168] **************************************
[ 251.758096] CPU:0, Error:cbb-fabric, Errmon:2
[ 251.762581] Error Code : PWRDOWN_ERR
[ 251.768152] Error Code : PWRDOWN_ERR
[ 251.772182] MASTER_ID : CCPLEX
[ 251.775681] Address : 0x2542000000
[ 251.779537] Cache : 0x0 – Device Non-Bufferable
[ 251.784736] Protection : 0x2 – Unprivileged, Non-Secure, Data Access
[ 251.791722] Access_Type : Read
[ 251.795210] Access_ID : 0x12
[ 251.795212] Fabric : cbb-fabric
[ 251.802111] Slave_Id : 0xa
[ 251.805247] Burst_length : 0x0
[ 251.808742] Burst_type : 0x1
[ 251.812056] Beat_size : 0x3
[ 251.815282] VQC : 0x0
[ 251.818061] GRPSEC : 0x7e
[ 251.821109] FALCONSEC : 0x0
[ 251.824338] **************************************
It seems like the new addresses are not powered for some reason. Is there any other configuration I’m missing? Or is this simply something that is impossible to do?
Please note that the memory accessing works with no problem with the original memory mapping and size, which means there is no problem from the PCIe Endpoint(FPGA) side.