PCIe access to reserved memory causes iommu fault
Mar 04, 2024
I’m trying to read/write to reserved memory from a PCI device, but when the PCI device tries to access reserve memory, I get an iommu error.
Here is the reserve memory entry I added to my device tree.
reserved-memory {
#address-cells = <0x02>;
#size-cells = <0x02>;
ranges;
ntb_dma_buffer {
reg = <0x1 0x00000000 0x0 0x08000000>; // 128 MB
no-map;
status = "okay";
};
};
This is the error I’m seeing on a Orin NX running Jetson Linux R35.4.1 when the PCI device is attempting to DMA from reserved memory.
[ 363.909202] arm-smmu 12000000.iommu: Unhandled context fault: fsr=0x80000402, iova=0x100000000, fsynr=0x6c0001, cbfrsynra=0x13, cb=5
[ 363.924244] mc-err: (255) csr_pcie4r: EMEM address decode error
[ 363.930353] mc-err: status = 0x200640e0; hi_addr_reg = 0x000000ff addr = 0xffffffff00
[ 363.938600] mc-err: secure: yes, access-type: read
I’ve also attempted to disable iommu for pcie4r by removing the lines from pcie@14160000 in the device tree.
iommus = <0x06 0x13>;
iommu-map = <0x00 0x06 0x13 0x1000>;
dma-coherent;
iommu-map-mask = <0x00>;
My attempt to disable iommu for the PCIe controller results in a different iommu fault when trying to read reserved memory.
[ 808.754503] arm-smmu 12000000.iommu: Unexpected global fault, this could be serious
[ 808.762424] arm-smmu 12000000.iommu: GFSR 0x80000002, GFSYNR0 0x00000000, GFSYNR1 0x00000013, GFSYNR2 0x00000000
[ 808.775625] mc-err: (255) csr_pcie4r: EMEM address decode error
[ 808.781743] mc-err: status = 0x200640e0; hi_addr_reg = 0x000000ff addr = 0xffffffff00
[ 808.789984] mc-err: secure: yes, access-type: read
What do I need to do to get this to work, and prevent an iommu fault when a PCI device is trying to access reserved memory?