Orin Dev Kit Ethernet over PCIe EP performance

Hi,

Following PCIe Endpoint Mode — Jetson Linux Developer Guide documentation and using latest JetPack 5.1, getting low speeds ~5Gb/s relative to theoretical for PCIe 4.0 x8 which is ~128Gb/s. Uplink and downlink speeds initiated from either end are mostly symmetrical. Orin is set to max power.

iperf3 -c 192.168.2.1

[ ID] Interval           Transfer     Bitrate         Retr  Cwnd

[  5]   0.00-1.00   sec   482 MBytes  4.04 Gbits/sec    0   1.97 MBytes       

[  5]   1.00-2.00   sec   481 MBytes  4.03 Gbits/sec    0   1.97 MBytes       

[  5]   2.00-3.00   sec   476 MBytes  3.99 Gbits/sec    0   1.97 MBytes       

[  5]   3.00-4.00   sec   479 MBytes  4.02 Gbits/sec    0   1.97 MBytes       

[  5]   4.00-5.00   sec   476 MBytes  3.99 Gbits/sec    0   1.97 MBytes       

[  5]   5.00-6.00   sec   475 MBytes  3.99 Gbits/sec    0   1.97 MBytes       

[  5]   6.00-7.00   sec   425 MBytes  3.57 Gbits/sec    0   1.97 MBytes       

[  5]   7.00-8.00   sec   438 MBytes  3.67 Gbits/sec    0   1.97 MBytes       

[  5]   8.00-9.00   sec   465 MBytes  3.91 Gbits/sec    0   1.97 MBytes       

[  5]   9.00-10.00  sec   470 MBytes  3.94 Gbits/sec    0   1.97 MBytes       

- - - - - - - - - - - - - - - - - - - - - - - - -

[ ID] Interval           Transfer     Bitrate         Retr

[  5]   0.00-10.00  sec  4.56 GBytes  3.91 Gbits/sec    0             sender

[  5]   0.00-10.00  sec  4.56 GBytes  3.91 Gbits/sec                  receiver



iperf3 -s

[ ID] Interval           Transfer     Bitrate         Retr  Cwnd

[  5]   0.00-1.00   sec   749 MBytes  6.28 Gbits/sec                  

[  5]   1.00-2.00   sec   523 MBytes  4.38 Gbits/sec                  

[  5]   2.00-3.00   sec   436 MBytes  3.66 Gbits/sec                  

[  5]   3.00-4.00   sec   453 MBytes  3.80 Gbits/sec                  

[  5]   4.00-5.00   sec   447 MBytes  3.75 Gbits/sec                  

[  5]   5.00-6.00   sec   457 MBytes  3.84 Gbits/sec                  

[  5]   6.00-7.00   sec   666 MBytes  5.59 Gbits/sec                  

[  5]   7.00-8.00   sec   785 MBytes  6.58 Gbits/sec                  

[  5]   8.00-9.00   sec   795 MBytes  6.67 Gbits/sec                  

[  5]   9.00-10.00  sec   785 MBytes  6.58 Gbits/sec                  

[  5]  10.00-10.00  sec  1.19 MBytes  2.94 Gbits/sec                  

- - - - - - - - - - - - - - - - - - - - - - - - -

[ ID] Interval           Transfer     Bitrate

[  5]   0.00-10.00  sec  5.95 GBytes  5.11 Gbits/sec                  receiver

From lspci logs it looks like RP is downgraded to 8GT/s but EP is also downgraded to x1?
RP.eth0.log (4.3 KB) LnkSta: Speed 8GT/s (downgraded), Width x8 (ok)
EP.eth1.log (4.7 KB) LnkSta: Speed 2.5GT/s (downgraded), Width x1 (ok)

Does virtual Ethernet use DMA?

Do higher speeds require patching Orin EP or PC RP kernel or tvnet, or PCIe or Ethernet buffer sizes?

Thanks.