PCIE Lane Reversal on C5 and C0
Hi Folks -
We are dealing with a PCIE lane reversal issue and would appreciate your input. This question is similar to the posts below, but we have not had success with lane reversal:
- PCIe lane reversal on C0
- PCIe C5 Lane Reversal
- Question about pcie lane reversal
We do not see lane reversal work cleanly. We are using a test setup with two devkits, one configured as Rootport and one configured as Endpoint. We are using Jetpack 4.6.
Straight Thru
When we use a straight thru crosslink (a rigid, passive PCBA with appropriately impedance matched routing) that maps Lane 0 to Lane 0, Lane 1 to Lane 1, etc, we close a link as described in all the various documentations and on this forum. The link status is reports as 16 GT/s and x8 width using lspci -vv. This appears to be working as specified.
Lane Reversal
When we use a crossover PCBA that maps Lane 0 to Lane 7, 1 → 6, 2 → 5, 3 → 4, 4 → 3, 5 → 2, 6 → 1, and 7 → 0, it does not work.
When we modify the pcie driver to assert TX_LANE_FLIP_EN RX _LANE _FLIP _EN and AUTO_LANE_FLIP_CTRL_EN before releasing LTSSM_EN, it does not work.
When we modify the pcie driver to assert the PRE_DET_LANE register to 0b011, (corresponding to an x8 lane reversal according to the TRM,) the link closes, but only at x1.
Configurations
We have tried several configurations, a few interesting ones are enumerated here:
| Description | with Straight PCBA | with Lane Reversed PCBA |
|---|---|---|
| Unmodified | x8 16GT/s | FAILS |
| PRE_DET_LANE = 0b011, enables | x1 16 GT/s | x1 16 GT/s |
| PRE_DET_LANE = 0b000, enables | x8 16GT/s | FAILS |
| only PRE_DET_LANE = 0b011 | x1 16 GT/s | x1 16 GT/s |
We have also tried some C0 to C5 links with similar results. We can get an x1 link to close, but we cannot get an x4 link to close.
Questions
We’ve been through the forums, online searches, and “PCI Express Technology” by Jackson and Budruk. We are clearly missing something, but have yet to figure out the magic incantation to get link training to recognize lane reversal.
- Has anyone successfully made a lane reversed crossover between two devkits work?
- Are there any more diagnostics we can run that will shed light on the problem? We’ve tried to parse lspci -vv but we do not see anything relevant to lane reversal.
- Are there any suggestions for procedure changes, code changes, or tests we can run?
Thanks in advance for reading this lengthy post, and we very much appreciate your input.
Best regards,
sw