PatC wrote:
> samliu wrote:
>> The problem is strange.
>> Our platform is ML555 with Xilinx V5. We need transport data from PC to
>> DRAM on the board through PCI Express Edge. And we successfully did
>> this on
>> DELL XPS with nForce 680i motherboard. But when we set up to do it on
>> DELL
>> t5400 workstation with Intel 5400 we got random errors that the data we
>> loopback from the SDRAM on the board were wrong and out of order.
>> I just wondering that which part can produce such errors, PCI Express DMA
>> app or DDR2 Controller? By the way, what we using are all sample
>> application provided by Xilinx.
>>
>> Thanks,
>> Sam Liu
>
> Hi,
>
> This problem bit me some months ago, and it's actually documented in
> the PCIe specification. The completions coming back to the board
> responding to non-posted requests, are split in several smaller packets.
> This small packets can 'pass' other packets.
>
> ie.
> Requests NPA, NPB, NPC come back on some motherboards like you'd expect:
>
> CA1, CA2, CA3, CB1, CB2, CB3, CC1, CC2, CC3
>
> But, on other mo-bos:
>
> CA1, CB1, CC1, CA2, CB2, CC2, CA3, CB3, CC3.
>
> I know it looks crazy. If you were stacking the data as it comes (ie.
> into a FIFO), you'd need to sort it before storing it. I used an array
> of 8 FIFOs to hold the data temporarily until all pieces of a completion
> were received.
> Another approach would be to store the data by their packet address,
> so it orders it as it is received. This way is how I perceive they
> intended the interface to be used, ie. mapped into memory.
>
> HTH,
> -P@
>
>

Each request gets a Tag value, and the completions have matching Tag
values. You need to match up the tags to see which completions are
connected to which requests.

If you had a real-time PCI Express Analyzer, admittedly a lot of money,
you would see this.

RB