FPGA Central

The problem is strange.
Our platform is ML555 with Xilinx V5. We need transport data from PC t
DRAM on the board through PCI Express Edge. And we successfully did this o
DELL XPS with nForce 680i motherboard. But when we set up to do it on DEL
t5400 workstation with Intel 5400 we got random errors that the data w
loopback from the SDRAM on the board were wrong and out of order.
I just wondering that which part can produce such errors, PCI Express DM
app or DDR2 Controller? By the way, what we using are all sampl
application provided by Xilinx.

Thanks,
Sam Liu

samliu wrote:
> The problem is strange.
> Our platform is ML555 with Xilinx V5. We need transport data from PC to
> DRAM on the board through PCI Express Edge. And we successfully did this on
> DELL XPS with nForce 680i motherboard. But when we set up to do it on DELL
> t5400 workstation with Intel 5400 we got random errors that the data we
> loopback from the SDRAM on the board were wrong and out of order.
> I just wondering that which part can produce such errors, PCI Express DMA
> app or DDR2 Controller? By the way, what we using are all sample
> application provided by Xilinx.
>
> Thanks,
> Sam Liu

Hi,

This problem bit me some months ago, and it's actually documented in
the PCIe specification. The completions coming back to the board
responding to non-posted requests, are split in several smaller packets.
This small packets can 'pass' other packets.

ie.
Requests NPA, NPB, NPC come back on some motherboards like you'd expect:

CA1, CA2, CA3, CB1, CB2, CB3, CC1, CC2, CC3

But, on other mo-bos:

CA1, CB1, CC1, CA2, CB2, CC2, CA3, CB3, CC3.

I know it looks crazy. If you were stacking the data as it comes (ie.
into a FIFO), you'd need to sort it before storing it. I used an array
of 8 FIFOs to hold the data temporarily until all pieces of a completion
were received.
Another approach would be to store the data by their packet address,
so it orders it as it is received. This way is how I perceive they
intended the interface to be used, ie. mapped into memory.

HTH,
-P@

PatC wrote:
> samliu wrote:
>> The problem is strange.
>> Our platform is ML555 with Xilinx V5. We need transport data from PC to
>> DRAM on the board through PCI Express Edge. And we successfully did
>> this on
>> DELL XPS with nForce 680i motherboard. But when we set up to do it on
>> DELL
>> t5400 workstation with Intel 5400 we got random errors that the data we
>> loopback from the SDRAM on the board were wrong and out of order.
>> I just wondering that which part can produce such errors, PCI Express DMA
>> app or DDR2 Controller? By the way, what we using are all sample
>> application provided by Xilinx.
>>
>> Thanks,
>> Sam Liu
>
> Hi,
>
> This problem bit me some months ago, and it's actually documented in
> the PCIe specification. The completions coming back to the board
> responding to non-posted requests, are split in several smaller packets.
> This small packets can 'pass' other packets.
>
> ie.
> Requests NPA, NPB, NPC come back on some motherboards like you'd expect:
>
> CA1, CA2, CA3, CB1, CB2, CB3, CC1, CC2, CC3
>
> But, on other mo-bos:
>
> CA1, CB1, CC1, CA2, CB2, CC2, CA3, CB3, CC3.
>
> I know it looks crazy. If you were stacking the data as it comes (ie.
> into a FIFO), you'd need to sort it before storing it. I used an array
> of 8 FIFOs to hold the data temporarily until all pieces of a completion
> were received.
> Another approach would be to store the data by their packet address,
> so it orders it as it is received. This way is how I perceive they
> intended the interface to be used, ie. mapped into memory.
>
> HTH,
> -P@
>
>

Each request gets a Tag value, and the completions have matching Tag
values. You need to match up the tags to see which completions are
connected to which requests.

If you had a real-time PCI Express Analyzer, admittedly a lot of money,
you would see this.

RB