DMA issue while writing data to hard disk

Hi there,
Before I post my problem, let me present the overall picture.
We have developed a realtime embedded data acquisition system based on
A/D, FPGA, SATA controller, SATA HDD, microcontroller and Ethernet.
The purpose of the system is to record seismic data from the sensors,
digitize them and write them on the SATA HDD. Later, the recorded data
can be retrieved through Ethernet.
We have used Spartan 3 FPGA, Silicon Image SATA controller (Sil3512)
and SATA HDD. There is no OS in our system, we have developed our own
simple FAT, the FPGA acts as the host device which talks to the SATA
controller through a dedicated PCI interface. There is no PCI slot on
our system, we have used opencores PCI bridge on the FPGA which talks
to the SATA controller through its PCI interface. There are two ping
pong buffers in the FPGA, each 512 Kbytes. The digitized data is
stored in these buffers in a ping pong fashion at 36 Mbytes/s. The
FPGA programs the SATA controller DMA. The data is read out from the
buffers at 66 MHz and transported on the PCI bus at 33 MHz towards
SATA controller when the SATA controller initiates the read DMA. The
system works perfect when the incoming data rate is 24 Mbytes/sec. But
as we increase the data rate to 36 Mbytes/s (which is what we
require), we see loss of data occurring. Here is the algorithm for
data recording:

1. Continue filling buffer 1 and buffer 2 with digitized data in a
round robin fashion.
2. As soon as buffer 1 is full, program the DMA for SATA controller so
that it comes and reads out the filled buffer while buffer 2 is being
filled.
3. Wait for the DMA done from the SATA controller from the first DMA.
4. On finding DMA done, wait for buffer2 to be filled.
5. As soon as buffer 2 is filled, program the DMA for SATA controller
so that it comes and reads out the filled buffer 2 while buffer 1 is
being filled.
6. Wait for the DMA done from the SATA controller from the second DMA.
7. On finding DMA done, wait for buffer1 to be filled.
8. Repeat steps 2-7 till the data acquisition is stop.


The data loss occurs due to delay in step 3 and step 6 where the FPGA
waits for the DMA done signal from the SATA controller. At times, the
SATA controller takes longer than anticipated and due to this, data
buffers overflow. 36Mbytes/s should not be an aggressive data rate for
the SATA controller + SATA HDD specially when the PCI link is
dedicated only between the host and the SATA controller.. We have
tried out different numbers with PCI bridge configuration and SATA
controller PCI configuration but things have not improved. We are
using UDMA 6 mode. Anyone out there to guide us how to handle this
problem? Let me know if more information is required. Thanks in
advance.

Best Regards
Farhan

On Tue, 27 May 2008 01:45:30 -0700 (PDT), maverick wrote:
*snip*
What is the write speed of the drive?
How often will there be a track to track delay?
How often will there be a rotation delay?
Have you meassured the time of each part of the process?
Have you considered use more than 2 buffers? maybe smaller?
/hjj

maverick wrote:
Hi there,

Hello. My first guess would be that you don't have the drive's
write-back cache enabled, and that you're therefore missing a disk rev
between each write - just enough to allow 24 MB/sec throughput but not
enough to allow 36 MB/sec.

If you don't want to enable the write-back cache, you'll need to use
command queuing to let the disk accept the next buffer's data before the
current data has finished being transferred to the platters.

- bill

SATA controller takes longer than anticipated and due to this, data
buffers overflow.

This is a classic producer/consumer thing.

If you have slow consumer (SATA) and fixed rate of the producer (acquisition),
then the amount of data in the pipe will grow, at least grow in peaks, not in
sustained size.

So, allocate more memory for the pipe.

Surely you will not be able to work if the _sustained_ rate of SATA is slower
then sustained rate of the acquisition - the long enough process will consume
any amount of the available RAM.

But, to handle the negative peaks in SATA performance, larger pipeline is OK.
It will be filled only during these peaks, and drained to nearly zero at usual
time if SATA is faster then acquisition.

Also note that the larger the pipeline, the larger the time lags, but, for disk
writing, this is not relevant.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation

http://www.storagecraft.com