Rackmount RAID With Consistent 20 MB/sec+ Average Read and Write Speeds?

What low end fibre channel rackmount RAID arrays will give me a consistent
20 MB / second average read and write speed with a reasonable number of 15K
RPM SCSI drives in a RAID 5 or RAID 6 configuration?

I've been using Compaq RA4x000 arrays as a cheap disk source for
applications that do not require very high disk performance. I had
assumed the RA4x000 RAID systems would bottleneck on the 1 Gbit fibre
channel interface, which is far more throughput than I could get out even
six drives in a RAID 0 array. Well, I was wrong. I measured performance
today on a Proliant 6400R with 4 GB of memory, copying from a RAID 5 array
of five 15K rpm SCSI drives to a RAID 5 array with six 15K rpm SCSI drives.
Each array is located on a separate RA4x00 locally attached by a separate
Compaq FC 64-bit PCI card. I am getting an absolutely miserable 2.5 MB per
second average write performance at the destination RA4x00 array.

How is the above result possible? Each drive should pull data at minimum 3
MB per second, and five 15K drives in a RAID 5 should give me at minimum 10
MB/second read and write performance. I don't have bottlenecks in system
memory, on the PCI bus, on the fibre channel bus. I just don't see where
the 2.5MB/sec could be coming from unless it's hard coded into the array
itself. I was measuring performance on the "physicaldrive" in Windows
performance monitor, read and write, on each drive defined by the separate
RA4x000 arrays.

With my commodity home-user SATA arrays I am getting 30 MB/second read
performance, over gigabit ethernet at that. It just shouldn't be that
hard to find a business focused RAID array that can give consistent 20
MB/second or faster average read and write speeds? What are my options?

--
Will

Will wrote:
What low end fibre channel rackmount RAID arrays will give me a consistent
20 MB / second average read and write speed with a reasonable number of 15K
RPM SCSI drives in a RAID 5 or RAID 6 configuration?

I've been using Compaq RA4x000 arrays as a cheap disk source for
applications that do not require very high disk performance. I had
assumed the RA4x000 RAID systems would bottleneck on the 1 Gbit fibre
channel interface, which is far more throughput than I could get out even
six drives in a RAID 0 array. Well, I was wrong. I measured performance
today on a Proliant 6400R with 4 GB of memory, copying from a RAID 5 array
of five 15K rpm SCSI drives to a RAID 5 array with six 15K rpm SCSI drives.
Each array is located on a separate RA4x00 locally attached by a separate
Compaq FC 64-bit PCI card. I am getting an absolutely miserable 2.5 MB per
second average write performance at the destination RA4x00 array.

How is the above result possible? Each drive should pull data at minimum 3
MB per second, and five 15K drives in a RAID 5 should give me at minimum 10
MB/second read and write performance. I don't have bottlenecks in system
memory, on the PCI bus, on the fibre channel bus. I just don't see where
the 2.5MB/sec could be coming from unless it's hard coded into the array
itself. I was measuring performance on the "physicaldrive" in Windows
performance monitor, read and write, on each drive defined by the separate
RA4x000 arrays.

With my commodity home-user SATA arrays I am getting 30 MB/second read
performance, over gigabit ethernet at that. It just shouldn't be that
hard to find a business focused RAID array that can give consistent 20
MB/second or faster average read and write speeds? What are my options?

Well, it could be that you know what you're talking about and just
didn't think it necessary to provide all the details. But just in case:

The theoretical worst-case write bandwidth of a RAID-5 array of 15k rpm
drives is something like 50 kilobytes per second if you're performing
single-sector (512-byte) updates: it takes an average of about 6 ms. to
randomly position to and read in the target sector (and its associated
parity on a different disk, which we'll assume can be accessed in
parallel) and another 4 ms. to rotate around again to update the sector
and its associated parity, which means about 10 ms. for each 512 bytes
transferred - Q.E.D. If your array is accumulating such updates in a
non-volatile write-back cache it can leverage the disks' queue
optimization to at least double that performance (to over 100 KB/sec).
If you were using 4KB transfers rather than 512-byte transfers this
would bring worst-case throughput (with the aid of the non-volatile
write-back cache) up to a princely 1 MB/sec or so.

So it's possible that the problem is not with your array but with the
way you're using it - e.g., copying lots of very small files, perhaps.
If so, and that's the way you need to use the array, the only way you'll
get higher speeds is to go to solid-state disks (or perhaps you could
move to Solaris and use its new ZFS file system, which might do well on
this kind of workload because it can aggregate small writes into large
ones, much as a log-structured file system does).

On the other hand, if you're transferring large contiguous chunks of
data and still seeing this level of performance, then something is wrong
with your array: fix it, and it'll fix your problem.

- bill

"Bill Todd" wrote in message
news:36OdneA6NJx_hp_bnZ2dnUVZ_vCknZ2d@metrocastcab levision.com...
On the other hand, if you're transferring large contiguous chunks of data
and still seeing this level of performance, then something is wrong with
your array: fix it, and it'll fix your problem.

The performance of 2.5 MB/sec average write speed was seen while reading and
writing a single 250MB file, and the file was all contiguous on a 72 GB
logical drive that had about 40 GB unused. I examined the file's
allocation units on the drive with a file defrag application to verify that
everything was contiguous.

So it kind of does look like something basic is broken - possibly the array
just is not very good. I did try copying in both directions from and to
the same array, so it's not a single bad piece of hardware.

--
Will

Will wrote:
"Bill Todd" wrote in message
news:36OdneA6NJx_hp_bnZ2dnUVZ_vCknZ2d@metrocastcab levision.com...
On the other hand, if you're transferring large contiguous chunks of data
and still seeing this level of performance, then something is wrong with
your array: fix it, and it'll fix your problem.

The performance of 2.5 MB/sec average write speed was seen while reading and
writing a single 250MB file, and the file was all contiguous on a 72 GB
logical drive that had about 40 GB unused. I examined the file's
allocation units on the drive with a file defrag application to verify that
everything was contiguous.

So it kind of does look like something basic is broken - possibly the array
just is not very good. I did try copying in both directions from and to
the same array, so it's not a single bad piece of hardware.

Hmmm. You don't say what OS you're using. Most recent versions of
Windows tend to write large contiguous files in 64 KB chunks - but even
if your array had its write-back cache disabled, and even if Windows
weren't queuing up requests in advance (such that they would be combined
at the disk - though that could get thwarted if the RAID stripe were
also 64 KB per stripe segment), it should still get something more like
8 MB/sec throughput under those conditions (64 KB every 8 ms. or so,
even if the write requests didn't align with the RAID stripe segments
such that each write spanned two disks).

If the array and OS were cooperating as they should be doing, you ought
to be able to write large contiguous files (which should be being
streamed out to the array without waiting for it to finish each request
before queuing the next) at something close to N-1 times 40 MB/sec
(where N is the number of disks in the RAID stripe) - or in your case
around 100 MB/sec as limited by the FC link. So something isn't right.

- bill

In article ,
"Will" wrote:

What low end fibre channel rackmount RAID arrays will give me a consistent
20 MB / second average read and write speed with a reasonable number of 15K
RPM SCSI drives in a RAID 5 or RAID 6 configuration?

I've been using Compaq RA4x000 arrays as a cheap disk source for
applications that do not require very high disk performance. I had
assumed the RA4x000 RAID systems would bottleneck on the 1 Gbit fibre
channel interface, which is far more throughput than I could get out even
six drives in a RAID 0 array. Well, I was wrong. I measured performance
today on a Proliant 6400R with 4 GB of memory, copying from a RAID 5 array
of five 15K rpm SCSI drives to a RAID 5 array with six 15K rpm SCSI drives.
Each array is located on a separate RA4x00 locally attached by a separate
Compaq FC 64-bit PCI card. I am getting an absolutely miserable 2.5 MB per
second average write performance at the destination RA4x00 array.

How is the above result possible? Each drive should pull data at minimum 3
MB per second, and five 15K drives in a RAID 5 should give me at minimum 10
MB/second read and write performance. I don't have bottlenecks in system
memory, on the PCI bus, on the fibre channel bus. I just don't see where
the 2.5MB/sec could be coming from unless it's hard coded into the array
itself. I was measuring performance on the "physicaldrive" in Windows
performance monitor, read and write, on each drive defined by the separate
RA4x000 arrays.

With my commodity home-user SATA arrays I am getting 30 MB/second read
performance, over gigabit ethernet at that. It just shouldn't be that
hard to find a business focused RAID array that can give consistent 20
MB/second or faster average read and write speeds? What are my options?

I've done a lot of write performance testing on SATA drives lately, and
750 GB Seagates consistently get 70-80 megabytes per second and that's
on their own. A 4-disk RAID 6 of these drives is similar, maybe a tad
quicker.

I'm bypassing its page cache (Linux) and using an 800 MB file which
overpowers the RAID card's own cache, but if I weren't bypassing the
page cache, the numbers would be definitely be better. It'd have no
trouble saturating a gigabit link.

Also, in my other setup, I have a failover cluster, which consists of
two systems each with their own SCSI to SATA based RAID 6 and I'm doing
network RAID 1 between them. That averages about 60 megabytes per second
to take data on one system and write it to both RAIDs.