1.485 Gbit/s to and from HDD subsystem

Al Dykes wrote:

Applying averages to a specific instance of the equipment is a classic
mistake.

In general, applying averages without knowing the distribution is fraught with
error. If these drives have a deep bathtub failure distribution (most either
fail relatively early in life or late in life but relatively few fail near the
average) then assuming average life is a crap shoot.

Personally, I start worrying about a "proven" drive once it gets around five
years old, regardless of what the manufacturer's claims are. My
not-very-scientific observation is that they tend to go around that age.

Mike

On Wed, 06 Dec 2006 14:28:09 +0100, Spoon
wrote:

Ryan Godridge wrote:

I'd second all of Tony's suggestions. For the price of 4 Raptors you
might also consider 6 or 7 less expensive sata drives. They might
give you your throughput with more ceiling for a lower cost.

I've read about Physical Track Positioning.
http://www.nyx.net/~sgjoen/disk1.html#ss6.8

For example, the Raptor WD1500 manages ~88 MB/s on outer tracks and
~60 MB/s on inner tracks, while the Barracuda 7200.10 starts at ~80 MB/s
on outer tracks and ends at ~40 MB/s on inner tracks.

http://anandtech.com/printarticle.aspx?i=2760

However, the WD1500 holds "only" 150 GB while the 7200.10 holds 750 GB.
If one looks at the throughput of the 7200.10 on the first 500 GB, it
never falls below 60 MB/s. And the throughput on the first 650 GB never
falls below 50 MB/s.

If it were easy to specify that one wants to use "only the outer x GB"
then one can get good performance from large disks.

Has anyone played with this at all? in Windows? in Linux?

Regards.

Hmm good points on the inner tracks.

Are these requirements burst or sustained transfer rates? If burst,
how much data at a time?

Are you going to be reading and writing concurrently?

The more I look at this, the chunkier the system required seems to
get.

Ryan Godridge wrote:

Spoon wrote:

Ryan Godridge wrote:

I'd second all of Tony's suggestions. For the price of 4 Raptors you
might also consider 6 or 7 less expensive sata drives. They might
give you your throughput with more ceiling for a lower cost.

I've read about Physical Track Positioning.
http://www.nyx.net/~sgjoen/disk1.html#ss6.8

For example, the Raptor WD1500 manages ~88 MB/s on outer tracks and
~60 MB/s on inner tracks, while the Barracuda 7200.10 starts at ~80 MB/s
on outer tracks and ends at ~40 MB/s on inner tracks.

http://anandtech.com/printarticle.aspx?i=2760

However, the WD1500 holds "only" 150 GB while the 7200.10 holds 750 GB.
If one looks at the throughput of the 7200.10 on the first 500 GB, it
never falls below 60 MB/s. And the throughput on the first 650 GB never
falls below 50 MB/s.

If it were easy to specify that one wants to use "only the outer x GB"
then one can get good performance from large disks.

Has anyone played with this at all? in Windows? in Linux?

Are these requirements burst or sustained transfer rates? If burst,
how much data at a time?

Sustained transfer rates.

E.g. capturing 1 minute of HD-SDI video means sequentially writing
186 MB/s for 1 minute, i.e. approximately 11 GB. Likewise, playing
out 1 minute means sequentially reading 11 GB.

Are you going to be reading and writing concurrently?

No. We'll capture a stream once, then play it out over and over.

The more I look at this, the chunkier the system required seems to
get.

Chunky?

The little lost angel wrote:

Spoon wrote:

It is interesting to note that Western Digital claims "1.2 million hours
MTBF at 100% duty cycle".

1.2 million hours = 50000 days i.e. ~137 years :-)

If one were to believe the marketing claims, a RAID-0 array of 2 or even
4 disks should still be quite reliable.

Actually, I think it means that if Western Digital sells 1.2 million
drives a month, one will fail every hour assuming 100% duty cycle.

I took it to mean:

Let X be the time between two failures, then P(X = t) = 1/2 * t/MTBF

i.e. a continuous uniform distribution over [0, 2*MTBF]

With these assumptions, P(X = 365) -- the probability for a disk to
fail within the first year -- equals 0.00365

So doing some maths, with some help, it works out to be around a 7.3%
chance of failure every year for 4 drives.

Our figures differ.

Assuming independent random variables, the probability for (at least)
one disk among 4 to fail within the first year equals 1 - 0.99635^4,
i.e. ~0.01452, i.e. roughly 4 * P(X = 365)

How did you reach your result?

Assuming an exponential distribution,

P(X = t) = 1 - e^(-t/MTBF)

P(X = 365) = 0.00727

In this case, the probability for (at least) one disk among 4 to fail
within the first year equals ~0.02878.

Regards.

On Wed, 06 Dec 2006 16:27:19 -0500, Michael Daly
wrote:

Al Dykes wrote:

Applying averages to a specific instance of the equipment is a classic
mistake.

In general, applying averages without knowing the distribution is fraught with
error. If these drives have a deep bathtub failure distribution (most either
fail relatively early in life or late in life but relatively few fail near the
average) then assuming average life is a crap shoot.

Personally, I start worrying about a "proven" drive once it gets around five
years old, regardless of what the manufacturer's claims are. My
not-very-scientific observation is that they tend to go around that age.

Keep in mind that the MTBF for drives only apply within the expected
life cycle of the drive, which is normally either 3 or 5 years
(usually the same as the warranty period), so any failures beyond 5
years just aren't counted at all. A lot of manufacturers also ignore
the first 90 days when they calculate the MTFB, saying that this is
not part of the Useful Life Cycle of the drive (pretty much BS, but
serves to boost MTBF numbers quite nicely).

Long story short, MTBF means ****-all for any practical purpose.
--
Tony Hill
hilla underscore 20 at yahoo dot ca

On Wed, 06 Dec 2006 14:49:24 +0100, Spoon
wrote:

Tony Hill wrote:

Spoon wrote:

I've been asked to build a system capable of reading and writing "raw"
high-definition video, namely HD-SDI.

http://en.wikipedia.org/wiki/Serial_Digital_Interface

Yeouch! That's no small feat! Keep in mind that you aren't just
going to need to worry about the hard drives, but also getting the
data to and from somewhere useful! That probably means some pretty
specialized video equipment (though I'm guessing you already know
about that part!) and probably some pretty beefy LAN.

You're right. I also need an HD-SDI PCIe board with Linux 2.6 support.

If you plan on keeping costs semi-reasonable (ie no SCSI) then the
Raptor 150/Raptor X is pretty much the only drive that will fit the
bill for you. 4 of those should indeed do the trick, and I'm quite
certain that you're going to want the extra 50MB/s+ worth of
theoretical headroom.

Is it possible to use only the outer tracks of larger disks?
(As discussed in another message.)

It should be possible, though I've never done it. Disks pretty much
always work from the outside in, so the first sector will be on the
outmost section of the platters while the last sector will be at the
innermost section.

With a single disk this is easy, you just assign the partition to only
use the first x sectors of the disk and then ignore the last y
sectors, quite easy to do in Linux fdisk and I believe Windows works
more or less the same way but with megabytes and gigabytes instead of
sectors. This gets trickier with a RAID array though, since your
partitions are actually being made by the RAID controller while what
the OS sees is a sort of virtual disk. As a result it will be up to
the controllers firmware to decide how the partition is going to
reside on the physical disk. I would take a guess that it will pretty
much always just do the simpliest thing and follow the same format as
for a single hard disk, putting the first sectors of the virtual disk
on the first sector of each of the physical disks.

A few other points of note, many of which you are probably well aware
of already, but others might be new:

1. You'll almost certainly want a dual-core processor (if not 4
cores). This data streaming on it's own is going to be enough to
swamp a fairly capable single core. With only a single-core chip any
other tasks (programs, OS, whatever) are going to start eating into
your performance. A dual-core chip should go a long way to keeping
things running smoothly.

I'm aiming for socket AM2 Athlon 64 X2 4600+ (dual core, 2.4 GHz).
(I'm also considering Core 2 Duo.)

Either option should be fairly good, though I would tend to favor the
Core 2 Duo myself. There is more headroom for higher processors on
the Core 2 Duo platform at the moment, if it turns out that the chip
you select doesn't provide enough ummpf for what you need, including a
quad-core chip at the very top-end of things.

Price/performance also tends to favor Core 2 Duo most of the time as
well, though of course that depends somewhat on your exact
applications, which likely hasn't been very well benchmarked!

2. Enough memory that you basically won't ever need to worry about
paging out your OS or applications.

Aiming for 2-4 GB.

Good call! 2GB actually will probably be sufficient so long as your
software operates properly. Caching the video stream is going to be
pretty much pointless since you'll overfill any cache in a matter of
seconds anyway, so really you're main concern is that nothing else
runs out of memory and tries to swap. In Linux it may actually be
acceptable to not have a swap partition at all, though the different
paging mechanisms of Windows might make that extremely difficult and
not necessarily intelligent on that platform.

5. Be sure that your case has lots of airflow. At the very least
you're going to have 4 drives spitting out a fair chunk of heat along
with one fairly high-end processor. And whatever you're using to take
data in and spit it out again are also going to be some high-end
parts. All in all, that's a LOT of heat being generated in a case,
even if it is going to be a pretty large case. Now SATA is a godsend
here when compared to PATA, since you'll have MUCH less ribbon cable
cluttering up your case, but you'll still need to make sure that the
cables stay neatly tied up and you've got fans sucking and blowing air
effectively throughout the case.

For reference, the WD1500 dissipates 10W in use, 9W idle.
http://www.westerndigital.com/en/pro...sp?DriveID=189

Figure at least 4 of those plus one or two drives as
boot/OS/application drive and you're looking at a minimum of 50W for
the drives alone.

Hmm, it's unfortunate that the documents don't list peak power at
spin-up. Usually the initial spin-up is when these drives consume the
most power. Some RAID controllers allow you to specify a delay
between spinning up drives so that the whole array doesn't try to spin
up at once and overload the powersupply.
--
Tony Hill
hilla underscore 20 at yahoo dot ca

In article ,
says...
On Wed, 06 Dec 2006 16:27:19 -0500, Michael Daly
wrote:

Al Dykes wrote:

Applying averages to a specific instance of the equipment is a classic
mistake.

In general, applying averages without knowing the distribution is fraught with
error. If these drives have a deep bathtub failure distribution (most either
fail relatively early in life or late in life but relatively few fail near the
average) then assuming average life is a crap shoot.

Personally, I start worrying about a "proven" drive once it gets around five
years old, regardless of what the manufacturer's claims are. My
not-very-scientific observation is that they tend to go around that age.

Keep in mind that the MTBF for drives only apply within the expected
life cycle of the drive, which is normally either 3 or 5 years
(usually the same as the warranty period), so any failures beyond 5
years just aren't counted at all. A lot of manufacturers also ignore
the first 90 days when they calculate the MTFB, saying that this is
not part of the Useful Life Cycle of the drive (pretty much BS, but
serves to boost MTBF numbers quite nicely).

The useful life is generally stated. Yes there is a bathtub curve
for fails, but the MTBF includes the head end. If there is a
serious issue with early fails, manufacturers use burn-in
techniques..

Long story short, MTBF means ****-all for any practical purpose.

Only to the uninformed.

--
Keith

On Thu, 07 Dec 2006 16:27:19 +0100, Spoon
wrote:

The little lost angel wrote:
So doing some maths, with some help, it works out to be around a 7.3%
chance of failure every year for 4 drives.

Our figures differ.

Assuming independent random variables, the probability for (at least)
one disk among 4 to fail within the first year equals 1 - 0.99635^4,
i.e. ~0.01452, i.e. roughly 4 * P(X = 365)

How did you reach your result?

Not sure, I failed my maths consistently and so asked a maths/stats
lecturer friend for a formula to calculate the probability of a
failure and 4 failures, within 1 year if the time between two failures
is given as 1.2M hours. She gave me a formula, involving integrating
from time 0 to 1 year along with some utterances about assuming some
distribution thing. But couldn't figure it out so she calculated for
me in the end pPp Any errors is likely entirely my fault since she
simply gave me what I asked for.

But I do know the final part of the equations involved 1 - 0.9xxxxxx
^4 too Pp

--
A Lost Angel, fallen from heaven
Lost in dreams, Lost in aspirations,
Lost to the world, Lost to myself

On Thu, 7 Dec 2006 20:46:00 -0500, krw wrote:
Keep in mind that the MTBF for drives only apply within the expected
life cycle of the drive, which is normally either 3 or 5 years
(usually the same as the warranty period), so any failures beyond 5
years just aren't counted at all. A lot of manufacturers also ignore
the first 90 days when they calculate the MTFB, saying that this is
not part of the Useful Life Cycle of the drive (pretty much BS, but
serves to boost MTBF numbers quite nicely).

The useful life is generally stated.

5 years ago, yes. Now it's like pulling teeth trying to get this
information.

Yes there is a bathtub curve
for fails, but the MTBF includes the head end. If there is a
serious issue with early fails, manufacturers use burn-in
techniques..

You're giving companies too much credit. I wouldn't say that most
companies, or even many companies, ignore the first 90 days. However
I suspect that if you were to dig deep enough, you could find some.
Keep in mind that there is no standard for MTBF measurements.

For hard drives it has also become common to define the duty cycle as
being 8x5 for desktop drives rather than 24x7. This change in
definitnion alone could potentially increases MTBF by a factor of 4
with no improvement in reliability.

Long story short, MTBF means ****-all for any practical purpose.

Only to the uninformed.

To be useful MTBF requires that you have a statistically relevant
sample size, it requires that you know how MTBF has been defined by
the manufacturer and it requires that your sample distribution is wide
enough that it will not be affected by variations in production runs
and of course, it requires that the MTBF was accurate in the first
place! Case in point, the imfamous IBM 75GXP drives were rated for an
MTBF of 1,000,000 hours, but their actual MTBF was SIGNIFICANTLY
lower.

The best-case scenario is that MTBF is an approximation of a statistic
based on using historical data to estimate real-world failures from
environmentally controlled failure rates. It's one of those numbers
that mostly works pretty well most of the time, but it's definitely
not the sort of thing that you want to take risks with important data
over.
--
Tony Hill
hilla underscore 20 at yahoo dot ca

On Thu, 07 Dec 2006 15:32:32 +0100, Spoon
wrote:

Ryan Godridge wrote:

Snip

Are these requirements burst or sustained transfer rates? If burst,
how much data at a time?

Sustained transfer rates.

E.g. capturing 1 minute of HD-SDI video means sequentially writing
186 MB/s for 1 minute, i.e. approximately 11 GB. Likewise, playing
out 1 minute means sequentially reading 11 GB.

So you have to have guaranteed 186 MB/S write speed for the length of
the capture. This might be more problematic than the read speed.
What figures are you seeing for sustained writes?

Are you going to be reading and writing concurrently?

No. We'll capture a stream once, then play it out over and over.

The more I look at this, the chunkier the system required seems to
get.

Chunky?

Chunky - big, powerful.