what raid can do upgrades?

Where do I get a raid system that can handle upgrades?
If I have 2 100gb disks giving 100gb mirrored space,
these are the most likely upgrade scenarios:

Add a 200gb disk, yielding 200gb of mirrored space (each half of the 200gb
disk mirrored by one of the 100gb disks). If a 100gb disk later fails,
buy a 300gb disk and get 300gb mirrored space (300gb on one side,
100gb+200gb on the other side).

Add a 300gb disk, yielding 200gb of mirrored space + 100gb unmirrored.

Replace a failed 100gb disk with a 300gb disk, yielding 100gb mirrored +
200gb unmirrored.

Replace a failed 100gb disk with 200gb and 300gb disks, yielding 300gb
mirrored space (1/3 of the 300gb disk mirrored by the remaining 100gb disk).

Recognize that I only have 30gb of actively changing important data that
really needs to be mirrored and the rest can be used for static, temporary,
and slowly changing data, so when my 100gb starts filling up I can increase
available disk space from 100gb mirrored to 30gb mirrored + 2x70gb
unmirrored = 170gb total without buying any new hardware - if the raid
system can only do its thing on the 30gb partitions and leave the rest for
the operating system to handle normally.

If I have 2 100gb disks mirrored, the chances I'm going to want to replace
a failed 100gb disk with another 100gb disk are virtually nil unless
one fails very prematurely, and the chances I'm going to want to buy
2 more disks to get any more space are also virtually nil.
I haven't heard of a case that can hold an unlimited number of disks;
having to buy pairs reduces its lifetime by as much as a factor of 2.
Not only that, but by the time a disk fails, which should be =5 years if
they're much good, 100gb disks may not be available any more, so I would be
stuck in a dead end with a system that can't handle more than one size.

The prevalent fixation on performance above all other criteria is misguided
for ordinary users. I for one will not be able to detect a difference of a
millisecond here and a millisecond there, but I definitely notice being put
into a proprietary straightjacket that is incapable of handling any of the
very most likely upgrade scenarios.

I understand linux can do these but what if I need to be able to run
windows too?
Is raid software available to allow win9x and win2k and linux to boot and use
the same raid configuration using standard disks without a raid controller?
Does anyone out there have real-world experience using VMware or Connectix
Virtual PC or WINE or Lindows to run windows software on linux?

What about an industry standard for raid disk formats, so different
operating systems and software and hardware systems are interchangeable,
if a raid card fails and the vendor has gone out of the business I can
buy a different controller and keep running, or I can start with raid
software and later buy a hardware controller, plug it in, and be running
in a few minutes?

--
delete NOSPAM to reply by email

On Wed, 25 Feb 2004 14:28:33 -0800, Walter Epp
wrote:

Where do I get a raid system that can handle upgrades?
If I have 2 100gb disks giving 100gb mirrored space,
these are the most likely upgrade scenarios:

Add a 200gb disk, yielding 200gb of mirrored space (each half of the 200gb
disk mirrored by one of the 100gb disks). If a 100gb disk later fails,
buy a 300gb disk and get 300gb mirrored space (300gb on one side,
100gb+200gb on the other side).

Add a 300gb disk, yielding 200gb of mirrored space + 100gb unmirrored.

Replace a failed 100gb disk with a 300gb disk, yielding 100gb mirrored +
200gb unmirrored.

Replace a failed 100gb disk with 200gb and 300gb disks, yielding 300gb
mirrored space (1/3 of the 300gb disk mirrored by the remaining 100gb disk).

Recognize that I only have 30gb of actively changing important data that
really needs to be mirrored and the rest can be used for static, temporary,
and slowly changing data, so when my 100gb starts filling up I can increase
available disk space from 100gb mirrored to 30gb mirrored + 2x70gb
unmirrored = 170gb total without buying any new hardware - if the raid
system can only do its thing on the 30gb partitions and leave the rest for
the operating system to handle normally.

If I have 2 100gb disks mirrored, the chances I'm going to want to replace
a failed 100gb disk with another 100gb disk are virtually nil unless
one fails very prematurely, and the chances I'm going to want to buy
2 more disks to get any more space are also virtually nil.
I haven't heard of a case that can hold an unlimited number of disks;
having to buy pairs reduces its lifetime by as much as a factor of 2.
Not only that, but by the time a disk fails, which should be =5 years if
they're much good, 100gb disks may not be available any more, so I would be
stuck in a dead end with a system that can't handle more than one size.

The prevalent fixation on performance above all other criteria is misguided
for ordinary users. I for one will not be able to detect a difference of a
millisecond here and a millisecond there, but I definitely notice being put
into a proprietary straightjacket that is incapable of handling any of the
very most likely upgrade scenarios.

I think you're kind of overthinking this. No hardware is all things
to all people and ideal for all upgrade paths. You kind of just need
to get what you need now and replace it when the upgrades become too
diffacult or costly. Generally with RAID your best bet is to use
identical drives, and if you can, with identical firmwares.

That being said I recommend picking up a Mylex U160 card on ebay with
64Megs or ram or more and start with 3x 36 gig drives. You can easily
expand the array as needed with MORE technology and can get good
performance and useful raid levels not possible with cheaper configs.
If the case is crammed or PSU isn't beefy, or you expect the array to
grow in the next year or two, pick up a decent external box for the
drives. Some ppl will tell you otherwise but I recommend enabling
spindle sync in many situations so SCA2 drives can be the easiest way
of experimenting with this.


I understand linux can do these but what if I need to be able to run
windows too?
Is raid software available to allow win9x and win2k and linux to boot and use
the same raid configuration using standard disks without a raid controller?
Does anyone out there have real-world experience using VMware or Connectix
Virtual PC or WINE or Lindows to run windows software on linux?

I tend to use virtual disks for virtual machines so I have no
experience with what you are asking. Really any limitations or extra
steps should be related to the guest OS and not VMware, et al. AFAIK
access to the devices is not direct but instead goes through the host
OS.


What about an industry standard for raid disk formats, so different
operating systems and software and hardware systems are interchangeable,
if a raid card fails and the vendor has gone out of the business I can
buy a different controller and keep running, or I can start with raid
software and later buy a hardware controller, plug it in, and be running
in a few minutes?

In general this low-level stuff is card specific or manufacturer
specific. That is why if you run a business on a RAIDed server you
should have a spare controller and drives on site. Also you need a
backup of the card's config so you can swap in a new card without
having to reinitializing the array. Just remember there are many
reasons why RAID does not take the place of backup. One of them is
that if your controller or multiple drives die you can always replace
it with something different and restore the data from tape, etc.

Previously Walter Epp wrote:
Where do I get a raid system that can handle upgrades?
If I have 2 100gb disks giving 100gb mirrored space,
these are the most likely upgrade scenarios:

[Scenarios snipped]

No RAID can do this. But there is a way around, at least with Linux
software RAID-1: Put the array in degraded mode by removing one disk
(i.e. "raidsetfaulty disk2", "raidhotremove disk2". Then add your
new, large one and create a degraded RAID-1 array on it (i.e. just
one disk in the array). Copy your stuff over and verify. Then erase
the old disk, and add it to the new array. If you want to add 200GB to
2*100GB, instead just add the new disk, create 200GB degraded RAID-1,
copy over. Delete the old array, combine the two 100GB into one 200GB
RAID-0 array and then add this RAID-0 array as second disk the the new
RAID-1 array. After resync you will have a 200GB non-degraded RAID1
array.

If you don't want to use full disks, Linux does software-RAID on
partitions as well. (In fact I use it only on partitions...) This
allows you to use only part of a new disk or disks of different sizes.

The prevalent fixation on performance above all other criteria is misguided
for ordinary users.

Yes, indeed. Very much so.

I for one will not be able to detect a difference of a millisecond
here and a millisecond there, but I definitely notice being put into
a proprietary straightjacket that is incapable of handling any of
the very most likely upgrade scenarios.

What can I say? "+1, Insightful", I think...

And in addition with hardware RAID, you need to keep a spare controller
around, since your _controller_ may die.

I understand linux can do these but what if I need to be able to run
windows too?

Difficult. Not compatible.

Is raid software available to allow win9x and win2k and linux to
boot and use the same raid configuration using standard disks
without a raid controller?

Do. Different admin-info on the disks. (I.e. icompatible by choice.)

Does anyone out there have real-world
experience using VMware or Connectix Virtual PC or WINE or Lindows
to run windows software on linux?

Vmware works fine for Office and other business-type stuff. Not very
usable for real-time like video-conferencing or gaming. At least last
time I tried (which was a very long time ago.) Might work well
if your CPU needs are significantly lower than what the host computer
provides.

My solution is to have dual-boot and some unraided space for Windows
at the beginning of the first disk, and manual backup to the raided
space under Linux. But since I have most of my daya and work on Linux
and use MS only for gaming, this is no problem at all.

What about an industry standard for raid disk formats, so different
operating systems and software and hardware systems are
interchangeable, if a raid card fails and the vendor has gone out of
the business I can buy a different controller and keep running, or I
can start with raid software and later buy a hardware controller,
plug it in, and be running in a few minutes?

That would be nice. And no lock-in to a particular vendor anymore.
But there is no such thing. The data-part is not the problem. That
is standard (at least for RAID-1, simply 1:1 copy). The problem is
the additional block with the RAID administrative information.

Linux e.g. puts this at the end, so you can use a Partition
from a RAID-1 array also directly, unraided. Other vendors do
things differently, and, as far as I can see, intentionally
different from each other.

Arno
--
For email address: lastname AT tik DOT ee DOT ethz DOT ch
GnuPG: ID:1E25338F FP:0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F
"The more corrupt the state, the more numerous the laws" - Tacitus

Walter Epp wrote:

Where do I get a raid system that can handle upgrades?
If I have 2 100gb disks giving 100gb mirrored space,
these are the most likely upgrade scenarios:

Add a 200gb disk, yielding 200gb of mirrored space (each half of the 200gb
disk mirrored by one of the 100gb disks). If a 100gb disk later fails,
buy a 300gb disk and get 300gb mirrored space (300gb on one side,
100gb+200gb on the other side).

Add a 300gb disk, yielding 200gb of mirrored space + 100gb unmirrored.

Replace a failed 100gb disk with a 300gb disk, yielding 100gb mirrored +
200gb unmirrored.

Replace a failed 100gb disk with 200gb and 300gb disks, yielding 300gb
mirrored space (1/3 of the 300gb disk mirrored by the remaining 100gb
disk).

Recognize that I only have 30gb of actively changing important data that
really needs to be mirrored and the rest can be used for static,
temporary, and slowly changing data, so when my 100gb starts filling up I
can increase available disk space from 100gb mirrored to 30gb mirrored +
2x70gb unmirrored = 170gb total without buying any new hardware - if the
raid system can only do its thing on the 30gb partitions and leave the
rest for the operating system to handle normally.

If I have 2 100gb disks mirrored, the chances I'm going to want to replace
a failed 100gb disk with another 100gb disk are virtually nil unless
one fails very prematurely, and the chances I'm going to want to buy
2 more disks to get any more space are also virtually nil.
I haven't heard of a case that can hold an unlimited number of disks;
having to buy pairs reduces its lifetime by as much as a factor of 2.
Not only that, but by the time a disk fails, which should be =5 years if
they're much good, 100gb disks may not be available any more, so I would
be stuck in a dead end with a system that can't handle more than one size.

The prevalent fixation on performance above all other criteria is
misguided for ordinary users.

The fixation with RAID is on reliability, not performance, in most cases.
The one major exception is the RAID 0 setups used for video capture but
even they're going by the wayside as drive performance increases.

I for one will not be able to detect a difference of
a millisecond here and a millisecond there, but I definitely notice being
put into a proprietary straightjacket that is incapable of handling any of
the very most likely upgrade scenarios.

Generally the way a RAID is upgraded if the original disk model is not
available is to use a larger disk and only use part of the capacity. Most
administrators are not that desperate for disk space that they'd try to
reconfigure a working 100 gig RAID with 3 100 gig disks into a 200 gig RAID
with 2 100 gig disks and a 200. If they needed 200 gig then they'd put
together a 200 gig RAID and divert the 100 gig drives to another use.

I understand linux can do these but what if I need to be able to run
windows too?
Is raid software available to allow win9x and win2k and linux to boot and
use the same raid configuration using standard disks without a raid
controller? Does anyone out there have real-world experience using VMware
or Connectix Virtual PC or WINE or Lindows to run windows software on
linux?

What about an industry standard for raid disk formats, so different
operating systems and software and hardware systems are interchangeable,
if a raid card fails and the vendor has gone out of the business I can
buy a different controller and keep running, or I can start with raid
software and later buy a hardware controller, plug it in, and be running
in a few minutes?

Your best bet is to find software that will allow you to configure the RAID
the way you want to and then set that RAID up in a server and run your
multiboot system on a different machine accessing the data via LAN.

Virtual PC for Intel boxen runs only on Windows--it is a Microsoft
product--there used to be a version for OS/2 but it has been discontinued.
WINE works for some things, not for others. Lindows just a Linux
distribution--it uses WINE to run what Windows apps it can handle and
doesn't install it by default. VMware works very nicely however you have
to be aware that some of the hardware is necessarily emulated, and that you
have Windows or whatever guest your running on it sharing the whole CPU
with a different OS--there's a performance hit that is greater or lesser
depending on exactly what you're doing but it's always there.

--
delete NOSPAM to reply by email

--
--John
Reply to jclarke at ae tee tee global dot net
(was jclarke at eye bee em dot net)