Looking to build storage of about 3TB -- recommendations?

Hi All

For my research I need large storage for satellite data files. I am
looking to build storage for about 1TB redundant array and 2TB
volatile array. I have looked around at snap servers or other SAN
solutions. I also have looked at scsi solutions but don't think that
scsi is economicable. What I am proposing is to build 2 arrays of SATA
drives with the following setup:

1 3u supermicro chasis with 15 hotswappable sata bays
2 3ware RAID controller (for ex 9550SX-8LP)
12 WD SATA HDS (400GB)

RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB

This will all be hooked to dual xeon chips with and internal boot/sys
mount. Will probably use win2k3 server due to software requirements.

The second array is for temporary storage and can crash and we will not
loose much except the need to download again. The first array needs to
be fault tolerant. We envision swapping out drives in array 2 as
larger drives become available.

My questions a

+ Can anyone propose reasons NOT to do this?
+ Are the better solutions?
+ NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
MUCH more that I have seen

thanks
daben

daben wrote:
Hi All

For my research I need large storage for satellite data files. I am
looking to build storage for about 1TB redundant array and 2TB
volatile array. I have looked around at snap servers or other SAN
solutions. I also have looked at scsi solutions but don't think that
scsi is economicable. What I am proposing is to build 2 arrays of SATA
drives with the following setup:

1 3u supermicro chasis with 15 hotswappable sata bays
2 3ware RAID controller (for ex 9550SX-8LP)
12 WD SATA HDS (400GB)

RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB

This will all be hooked to dual xeon chips with and internal boot/sys
mount. Will probably use win2k3 server due to software requirements.


The second array is for temporary storage and can crash and we will not
loose much except the need to download again. The first array needs to
be fault tolerant. We envision swapping out drives in array 2 as
larger drives become available.

My questions a

+ Can anyone propose reasons NOT to do this?

+ Are the better solutions?
+ NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
MUCH more that I have seen

thanks
daben

Nor sure I understand how you're going to connect the server(s) to the
array. Fibre channel could add significantly to the cost. iSCSI could
create a 1Gb/sec bottleneck, file serving as in CIFS would probably be
even slower, but you don't mention performance reqs.

Also, how are you going to back up array 1?

Another point of concern is that recovery from a disk error on array 1
could take hours or days. I would go with raid 6 and with RAID enhanced
SATA drives (WD and Seagate have them).

In article .com,
Spindle wrote:

daben wrote:
Hi All

For my research I need large storage for satellite data files. I am
looking to build storage for about 1TB redundant array and 2TB
volatile array. I have looked around at snap servers or other SAN
solutions. I also have looked at scsi solutions but don't think that
scsi is economicable. What I am proposing is to build 2 arrays of SATA
drives with the following setup:

1 3u supermicro chasis with 15 hotswappable sata bays
2 3ware RAID controller (for ex 9550SX-8LP)
12 WD SATA HDS (400GB)

RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB

This will all be hooked to dual xeon chips with and internal boot/sys
mount. Will probably use win2k3 server due to software requirements.


The second array is for temporary storage and can crash and we will not
loose much except the need to download again. The first array needs to
be fault tolerant. We envision swapping out drives in array 2 as
larger drives become available.

My questions a

+ Can anyone propose reasons NOT to do this?

+ Are the better solutions?
+ NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
MUCH more that I have seen

thanks
daben

Nor sure I understand how you're going to connect the server(s) to the
array. Fibre channel could add significantly to the cost. iSCSI could
create a 1Gb/sec bottleneck, file serving as in CIFS would probably be
even slower, but you don't mention performance reqs.


Is there a comodity raid product that allows me to expand the array by
poping larger drives in, one at a time and leting the raid recovery
work?

What's the status of Infiniband?

http://en.wikipedia.org/wiki/Infiniband


--
a d y k e s @ p a n i x . c o m

Don't blame me. I voted for Gore.

This storage is for server that it is connected to .. it would not be
network storage. In other words, all the processintg and data access
will be on the machine. It will not be a file server in the normal
sense. This is kinda a special case of just needing a lotof storage
for local processing.

Back up will be accomplished with a lot of tapes .

In article . com,
daben wrote:
This storage is for server that it is connected to .. it would not be
network storage. In other words, all the processintg and data access
will be on the machine. It will not be a file server in the normal
sense. This is kinda a special case of just needing a lotof storage
for local processing.

Back up will be accomplished with a lot of tapes .



It's possible you can do what you want with dfs, part of the server
ntfs file system if you are loooking to RAID just to aggregate disks
it might be acceptable.





--
a d y k e s @ p a n i x . c o m

Don't blame me. I voted for Gore.

daben wrote:

Hi All

For my research I need large storage for satellite data files. I am
looking to build storage for about 1TB redundant array and 2TB
volatile array. I have looked around at snap servers or other SAN
solutions. I also have looked at scsi solutions but don't think that
scsi is economicable. What I am proposing is to build 2 arrays of SATA
drives with the following setup:

1 3u supermicro chasis with 15 hotswappable sata bays
2 3ware RAID controller (for ex 9550SX-8LP)
12 WD SATA HDS (400GB)

RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB

This will all be hooked to dual xeon chips with and internal boot/sys
mount. Will probably use win2k3 server due to software requirements.

The second array is for temporary storage and can crash and we will not
loose much except the need to download again. The first array needs to
be fault tolerant. We envision swapping out drives in array 2 as
larger drives become available.

My questions a

+ Can anyone propose reasons NOT to do this?
+ Are the better solutions?
+ NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
MUCH more that I have seen


Hi Daben,

I have created similar servers with similar equipment (3Ware,
Supermicro,Xeon's and Opterons). My preference is Opterons due to
better performance for the money, at least for our ocean models but as
far as the storage goes what you have should work very well. The only
other concern might be the power supply size for that many drives. I
have 11 drives with dual opterons in a Supermicro 933 case and the
triple-redundant 600 watt PS. It works fine.

Good luck,

Steve

Hi Steve

Thanks for your input. Our latest thinking is to go with 2+ servers.
We too have found that the AMD chips are far faster than the Xeon chips
but I have some doubts about the AMD chipsets and their reliability
over long periods of time. Maybe you can comment on this. Therefore
our plan is as follows:

+ Storage server as above with the Xeon chips (we already have them
from a previous purchase).
+ 1U dual AMD Opteron 275 server with 2GB RAM, 1 sys disk

We would anticipate the ability to add more 1U servers to increase our
computing power over time. I don't think that Blade servers or fiber
is right for our applications or budgets.

Comments?

thanks
daben

Al Dykes wrote:

Is there a comodity raid product that allows me to expand the array by
poping larger drives in, one at a time and leting the raid recovery
work?

Infortrend has that. But it is a dangerous method: drive failure may
kill you.

That said these are rather affordable so why not buy two...



Thomas

daben wrote:

Hi Steve

Thanks for your input. Our latest thinking is to go with 2+ servers.
We too have found that the AMD chips are far faster than the Xeon chips
but I have some doubts about the AMD chipsets and their reliability
over long periods of time. Maybe you can comment on this. Therefore
our plan is as follows:


We have had some for close to two years now and haven't had an issue. I
think if you go with a reputable brand you should be fine. We have been
using Tyan with good results although I am glad that I didn't start
using Opterons a little earlier because there were some problems with
some early Tyan boards. At this point I think things are pretty well
established and safe.

+ Storage server as above with the Xeon chips (we already have them
from a previous purchase).
+ 1U dual AMD Opteron 275 server with 2GB RAM, 1 sys disk

We would anticipate the ability to add more 1U servers to increase our
computing power over time. I don't think that Blade servers or fiber
is right for our applications or budgets.


This is similar to what I have done. I started out with a 2 TB storage
server with two 1.6 Ghz Opterons. We use it to run models on too with
good results. It does fine doing both storage and computing. Then we
started adding 1U dual-Opterons and when we needed more storage we
bought a 6.4 TB RAID array and connected it to one of the 1U compute
servers via SCSI. All volumes are NFS mounted to all machines and
performance is acceptable with Gigabit ethernet, even without Jumbo
frames. I originally wanted to set up a SAN with GFS or GPFS instead of
NFS but the FibreChannel hardware was too expensive. As we add compute
servers maybe the NFS performance will become an issue but I'll deal
with that as it comes.

Good luck,

Steve

Be a little careful here on the Opteron vs. Xeon speed comparisons. In
raw compute speeds the Opteron is definately faster. But the Intel
chipsets support a much higher performance in I/O throughput.

So consider your application. If your system has a PCI based RAID
controller, and multiple Gbit or 10gig NICs, then you could be easily
saturating the I/O bandwidth of an Opteron based platform. If you are
doing a OS based RAID, then you probably need all the CPU speed you can
get, so the Opteron will yield the best performance. A hardware based
RAID solution won't need that kind of CPU power, you will be
bottlenecked on the disk subsystem and the CPU will be idle a lot of
the time.

I don't know what your complete appliaction entails, but keep in mind
that CPU speed is always the best thing to optimize.

-Jim

daben wrote:
Hi Steve

Thanks for your input. Our latest thinking is to go with 2+ servers.
We too have found that the AMD chips are far faster than the Xeon chips
but I have some doubts about the AMD chipsets and their reliability
over long periods of time. Maybe you can comment on this. Therefore
our plan is as follows:

+ Storage server as above with the Xeon chips (we already have them
from a previous purchase).
+ 1U dual AMD Opteron 275 server with 2GB RAM, 1 sys disk

We would anticipate the ability to add more 1U servers to increase our
computing power over time. I don't think that Blade servers or fiber
is right for our applications or budgets.

Comments?

thanks
daben