Shared Storage?

Hi,

I am going to be setting up a couple of linux servers in the near future
and these servers will have ever increasing storage needs..

One server will be a web/file server and the other will be a mail server..

I don't want to keep moving the data to bigger drives, I want to have
the servers use a shared storage facility that runs RAID and that
capacity can be increased as needed.. Preferably using ATA or SATA
drives to keep the cost down..

I have looked briefly at things like SAN's but these are far to
expensive for our limited budget..

I have also thought of NAS but I am not sure the access over
NFS/ethernet will be fast enough.. Even gigabit ethernet may become a
bottleneck.. I guess I could put a gig ethernet link between each server
and the storage server so each server has a dedicated gig ethernet
connection..

Anyway I guess you get the idea about what I am trying to do..

Has anyone setup something similar that didn't cost massive amounts of
money??

Anyone got any pointers to sites or products that would point me in the
right direction?

Thanks

WipeOut wrote:
Hi,

I am going to be setting up a couple of linux servers in the near
future and these servers will have ever increasing storage needs..

One server will be a web/file server and the other will be a mail
server..

I don't want to keep moving the data to bigger drives, I want to have
the servers use a shared storage facility that runs RAID and that
capacity can be increased as needed.. Preferably using ATA or SATA
drives to keep the cost down..

I have looked briefly at things like SAN's but these are far to
expensive for our limited budget..

What is your budget? You maybe ruling at FC attached storage (no need for a
switch) too quickly. A dual ported FC array with software on each host to
create a shared filesystem might be in your budget, but it's hard to tell.

Also, it would be easier to decide if your worries about NAS performance
where justified if we had some idea of what your performance requirements
are.

That said, if you don't think NAS is fast enough, and SAN is outside your
budget, then you've pretty much shot your options ;-)


--
Nik Simpson

Nik Simpson wrote:
WipeOut wrote:

Hi,

I am going to be setting up a couple of linux servers in the near
future and these servers will have ever increasing storage needs..

One server will be a web/file server and the other will be a mail
server..

I don't want to keep moving the data to bigger drives, I want to have
the servers use a shared storage facility that runs RAID and that
capacity can be increased as needed.. Preferably using ATA or SATA
drives to keep the cost down..

I have looked briefly at things like SAN's but these are far to
expensive for our limited budget..


What is your budget? You maybe ruling at FC attached storage (no need for a
switch) too quickly. A dual ported FC array with software on each host to
create a shared filesystem might be in your budget, but it's hard to tell.

I ruled out FC for a few reasons, I only saw SCSI drives and alone they
are to expensive per GB compared to PATA/SATA, The FC HBA's are very
steep as well probably costing as much as the whole server, and then
there is the switch and the array..


Also, it would be easier to decide if your worries about NAS performance
where justified if we had some idea of what your performance requirements
are.

I know this is a problem area becasue I have not got anything to base my
requirements on becasue its a new system..


That said, if you don't think NAS is fast enough, and SAN is outside your
budget, then you've pretty much shot your options ;-)


Yea, I know.. thats the problem..

I think NAS on a dedicated GigE netowrk and if possible a dedicated link
between each server and the NAS is probably the way to go..

For the NAS its self I will probably just use a Linux server and an 8 or
12 channel SATA RAID controller with 3 or 4 drives to start and then add
on as needed using LVM for the storage management.. I don't think I will
be able to do it much cheaper than that..

Thanks for you comments..

WipeOut wrote:
Nik Simpson wrote:
WipeOut wrote:

Hi,

I am going to be setting up a couple of linux servers in the near
future and these servers will have ever increasing storage needs..

One server will be a web/file server and the other will be a mail
server..

I don't want to keep moving the data to bigger drives, I want to
have the servers use a shared storage facility that runs RAID and
that capacity can be increased as needed.. Preferably using ATA or
SATA drives to keep the cost down..

I have looked briefly at things like SAN's but these are far to
expensive for our limited budget..


What is your budget? You maybe ruling at FC attached storage (no
need for a switch) too quickly. A dual ported FC array with software
on each host to create a shared filesystem might be in your budget,
but it's hard to tell.

I ruled out FC for a few reasons, I only saw SCSI drives and alone
they are to expensive per GB compared to PATA/SATA, The FC HBA's are
very steep as well probably costing as much as the whole server, and
then there is the switch and the array..

You need to look a bit harder, there are a number of SATA/PATA storage
systems with FC interfaces on the market, take a look at www.nexsan.com for
an example. Also, if you only have two systems, then you don't need a
switch, both systems can be direct attached to the array. Yes, the FC HBAs
are still quite pricey, but overall a SATA based FC array and HBAs for two
servers isn't that bad, unless your budget is *very* limited.

Another possible solution that would get rid of the FC costs is iSCSI, again
there are a number of iSCSI arrays that use SATA/PATA drives.



I think NAS on a dedicated GigE netowrk and if possible a dedicated
link between each server and the NAS is probably the way to go..

For the NAS its self I will probably just use a Linux server and an 8
or 12 channel SATA RAID controller with 3 or 4 drives to start and
then add on as needed using LVM for the storage management.. I don't
think I will be able to do it much cheaper than that..


My guess from the configuration you've described is that the I/O demands for
the two systems are not equal, I'd bet that the mailserver needs I/O
performance much more than the webserver, in which case, why not direct
attach the storage to the mail server and share some of its capacity with
the webserver?

As I said previously, giving us an idea of your budget, i.e. how much you
are prepared/able to spend would help in suggesting possible solutions.
--
Nik Simpson

Nik Simpson wrote:

You need to look a bit harder, there are a number of SATA/PATA storage
systems with FC interfaces on the market, take a look at www.nexsan.com for
an example. Also, if you only have two systems, then you don't need a
switch, both systems can be direct attached to the array. Yes, the FC HBAs
are still quite pricey, but overall a SATA based FC array and HBAs for two
servers isn't that bad, unless your budget is *very* limited.

Another possible solution that would get rid of the FC costs is iSCSI, again
there are a number of iSCSI arrays that use SATA/PATA drives.

Thanks for the link, I had not found them.. probably becasue I was
looking for suppliers in the UK..


My guess from the configuration you've described is that the I/O demands for
the two systems are not equal, I'd bet that the mailserver needs I/O
performance much more than the webserver, in which case, why not direct
attach the storage to the mail server and share some of its capacity with
the webserver?

Yes the email server will probably have a higher I/O requirement, but
the web server will also be working quite hard in terms of I/O becasue
its not just a standard website its a web based application that
incorporates file storage..

I was thinking of building the MySQL server with the larger storage
capacity and then sharing it with the other two similar to what you have
suggested..

The perfect solution would be a NAS or SAN or iSCSI system that I cauld
start with 3 drives and then add a 4th and 5th and 6th as I needed them
being able to easily extend the array and filesystem.. hotswap would be
nice but not really critical..


As I said previously, giving us an idea of your budget, i.e. how much you
are prepared/able to spend would help in suggesting possible solutions.

Budget I would estimate at around £1500 for the starting system with
three 250GB or 300GB drives and expansion capacity to do to 6 or 8
drives.. So I think NAS is about the only option at that budget and even
that may be pushing it..

WipeOut wrote:
Nik Simpson wrote:

You need to look a bit harder, there are a number of SATA/PATA
storage systems with FC interfaces on the market, take a look at
www.nexsan.com for an example. Also, if you only have two systems,
then you don't need a switch, both systems can be direct attached to
the array. Yes, the FC HBAs are still quite pricey, but overall a
SATA based FC array and HBAs for two servers isn't that bad, unless
your budget is *very* limited.

Another possible solution that would get rid of the FC costs is
iSCSI, again there are a number of iSCSI arrays that use SATA/PATA
drives.

Thanks for the link, I had not found them.. probably becasue I was
looking for suppliers in the UK..


I think you'll find that Nexsan is actually a UK company, though they
portray themselves as a US company these days, European HQ is:


Nexsan Technologies, Ltd.
Units 33-35 Parker Centre, Mansfield Road
Derby, DE21 4SZ U.K.



Budget I would estimate at around £1500 for the starting system with
three 250GB or 300GB drives and expansion capacity to do to 6 or 8
drives.. So I think NAS is about the only option at that budget and
even that may be pushing it..

Yup, 1500 quid isn't going to go far, I'd see if you can find a 1U server
chassis with four hot swap drive bays, there are a bunch of them around,
then add motherboard & controller to taste.


--
Nik Simpson

Nik Simpson wrote:

Budget I would estimate at around £1500 for the starting system with
three 250GB or 300GB drives and expansion capacity to do to 6 or 8
drives.. So I think NAS is about the only option at that budget and
even that may be pushing it..


Yup, 1500 quid isn't going to go far, I'd see if you can find a 1U server
chassis with four hot swap drive bays, there are a bunch of them around,
then add motherboard & controller to taste.



Yea, Thats what I thought as well.. Maybe it will work as a solution for
12-18 months by which time hopefully the company is off the ground and
making some money at which time I can look at the more enterprise level
options..

Looks like the 3ware controllers are about the best option for what I am
after and like you said a 1U or 2U server.. The rackmount chassis on
their own are quite pricey and thats before adding all the internals..

Thanks for your thoughts and comments..

WipeOut wrote:

I am going to be setting up a couple of linux servers in the near future
and these servers will have ever increasing storage needs..

One server will be a web/file server and the other will be a mail server..

I don't want to keep moving the data to bigger drives, I want to have
the servers use a shared storage facility that runs RAID and that
capacity can be increased as needed.. Preferably using ATA or SATA
drives to keep the cost down..

I have looked briefly at things like SAN's but these are far to
expensive for our limited budget..

I have also thought of NAS but I am not sure the access over
NFS/ethernet will be fast enough.. Even gigabit ethernet may become a
bottleneck.. I guess I could put a gig ethernet link between each server
and the storage server so each server has a dedicated gig ethernet
connection..

Anyway I guess you get the idea about what I am trying to do..

Has anyone setup something similar that didn't cost massive amounts of
money??

Oracle is doing interesting things with firewire (cheap) and cluster
file systems.

Anyone got any pointers to sites or products that would point me in the
right direction?

I would say explore the possibilities of firewire. It is low end, but I
assume you can mirror over controllers (linux software RAID is generally
OK and very fast).

Shared devices and software RAID is not OK I think. But I would think
Oracle has looked into that.

Or just bite and do not share the storage. Or do mirroring over an
internal and a firewire drive.


Thomas

"WipeOut" wrote in message ...
Hi,

I am going to be setting up a couple of linux servers in the near future
and these servers will have ever increasing storage needs..

One server will be a web/file server and the other will be a mail server..

I don't want to keep moving the data to bigger drives, I want to have
the servers use a shared storage facility that runs RAID and that
capacity can be increased as needed..

While Nik has offered good suggestions, he doesn't seem to have ever asked
you *why* you want to share the storage between the two servers rather than
simply use directly-attached storage with each one. That would give you two
separate DAS arrays, each of which could be expanded according to need - and
almost certainly the overall least-expensive solution, if there's no actual
need to share the storage.

- bill

Bill Todd wrote:
"WipeOut" wrote in message
...
Hi,

I am going to be setting up a couple of linux servers in the near
future and these servers will have ever increasing storage needs..

One server will be a web/file server and the other will be a mail
server..

I don't want to keep moving the data to bigger drives, I want to have
the servers use a shared storage facility that runs RAID and that
capacity can be increased as needed..

While Nik has offered good suggestions, he doesn't seem to have ever
asked you *why* you want to share the storage between the two servers
rather than simply use directly-attached storage with each one. That
would give you two separate DAS arrays, each of which could be
expanded according to need - and almost certainly the overall
least-expensive solution, if there's no actual need to share the
storage.

Certainly a good point, truly shared storage does add additional cost and
complexity to the solution. My bet is that the shared requirement is that he
doesn't really know what the final storage requirements will be and which
server will end up requiring the majority of the storage, in which case
starting with a shared storage architecture does make life easier in some
ways. But given the very small budget indicated in a later post, I'd be
inclined to used DAS as well, and probably wouldn't use external arrays at
all, just pick a server chassis with plenty of SATA/PATA hotswap drive bays
which would give ample room for expansion for some time (you can 2-3U
chassis that hold 3-4TB.) At the end of that time it might be time to relook
at the issue, but given the lifespan of the typical server these days, its
probably not going to happen until he's in the position of having to replace
the original servers anyway.


--
Nik Simpson