If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
Storage across multiple servers?
I am trying to find out how to do something that does not make much
sense to me. I have spoken to a few people who say they have taken multiple servers (1u in this case) and striped the drives on those 1u servers so that all servers in the group were seeing all data on all servers. Does this make sense to anyone? To put it in context, this came up during a conversation over the merits of NAS(nfs) vs SAN. The above idea was given to me as a 'cheaper' solution. I'm not really sure the 'cheaper' solution is what I want, but I was intruqued about how this is possible. Does anyone know how to do this or what this is called? It would seem it might be a 'cluster file system' but I see nothing like that when performing the usual Google searches. And...for the record, this was on Linux and possibly a *bsd. Thanks! -- JB |
#2
|
|||
|
|||
Look for Google File System. JB Orca wrote: I am trying to find out how to do something that does not make much sense to me. I have spoken to a few people who say they have taken multiple servers (1u in this case) and striped the drives on those 1u servers so that all servers in the group were seeing all data on all servers. Does this make sense to anyone? To put it in context, this came up during a conversation over the merits of NAS(nfs) vs SAN. The above idea was given to me as a 'cheaper' solution. I'm not really sure the 'cheaper' solution is what I want, but I was intruqued about how this is possible. Does anyone know how to do this or what this is called? It would seem it might be a 'cluster file system' but I see nothing like that when performing the usual Google searches. And...for the record, this was on Linux and possibly a *bsd. Thanks! -- JB |
#3
|
|||
|
|||
Yes, I have seen that, however, the Google File System is not something
someone other than Google can use, correct? I am looking for something that could be used by anyone. Many thanks. On 2004-11-22 09:57:26 -0500, Yura Pismerov said: Look for Google File System. JB Orca wrote: I am trying to find out how to do something that does not make much sense to me. I have spoken to a few people who say they have taken multiple servers (1u in this case) and striped the drives on those 1u servers so that all servers in the group were seeing all data on all servers. Does this make sense to anyone? To put it in context, this came up during a conversation over the merits of NAS(nfs) vs SAN. The above idea was given to me as a 'cheaper' solution. I'm not really sure the 'cheaper' solution is what I want, but I was intruqued about how this is possible. Does anyone know how to do this or what this is called? It would seem it might be a 'cluster file system' but I see nothing like that when performing the usual Google searches. And...for the record, this was on Linux and possibly a *bsd. Thanks! -- JB |
#4
|
|||
|
|||
|
#6
|
|||
|
|||
On Mon, 22 Nov 2004 19:54:42 -0500, JB Orca wrote:
On 2004-11-22 17:54:33 -0500, Faeandar said: On 21 Nov 2004 22:03:53 -0800, (JB Orca) wrote: I am trying to find out how to do something that does not make much sense to me. I have spoken to a few people who say they have taken multiple servers (1u in this case) and striped the drives on those 1u servers so that all servers in the group were seeing all data on all servers. Does this make sense to anyone? To put it in context, this came up during a conversation over the merits of NAS(nfs) vs SAN. The above idea was given to me as a 'cheaper' solution. I'm not really sure the 'cheaper' solution is what I want, but I was intruqued about how this is possible. Does anyone know how to do this or what this is called? It would seem it might be a 'cluster file system' but I see nothing like that when performing the usual Google searches. And...for the record, this was on Linux and possibly a *bsd. Thanks! -- JB Look at Polyserve, Ibrix, GFS, GPFS, etc. These are all software solutions for a High Performance File System. How the data is accessed is up to you, either directly from the node servers or re-shared as NFS. A couple of hardware based solutions in this space are Panasas and Acopia Networks. ~F Excellent. Thanks much. Can I assume that AFS would also fit in here as well? Thanks! No. AFS is a read-many replica file system but still only has a single writeable volume, and that resides on a single host. It's a great file system for traditional file IO and program reads in that it allows client side caching, multi-location replicas (for network performance), and transparent read failover to another replica in the event the current volume is unavailable. But it is in no way a performance file system like the ones I mentioned. Also, AFS in non-trivial and generally requires an almost-dedicated if not completely dedicated admin. ~F |
#7
|
|||
|
|||
On 2004-11-22 20:43:52 -0500, Faeandar said:
......snip.... No. AFS is a read-many replica file system but still only has a single writeable volume, and that resides on a single host. It's a great file system for traditional file IO and program reads in that it allows client side caching, multi-location replicas (for network performance), and transparent read failover to another replica in the event the current volume is unavailable. But it is in no way a performance file system like the ones I mentioned. Also, AFS in non-trivial and generally requires an almost-dedicated if not completely dedicated admin. ~F Ok. Excellent. I appreciate the info very much. What I need to accomplish is this: I have a system that will need to start with roughly 5 terabytes of storage space. It will very quickly grow to needing anywhere from 50-100 terabytes. The problem we are attempting to solve is this: what is the best option for the storage in this system? The original thought, before we realized how big it was going to get, was just a large RAID direct attach system. Then we thought about NAS or SAN, however, when I heard the talk of spanning storage space across multiple servers this seemed as though it might also be a good option. We are much more interested in the data being safe than we are in the raw speed of the devices. We can't have something _slow_ per se, however, if I have to sacrifice some transfer speed in order to have more safety for the files, that is acceptable. I am still reading about the systems mentioned and trying to figure out what would suit our needs. The idea of a 'RAID' of servers seems fantastic. If I can use the storage on 5 servers and stripe the data across them that would be great, however, I have noticed with some of the options that in order to add a new server the entire system needs to be taken down and re-configured and brought back up. That would not be possible for us as we really need to have as little downtime as humanly possible. (Don't we all!) The conversation about this is great and I really appreciate any input that can be given. Thanks much. JB |
#8
|
|||
|
|||
On Tue, 23 Nov 2004 12:09:09 -0500, JB Orca wrote:
On 2004-11-22 20:43:52 -0500, Faeandar said: .....snip.... No. AFS is a read-many replica file system but still only has a single writeable volume, and that resides on a single host. It's a great file system for traditional file IO and program reads in that it allows client side caching, multi-location replicas (for network performance), and transparent read failover to another replica in the event the current volume is unavailable. But it is in no way a performance file system like the ones I mentioned. Also, AFS in non-trivial and generally requires an almost-dedicated if not completely dedicated admin. ~F Ok. Excellent. I appreciate the info very much. What I need to accomplish is this: I have a system that will need to start with roughly 5 terabytes of storage space. It will very quickly grow to needing anywhere from 50-100 terabytes. The problem we are attempting to solve is this: what is the best option for the storage in this system? The original thought, before we realized how big it was going to get, was just a large RAID direct attach system. Then we thought about NAS or SAN, however, when I heard the talk of spanning storage space across multiple servers this seemed as though it might also be a good option. We are much more interested in the data being safe than we are in the raw speed of the devices. We can't have something _slow_ per se, however, if I have to sacrifice some transfer speed in order to have more safety for the files, that is acceptable. I am still reading about the systems mentioned and trying to figure out what would suit our needs. The idea of a 'RAID' of servers seems fantastic. If I can use the storage on 5 servers and stripe the data across them that would be great, however, I have noticed with some of the options that in order to add a new server the entire system needs to be taken down and re-configured and brought back up. That would not be possible for us as we really need to have as little downtime as humanly possible. (Don't we all!) The conversation about this is great and I really appreciate any input that can be given. Thanks much. JB First thing then forget AFS. There is a backup limitation of 8gb max per volume, this is across all backup software platforms that I am aware of. It's not an AFS limit, just fyi. You never mention what you're going to be doing with this data. Is it for a single server? multiple servers? Mult host access? Multi host write access? The technology you use really depends on the requirements of your data and users. Post a little more info on what you are trying to accomplish and we might be able to help. Storage is simply a means to an end, not the end itself. ~F |
#9
|
|||
|
|||
JB Orca wrote:
.... I have a system that will need to start with roughly 5 terabytes of storage space. It will very quickly grow to needing anywhere from 50-100 terabytes. With these kinds of numbers, you'll have lot of drives and thus drive failures will become quite common. Are you looking at using some RAID configuration to overcome this ? The problem we are attempting to solve is this: what is the best option for the storage in this system? The original thought, before we realized how big it was going to get, was just a large RAID direct attach system. Then we thought about NAS or SAN, however, when I heard the talk of spanning storage space across multiple servers this seemed as though it might also be a good option. This would be using your LAN to move data unless the server doing the I/O happens to have the target disk locally available, right ? I guess with gigabit ethernet this might not be such a problem anymore, except for processor overhead. A SAN will allow every server to use Fibre Channel to move the data, your LAN and server cpus won't be loaded nearly as much. Depending on your application load, you could save a lot on LAN switches and servers by spending more on a SAN. .... The idea of a 'RAID' of servers seems fantastic. If I can use the storage on 5 servers and stripe the data across them that would be great, however, I have noticed with some of the options that in order to add a new server the entire system needs to be taken down and re-configured and brought back up. The only reason to go this way instead of a regular SAN would be cost I guess; by using plain old scsi drives you'll cut down the cost significantly. But again my first question, do you plan on using some form of RAID (software, raid controller, raid enclosure,...) ? If you just plug in a bunch of scsi drives into a bunch of servers and start storing data on then, at a hundred terabytes worth of disks, you'll be running around shutting down hosts in order to swap out disks all day in my opinion. Arne Joris |
#10
|
|||
|
|||
On 2004-11-23 12:30:35 -0500, Faeandar said:
On Tue, 23 Nov 2004 12:09:09 -0500, JB Orca wrote: On 2004-11-22 20:43:52 -0500, Faeandar said: .....snip.... No. AFS is a read-many replica file system but still only has a single writeable volume, and that resides on a single host. It's a great file system for traditional file IO and program reads in that it allows client side caching, multi-location replicas (for network performance), and transparent read failover to another replica in the event the current volume is unavailable. But it is in no way a performance file system like the ones I mentioned. Also, AFS in non-trivial and generally requires an almost-dedicated if not completely dedicated admin. ~F Ok. Excellent. I appreciate the info very much. What I need to accomplish is this: I have a system that will need to start with roughly 5 terabytes of storage space. It will very quickly grow to needing anywhere from 50-100 terabytes. The problem we are attempting to solve is this: what is the best option for the storage in this system? The original thought, before we realized how big it was going to get, was just a large RAID direct attach system. Then we thought about NAS or SAN, however, when I heard the talk of spanning storage space across multiple servers this seemed as though it might also be a good option. We are much more interested in the data being safe than we are in the raw speed of the devices. We can't have something _slow_ per se, however, if I have to sacrifice some transfer speed in order to have more safety for the files, that is acceptable. I am still reading about the systems mentioned and trying to figure out what would suit our needs. The idea of a 'RAID' of servers seems fantastic. If I can use the storage on 5 servers and stripe the data across them that would be great, however, I have noticed with some of the options that in order to add a new server the entire system needs to be taken down and re-configured and brought back up. That would not be possible for us as we really need to have as little downtime as humanly possible. (Don't we all!) The conversation about this is great and I really appreciate any input that can be given. Thanks much. JB First thing then forget AFS. There is a backup limitation of 8gb max per volume, this is across all backup software platforms that I am aware of. It's not an AFS limit, just fyi. You never mention what you're going to be doing with this data. Is it for a single server? multiple servers? Mult host access? Multi host write access? The technology you use really depends on the requirements of your data and users. Post a little more info on what you are trying to accomplish and we might be able to help. Storage is simply a means to an end, not the end itself. ~F Good point...here's some additional info: The plan is to have multiple 'user-facing' servers that the users will interact with. Placing files, pulling files, etc. All of these front-end servers should use a shared storage system. The idea being this: if we have n number of front-end servers we can balance any load across them, as long as our shared storage system is robust enough we should be in decent shape. The majority of files being stored will be in the 5-40 meg range. We do not expect to have many over 40 megs. There will be a lot of files. So the idea is multi-host access, both read and write. "Storage is simply a means to an end, not the end itself." That is really well said. Perhaps I'm thinking about this too much.... I'm basically trying to be as thorough as possible and make sure I try to think of any possible solution before committing the time it will take to learn some new stuff and get a dev system up and running. The idea of the mutli-server file system seemed good to me for this one reason: it seemed like creating a RAID but using cheaper hardware with an easier path to adding storage. The idea of needing this much storage is a bit new to me, so I'm trying to learn as I go here. If I did a direct attached raid device, say a 2 terabyte raid, that would be great. But, when it comes time to expand the storage it seems like it would be a mess to add additional storage to that type of system, no? As I mentioned about, I'm kind of learning some of this as I go, so I appreciate that help and input. Many thanks. JB |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Enterprise Storage Management (ESM) FAQ Revision 2004/06/23 - Part 1/1 | Will Spencer | Storage & Hardrives | 0 | June 23rd 04 06:58 AM |
STORAGE SERVERS | Phil Jennings | Storage & Hardrives | 10 | May 8th 04 02:09 AM |
Enterprise Storage Management (ESM) FAQ Revision 2004/04/11 - Part 1/1 | Will Spencer | Storage & Hardrives | 0 | April 11th 04 07:24 AM |
Enterprise Storage Management (ESM) FAQ Revision 2004/02/16 - Part 1/1 | Will Spencer | Storage & Hardrives | 0 | February 16th 04 09:23 PM |
Terabyte Storage By Real-Storage | Real-Storage | Storage & Hardrives | 2 | October 23rd 03 04:18 PM |