If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
New to NAS, have idea, looking for software to do it.
I am a smalltime consultant with several clients having similar
scenarios. Typically they have less than 100 Windows workstations and anywhere from 1 to 10 servers. Some servers are virtual hosts - just getting started in the virtual world. My idea is this: If I have up to 100 PC's, often each one has many GB of free disk space on the local disk drives. Is there any software that would allow me to designate some portion of this unused disk space on all those PC's to a central server and combine them as one logical drive? I often think of it as the Borg episode on Star Trek. It seems like a waste to have all that free storage on workstations and no way to manage it as one resource. It would seem someone has thought of this and developed software to do that. Maybe iSCSI is the answer? Input appreciated. |
#2
|
|||
|
|||
New to NAS, have idea, looking for software to do it.
On Nov 16, 5:45*pm, Bob Meyers wrote:
I am a smalltime consultant with several clients having similar scenarios. Typically they have less than 100 Windows workstations and anywhere from 1 to 10 servers. Some servers are virtual hosts - just getting started in the virtual world. My idea is this: If I have up to 100 PC's, often each one has many GB of free disk space on the local disk drives. Is there any software that would allow me to designate some portion of this unused disk space on all those PC's to a central server and combine them as one logical drive? I often think of it as the Borg episode on Star Trek. It seems like a waste to have all that free storage on workstations and no way to manage it as one resource. It would seem someone has thought of this and developed software to do that. Maybe iSCSI is the answer? Certainly you can put an iSCSI host on each workstation with spare disk space, and then mount each of those LUNs on a server, and then put a volume on top of those. Basically this would be pretty trivial (if a bit tedious) to set up, the main limit you're going to run into is the maximum number of partitions you can put in a single volume. But, the big problem is the availability and reliability of the "disks" on the workstations. Not only are those workstation class disks, you're at the mercy of the workstation being up and running, and the users not doing anything bad to the slice of space dedicated to the common storage area. Some stuff like mirroring and RAID may give you the ability to survive missing workstations. And given the cost of disk space, this seems like far more hassle than its worth. More formal systems of this type are usually called "distributed storage" and there are several efforts on different OSs to implement that. But the focus is usually more on collecting the storage from a bunch of servers, rather than workstations. |
#3
|
|||
|
|||
New to NAS, have idea, looking for software to do it.
Bob Meyers wrote:
I am a smalltime consultant with several clients having similar scenarios. Typically they have less than 100 Windows workstations and anywhere from 1 to 10 servers. Some servers are virtual hosts - just getting started in the virtual world. My idea is this: If I have up to 100 PC's, often each one has many GB of free disk space on the local disk drives. Is there any software that would allow me to designate some portion of this unused disk space on all those PC's to a central server and combine them as one logical drive? I often think of it as the Borg episode on Star Trek. It seems like a waste to have all that free storage on workstations and no way to manage it as one resource. It would seem someone has thought of this and developed software to do that. Maybe iSCSI is the answer? Input appreciated. I don't know if doing something like this would be /practical/, but it might be fun! When you say "iSCSI", I guess you are thinking of having each machine host an iSCSI target serving out a section of the local hard disk. The severer would make use of these distributed chunks as volumes on a big raid drive. This is going to have a number of issues. Since you have workstations being switched on and off, or rebooted (they are windows workstations...), the iSCSI disks will go off-line. That means that you need significant redundancy in your raid setup, and are likely to spend a great deal of time doing re-builds. To avoid re-building each "drive" as they come on line again, you would want to use a raid system that supports write-intent bitmaps such as Linux md raid. Of course, more redundancy means more traffic, so this would be pretty hard on the network. Another idea would be to use some sort of distributed file system. Examples include GFS and Lustre. However, they are not designed for this sort of environment, so they will not be ideal. |
#4
|
|||
|
|||
New to NAS, have idea, looking for software to do it.
Bob Meyers wrote:
.... My idea is this: If I have up to 100 PC's, often each one has many GB of free disk space on the local disk drives. Is there any software that would allow me to designate some portion of this unused disk space on all those PC's to a central server and combine them as one logical drive? I often think of it as the Borg episode on Star Trek. It seems like a waste to have all that free storage on workstations and no way to manage it as one resource. It would seem someone has thought of this and developed software to do that. Well, yes and no. In the '90s there was a Berkeley project called 'Network of Workstations' that focused on networking commodity PCs into a high-performance compute cluster, and it included mechanisms (such as the Zebra file system, IIRC) for sharing their storage in the manner which you describe. And some more recent distributed file systems could probably be configured to use workstation storage in the manner that you describe, simply by using a portion of each workstation as one of their clustered 'servers'. But, as others have pointed out, storage is extremely inexpensive these days and part of what makes a PC 'personal' is that the owner controls it exclusively (i.e., has access to all its hardware if desired, including the option to turn it off). While there's still a reasonable case to be made for including a local disk in each PC there may not be much of a case for sharing this local storage externally: rather, a central server (or, for large installations, a central cluster of servers) can provide shared space for everyone at modest cost while reducing the load on the network significantly (since most needs will still be met by each local disk). There's plenty of software available to ensure that the central storage gets backed up consistently, which might be more of a problem with the data as distributed as you suggest. And you can use higher-quality components (e.g., ECC RAM, perhaps even 'end-to-end' checking and redundancy using something like the ZFS file system - or a more expensive commercial alternative from NetApp) to help ensure that the shared data maintains reasonable integrity. All in all, your users will likely have far fewer headaches - and this reduction in headaches will likely save more actual cash than the cost of the additional hardware. - bill |
#5
|
|||
|
|||
New to NAS, have idea, looking for software to do it.
On 2009-11-16 18:45:11 -0500, Bob Meyers said:
I am a smalltime consultant with several clients having similar scenarios. Typically they have less than 100 Windows workstations and anywhere from 1 to 10 servers. Some servers are virtual hosts - just getting started in the virtual world. My idea is this: If I have up to 100 PC's, often each one has many GB of free disk space on the local disk drives. Is there any software that would allow me to designate some portion of this unused disk space on all those PC's to a central server and combine them as one logical drive? I often think of it as the Borg episode on Star Trek. It seems like a waste to have all that free storage on workstations and no way to manage it as one resource. It would seem someone has thought of this and developed software to do that. Maybe iSCSI is the answer? Input appreciated. Stuff like this is relatively common in the virtualization world; think of the LeftHand VSA or similar. Unfortunately, most of the software that might allow you to do this in a physical world are really more oriented toward server infrastructure and not desktop infrastructure. So, is it possible? Probably. Is it recommended? Not necessarily. -- Scott |
#6
|
|||
|
|||
New to NAS, have idea, looking for software to do it.
|
#7
|
|||
|
|||
New to NAS, have idea, looking for software to do it.
On 2009³â11¿ù17ÀÏ, ¿ÀÀü8½Ã45ºÐ, Bob Meyers wrote:
I am a smalltime consultant with several clients having similar scenarios. Typically they have less than 100 Windows workstations and anywhere from 1 to 10 servers. Some servers are virtual hosts - just getting started in the virtual world. My idea is this: If I have up to 100 PC's, often each one has many GB of free disk space on the local disk drives. Is there any software that would allow me to designate some portion of this unused disk space on all those PC's to a central server and combine them as one logical drive? I often think of it as the Borg episode on Star Trek. It seems like a waste to have all that free storage on workstations and no way to manage it as one resource. It would seem someone has thought of this and developed software to do that. Maybe iSCSI is the answer? Input appreciated. Please refer to "LifeBoat" paper published in LISA'04. (http:// http://www.usenix.org/event/lisa04/t...bonkenburg.pdf) It proposes p2p-backup/restore architecture to utilize free resource of each client. Actually, it works in decentralized fashion, not centralized one you imagined, but I think it is the right choice when considering scalability problem. |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
New idea for fast bus | Jon Slaughter | General | 5 | December 23rd 08 08:49 PM |
What you think on this idea? | Bob | Homebuilt PC's | 3 | January 22nd 06 08:50 PM |
here's a powerful idea | Chris | Overclocking AMD Processors | 2 | June 4th 05 04:36 PM |
My great idea - help | Good! | Webcams | 0 | December 29th 03 08:27 AM |
Is trying this a bad idea? | ck26 | General | 4 | October 18th 03 08:07 PM |