If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
Decentralized/distributed filesystems questions
Hi all,
I am looking for a decentralized filesystem solution for non-enterprise use (lightweight) to replace several rsync-mirrored, software-based RAID-5 storage servers. I don't know if there's any actual solution out there which meets my fantasy, but here is some criteria that would be great: - Linux compatible (obviously) - Virtual filesystem OK, but real filesystem driver is probably (?) ideal. - Completely automatic synchronization of data, with automatic redundancy (like RAID), and the ability to set LEVELS of redundancy for either the FS as a whole, or certain groups/subsets of files within it. - Simple/automatic configuration: option to automatically allocate storage devices, or some % of new storage devices added to a node. überideal is hardware RAID style: pop in a drive, reboot the machine, everything is taken care of. - On a local level, nodes should NOT require a centralized tracker, but rather find each other via something akin to ZeroConf/Bonjour. Once found, a node should examine its own tables and, if known, attempt to authenticate automatically without fuss. On a local level there should be dynamic IP tolerance (an identifying hash should be used for identification). - Support for scaling across the Internet would be great. - In which case, I want to be able to tweak the total "sync" bandwidth used on each node as can similarly be done with rsync - And the WHOLE filesystem should be able to be accessed from any node which allows external connections, anywhere in the world. Chunks of data which are not local would be transferred via the internet, to that node, upon demand. - NO SPECIAL CLIENT SOFTWARE SHOULD BE REQUIRED for accessing shares (virtual filesystems presumably require a translator installed on the node in order to provide a mount point?) This seems to be a combination of a lot of methodologies, but I'm obviously taking a lot of inspiration from P2P filesharing protocols such as eDonkey and Gnutella... Any thoughts? Anything even close to this out there? Thanks in advance for any help -- should be a good conversation nonetheless. :-) -Wendell -- |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Decentralized/distributed filesystems questions | Wendell III | Storage & Hardrives | 0 | April 21st 07 07:04 PM |
Lecture on Parallel Filesystems | Fortuitous Technologies | Storage & Hardrives | 0 | January 11th 07 05:41 AM |
writing simultaneously to 2 or more network filesystems | [email protected] | Storage & Hardrives | 3 | June 21st 06 06:11 AM |
Distributed filesystem idea/information. | [email protected] | Storage & Hardrives | 4 | May 12th 05 04:02 PM |
Distributed Computing for this newsgroup | MikeTimbers | Overclocking | 1 | March 2nd 04 11:51 PM |