A computer components & hardware forum. HardwareBanter

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

Go Back   Home » HardwareBanter forum » General Hardware & Peripherals » Storage & Hardrives
Site Map Home Register Authors List Search Today's Posts Mark Forums Read Web Partners

Decentralized/distributed filesystems questions



 
 
Thread Tools Display Modes
  #1  
Old April 21st 07, 07:04 PM posted to comp.sys.ibm.pc.hardware.storage comp.arch.storage
Wendell III
external usenet poster
 
Posts: 5
Default Decentralized/distributed filesystems questions

Hi all,

I am looking for a decentralized filesystem solution for non-enterprise
use (lightweight) to replace several rsync-mirrored, software-based
RAID-5 storage servers. I don't know if there's any actual solution out
there which meets my fantasy, but here is some criteria that would be
great:

- Linux compatible (obviously)
- Virtual filesystem OK, but real filesystem driver is probably (?) ideal.
- Completely automatic synchronization of data, with automatic
redundancy (like RAID), and the ability to set LEVELS of redundancy for
either the FS as a whole, or certain groups/subsets of files within it.
- Simple/automatic configuration: option to automatically allocate
storage devices, or some % of new storage devices added to a node.
überideal is hardware RAID style: pop in a drive, reboot the machine,
everything is taken care of.
- On a local level, nodes should NOT require a centralized tracker, but
rather find each other via something akin to ZeroConf/Bonjour. Once
found, a node should examine its own tables and, if known, attempt to
authenticate automatically without fuss. On a local level there should
be dynamic IP tolerance (an identifying hash should be used for
identification).
- Support for scaling across the Internet would be great.
- In which case, I want to be able to tweak the total "sync" bandwidth
used on each node as can similarly be done with rsync
- And the WHOLE filesystem should be able to be accessed from any node
which allows external connections, anywhere in the world. Chunks of
data which are not local would be transferred via the internet, to that
node, upon demand.
- NO SPECIAL CLIENT SOFTWARE SHOULD BE REQUIRED for accessing shares
(virtual filesystems presumably require a translator installed on the
node in order to provide a mount point?)

This seems to be a combination of a lot of methodologies, but I'm
obviously taking a lot of inspiration from P2P filesharing protocols
such as eDonkey and Gnutella...

Any thoughts? Anything even close to this out there?

Thanks in advance for any help -- should be a good conversation
nonetheless. :-)
-Wendell
--

 




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Lecture on Parallel Filesystems Fortuitous Technologies Storage & Hardrives 0 January 11th 07 05:41 AM
writing simultaneously to 2 or more network filesystems [email protected] Storage & Hardrives 3 June 21st 06 06:11 AM
Distributed filesystem idea/information. [email protected] Storage & Hardrives 4 May 12th 05 04:02 PM
1 dedicated drive (mirrored), or thin stripe for redo filesystems? [email protected] Storage & Hardrives 1 February 28th 05 05:03 PM
Distributed Computing for this newsgroup MikeTimbers Overclocking 1 March 2nd 04 11:51 PM


All times are GMT +1. The time now is 12:39 PM.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 HardwareBanter.
The comments are property of their posters.