If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
Metadata Server ?
Hi,
does anybody know if it's possible to have a kind of metadata server ? I explain, with 25Tb of data and many directories of 100 000 files, when i try "ls -all" it's too long ..... I'd enough time to take a coffee So i would like to find a kind of metadata server synchronize with the nas file system. I think use a virtual distributed file system like Lustre or Gfarm. thanks in advance, Séb. |
#2
|
|||
|
|||
Metadata Server ?
On Fri, 19 Oct 2007 16:19:20 +0200, Sebastien Campion
wrote: Hi, does anybody know if it's possible to have a kind of metadata server ? I explain, with 25Tb of data and many directories of 100 000 files, when i try "ls -all" it's too long ..... I'd enough time to take a coffee So i would like to find a kind of metadata server synchronize with the nas file system. I think use a virtual distributed file system like Lustre or Gfarm. thanks in advance, Séb. If the file system supports a seperation of data and metadata, like SAMFs or Lustre, then you actually *have* to have a metadata server. If the fs does not support it then you can't have one, at least not without doing some very unsupported things. ~F |
#3
|
|||
|
|||
Metadata Server ?
Thanks, i learned more about the filesystem, it's WAFL by NetApp.
In documentation, i see "The WAFL file system store metadata in files, three files : - inode file - block-map file - inode map" But i don't know where i can find it. I'd only NFS (or CIFS) protocol enable (it's a NAS NetApp) so i don't know if it's feasible to connect Lustre on NFS. If it's ok, i will install Lustre. Séb. But i don't know where Faeandar wrote: On Fri, 19 Oct 2007 16:19:20 +0200, Sebastien Campion wrote: Hi, does anybody know if it's possible to have a kind of metadata server ? I explain, with 25Tb of data and many directories of 100 000 files, when i try "ls -all" it's too long ..... I'd enough time to take a coffee So i would like to find a kind of metadata server synchronize with the nas file system. I think use a virtual distributed file system like Lustre or Gfarm. thanks in advance, Séb. If the file system supports a seperation of data and metadata, like SAMFs or Lustre, then you actually *have* to have a metadata server. If the fs does not support it then you can't have one, at least not without doing some very unsupported things. ~F |
#4
|
|||
|
|||
Metadata Server ?
On 2007-10-22, Sebastien Campion wrote:
I'd only NFS (or CIFS) protocol enable (it's a NAS NetApp) so i don't know if it's feasible to connect Lustre on NFS. If it's ok, i will install Lustre. No, lustre is a completely different file system. It can't help your NetApp. 100.000 files in each directory is going to be painfull to list, no matter what file system you're using. Things that might help: - run a name service cache daemon (nscd) to cache uid/gid lookups - don't resolve uid/gid by using "ls -ln" - use "locate" as your "metadata server", if it contains enough metadata for your needs. - see if an fscache/cachefs can help http://people.redhat.com/~dhowells/fscache/FS-Cache.pdf - tune your NFS client http://www.netapp.com/library/tr/3183.pdf -jf |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Printer Status shows as Offline with SNMP checked on server 2003 print server | [email protected] | Printers | 0 | September 12th 07 01:35 PM |
Overhead of metadata block | [email protected] | Storage & Hardrives | 6 | June 27th 07 03:23 AM |
Seagate drive check utility reports "errors in metadatea file records, other errors critical errors in metadata.." | [email protected] | Storage (alternative) | 0 | February 3rd 06 04:48 AM |
Server Boards, SCSI, RAID, and building a server - questions | nope | Homebuilt PC's | 0 | July 6th 03 03:15 AM |