If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
|
Thread Tools | Display Modes |
|
#1
|
|||
|
|||
NAS solution, Netapp or EMC
Hi all,
I am looking for a NAS server to work over NFS for 15-20 pieces of my linux servers. These servers will be involved in heavy reading and writing (about 30Mbps up and down each server) of experimental data collected from a network of probes. I am thinking of using a NAS solution (I have got a GigE LAN already) from either NetApp (F825) or EMC (Celerra) as centralized storaged for all the servers. I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files 3) Reliability...should not crash or go down when under consistent heavy load. Any comments about their performance? p.s. sorry if this has been asked very often... |
#2
|
|||
|
|||
I had some bad experiences in an NFS environment posted to this group with
EMC posted to this newsgroup (google IP4700+galjan). I've since done installs Netapp in multiple types of environments, with no problems. It simply works. It's expensive stuff though. "darren" wrote in message ... Hi all, I am looking for a NAS server to work over NFS for 15-20 pieces of my linux servers. These servers will be involved in heavy reading and writing (about 30Mbps up and down each server) of experimental data collected from a network of probes. I am thinking of using a NAS solution (I have got a GigE LAN already) from either NetApp (F825) or EMC (Celerra) as centralized storaged for all the servers. I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files 3) Reliability...should not crash or go down when under consistent heavy load. Any comments about their performance? p.s. sorry if this has been asked very often... |
#3
|
|||
|
|||
I have used both EMC and NetApp NAS solution. The NetApp solution beats EMC
hands down for its simplicity, features and performance. EMC NAS is a combination of many individual h/w units (at least the ones I have used so far). To get it all work is a challenge by it self. You need a data mover (couple if you need failover) a control station (again a couple if you need redundancy) , a FC switch (OEM from brocade/mcdata) to connect to the back end storage and then the backend storage itself (clariion or symmetrix). The OS on the dmovers (DART) is a piece of crap. The CS runs redhat linux with many modified binaries/scripts to talk to the dmovers. They sell all this to you and professional services to get it setup, which would be a few months project. last year we spent 3 months to get 6 TB of NAS installed using their PS. Their dmovers have limitations on the number of volumes and size of each volumes it can host. so be ware. They will sell you more dmovers than you need. NetAppp is so simple. One filer head (2 if you need failover) a couple of disk shelves (depending on your capacity needs). Hook em up and power on. you are up and running in 20 minutes. This year we installed close to 18 TB in just under a week. Finally nothing beats Netapps on the snapshot feature. I don't think NetApp is expensive if you were to look at all the features it provides. but them EMC will drop their pants to match prices but can't do anything when it comes to performance. Good luck. -G "Paul" pgaljan_delete_between the wrote in message news:xmyjb.88985$a16.22600@lakeread01... I had some bad experiences in an NFS environment posted to this group with EMC posted to this newsgroup (google IP4700+galjan). I've since done installs Netapp in multiple types of environments, with no problems. It simply works. It's expensive stuff though. "darren" wrote in message ... Hi all, I am looking for a NAS server to work over NFS for 15-20 pieces of my linux servers. These servers will be involved in heavy reading and writing (about 30Mbps up and down each server) of experimental data collected from a network of probes. I am thinking of using a NAS solution (I have got a GigE LAN already) from either NetApp (F825) or EMC (Celerra) as centralized storaged for all the servers. I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files 3) Reliability...should not crash or go down when under consistent heavy load. Any comments about their performance? p.s. sorry if this has been asked very often... |
#4
|
|||
|
|||
Purchasing a new unit from NetApp can be expensive, however you can
purchase a unit with transferable protocol licenses. Do a search on google for ' used netapp' there are some very reputable dealers that have completely refurbished units with transferable licensing. "Sto RageŠ" wrote in message y.com... I have used both EMC and NetApp NAS solution. The NetApp solution beats EMC hands down for its simplicity, features and performance. EMC NAS is a combination of many individual h/w units (at least the ones I have used so far). To get it all work is a challenge by it self. You need a data mover (couple if you need failover) a control station (again a couple if you need redundancy) , a FC switch (OEM from brocade/mcdata) to connect to the back end storage and then the backend storage itself (clariion or symmetrix). The OS on the dmovers (DART) is a piece of crap. The CS runs redhat linux with many modified binaries/scripts to talk to the dmovers. They sell all this to you and professional services to get it setup, which would be a few months project. last year we spent 3 months to get 6 TB of NAS installed using their PS. Their dmovers have limitations on the number of volumes and size of each volumes it can host. so be ware. They will sell you more dmovers than you need. NetAppp is so simple. One filer head (2 if you need failover) a couple of disk shelves (depending on your capacity needs). Hook em up and power on. you are up and running in 20 minutes. This year we installed close to 18 TB in just under a week. Finally nothing beats Netapps on the snapshot feature. I don't think NetApp is expensive if you were to look at all the features it provides. but them EMC will drop their pants to match prices but can't do anything when it comes to performance. Good luck. -G "Paul" pgaljan_delete_between the wrote in message news:xmyjb.88985$a16.22600@lakeread01... I had some bad experiences in an NFS environment posted to this group with EMC posted to this newsgroup (google IP4700+galjan). I've since done installs Netapp in multiple types of environments, with no problems. It simply works. It's expensive stuff though. "darren" wrote in message ... Hi all, I am looking for a NAS server to work over NFS for 15-20 pieces of my linux servers. These servers will be involved in heavy reading and writing (about 30Mbps up and down each server) of experimental data collected from a network of probes. I am thinking of using a NAS solution (I have got a GigE LAN already) from either NetApp (F825) or EMC (Celerra) as centralized storaged for all the servers. I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files 3) Reliability...should not crash or go down when under consistent heavy load. Any comments about their performance? p.s. sorry if this has been asked very often... |
#5
|
|||
|
|||
On Thu, 16 Oct 2003 22:22:13 +0800, "darren"
wrote: Hi all, I am looking for a NAS server to work over NFS for 15-20 pieces of my linux servers. Linux has some NFS issues natively so take those into account. Run your mounts over tcp not up. huge performance difference. These servers will be involved in heavy reading and writing (about 30Mbps up and down each server) of experimental data collected from a network of probes. I am thinking of using a NAS solution (I have got a GigE LAN already) from either NetApp (F825) or EMC (Celerra) as centralized storaged for all the servers. In a NAS environment nothing beats NetApp overall. I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files For both 1 and 2 you need to understand that there is *no* server or filesystem that can speedily do what your talking about. The filer can do it but if you compare performance on any type of system this traffic pattern will always be behind. And backing up 20mil+ files is going to suck no matter what you do, unless you use something like Veritas' FlashBackup. 3) Reliability...should not crash or go down when under consistent heavy load. Filer's I've worked with (30+) have never crashed due to load. Other issues yes, but never load. Any comments about their performance? If performance is all your after you may want to look at BlueArc as well. I am a NetApp fan so this hurts, but for pure speed nothing so far is able to beat their box. They have asic's that handle the network stripping and creation of packets. Makes for much faster IO. One last piece of advise, stay clear of EMC whenever possible. Just my opinion of course.... ~F |
#6
|
|||
|
|||
"Faeandar" wrote in message ... On Thu, 16 Oct 2003 22:22:13 +0800, "darren" wrote: .... I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files For both 1 and 2 you need to understand that there is *no* server or filesystem that can speedily do what your talking about. That's kind of a subjective matter, isn't it? Unless you expect miracles, that is. IIUC, NetApp's file system should do a pretty good job of handling bulk small-file writes: it certainly captures relevant meta-data updates in stable RAM, and may capture the data itself there as well - such that it can be written to disk in large, efficient chunks. And fast access to large numbers of small files is first a matter of having the directory tree mostly in RAM to make look-ups fast (at least assuming a competent directory structure that doesn't require linear searches to find files within a directory): one would expect a NetApp server to do this if it was configured with sufficient memory, and you might pick up a few cache hits on recently-accessed file contents as well. Beyond that, experimental file systems have tried shuffling files on disk based on access patterns to allow a single physical disk access to pick up the file(s) that are likely to be needed next, but I don't know of any commercial file system that does this (though Windows - gasp! - reportedly has some such mechanism to expedite application loading). - bill |
#7
|
|||
|
|||
to be needed next, but I don't know of any commercial file system that does
this (though Windows - gasp! - reportedly has some such mechanism to expedite application loading). Starting with XP, Windows uses this prefetch for both app binaries and for kernel modules - the latter speeds up booting a lot. -- Maxim Shatskih, Windows DDK MVP StorageCraft Corporation http://www.storagecraft.com |
#8
|
|||
|
|||
In article ,
Maxim S. Shatskih wrote: to be needed next, but I don't know of any commercial file system that does this (though Windows - gasp! - reportedly has some such mechanism to expedite application loading). Starting with XP, Windows uses this prefetch for both app binaries and for kernel modules - the latter speeds up booting a lot. There's lots of file systems that prefetch blocks and objects... UNIX has prefetched blocks for sequentially accessed files since the late '70s at least. That doesn't sound like what Bill was talking about... could you go into more detail, perhaps? -- I've seen things you people can't imagine. Chimneysweeps on fire over the roofs of London. I've watched kite-strings glitter in the sun at Hyde Park Gate. All these things will be lost in time, like chalk-paintings in the rain. `-_-' Time for your nap. | Peter da Silva | Har du kramat din varg, idag? 'U` |
#9
|
|||
|
|||
"Peter da Silva" wrote in message ... In article , Maxim S. Shatskih wrote: to be needed next, but I don't know of any commercial file system that does this (though Windows - gasp! - reportedly has some such mechanism to expedite application loading). Starting with XP, Windows uses this prefetch for both app binaries and for kernel modules - the latter speeds up booting a lot. There's lots of file systems that prefetch blocks and objects... UNIX has prefetched blocks for sequentially accessed files since the late '70s at least. That doesn't sound like what Bill was talking about... could you go into more detail, perhaps? The mechanism that I was referring to involves analyzing access patterns at run-time and reorganizing the files on the disk to allow the disk's (or possibly OS's) prefetch behavior, if enabled, to fetch what is likely to be the next data requested. It's only slightly analogous to the prefetch mechanisms (sequential, strided, etc.) traditionally employed within a single file (with no attempt to reorganize the on-disk placement based on run-time analysis of access patterns). IIRC John Wilkes' group at HP did some work in this area, possibly in connection with the research that later became the AutoRAID product (and might even have productized the mechanism). And, as noted, Windows supports it in more limited (boot-time and/or application-load) contexts where repetitive behavior is the norm and access-pattern-capture is a one-time rather than on-going overhead. - bill |
#10
|
|||
|
|||
Starting with XP, Windows uses this prefetch for both app binaries and for
kernel modules - the latter speeds up booting a lot. There's lots of file systems that prefetch blocks and objects... UNIX has prefetched blocks for sequentially accessed files since the late '70s at least. Prefetch for sequentially accessed files is in NT since version 3.1, and is governed by a flag to CreateFile - you can turn it off if you want. You can also turn off the cache usage for a file at all, which makes the IO to this file being the same as IO to the raw partition. Thus, in NT, there is no need ever in software using raw partitions for data storage - use the noncached file instead. If such a file is not fragmented - which is usually true, since it is created as one huge chunk - then the FS overhead is around 20 machine instructrions (or so) per read/write operation. This simplifies the database administration greatly. No need in repartitioning for this purpose. That doesn't sound like what Bill was talking about... could you go Prefetch by reading the contents (or the metadata) of several files in a single IO operation. Prefetching pages of the executable image by formerly gathered stats on how it used to be executed. -- Maxim Shatskih, Windows DDK MVP StorageCraft Corporation http://www.storagecraft.com |
|
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
WL 400 Tuning and fix ( Solution customer care N° 4203 ) | Free | Compaq Computers | 0 | November 12th 04 01:01 AM |
Howto install the drivers for mobility Radeon 9200 on Compaq X1000 with Windows Server 2003 !!! (solution) | Razvan | Ati Videocards | 1 | August 12th 04 07:13 AM |
Headphone Solution for High End Rig | Gamer | Ati Videocards | 2 | December 18th 03 02:29 AM |
Soyo and ATI Compatiblity Problem in XP ----- SOLUTION FOUND!!!! | Hoss.N.Pfeffer | Ati Videocards | 1 | October 9th 03 09:43 PM |
Artec or Memorex 48U Scanner Solution | BMac | Scanners | 0 | September 25th 03 07:06 PM |