If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
NAS solution, Netapp or EMC
Hi all,
I am looking for a NAS server to work over NFS for 15-20 pieces of my linux servers. These servers will be involved in heavy reading and writing (about 30Mbps up and down each server) of experimental data collected from a network of probes. I am thinking of using a NAS solution (I have got a GigE LAN already) from either NetApp (F825) or EMC (Celerra) as centralized storaged for all the servers. I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files 3) Reliability...should not crash or go down when under consistent heavy load. Any comments about their performance? p.s. sorry if this has been asked very often... |
#2
|
|||
|
|||
I had some bad experiences in an NFS environment posted to this group with
EMC posted to this newsgroup (google IP4700+galjan). I've since done installs Netapp in multiple types of environments, with no problems. It simply works. It's expensive stuff though. "darren" wrote in message ... Hi all, I am looking for a NAS server to work over NFS for 15-20 pieces of my linux servers. These servers will be involved in heavy reading and writing (about 30Mbps up and down each server) of experimental data collected from a network of probes. I am thinking of using a NAS solution (I have got a GigE LAN already) from either NetApp (F825) or EMC (Celerra) as centralized storaged for all the servers. I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files 3) Reliability...should not crash or go down when under consistent heavy load. Any comments about their performance? p.s. sorry if this has been asked very often... |
#3
|
|||
|
|||
On Thu, 16 Oct 2003 22:22:13 +0800, "darren"
wrote: Hi all, I am looking for a NAS server to work over NFS for 15-20 pieces of my linux servers. Linux has some NFS issues natively so take those into account. Run your mounts over tcp not up. huge performance difference. These servers will be involved in heavy reading and writing (about 30Mbps up and down each server) of experimental data collected from a network of probes. I am thinking of using a NAS solution (I have got a GigE LAN already) from either NetApp (F825) or EMC (Celerra) as centralized storaged for all the servers. In a NAS environment nothing beats NetApp overall. I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files For both 1 and 2 you need to understand that there is *no* server or filesystem that can speedily do what your talking about. The filer can do it but if you compare performance on any type of system this traffic pattern will always be behind. And backing up 20mil+ files is going to suck no matter what you do, unless you use something like Veritas' FlashBackup. 3) Reliability...should not crash or go down when under consistent heavy load. Filer's I've worked with (30+) have never crashed due to load. Other issues yes, but never load. Any comments about their performance? If performance is all your after you may want to look at BlueArc as well. I am a NetApp fan so this hurts, but for pure speed nothing so far is able to beat their box. They have asic's that handle the network stripping and creation of packets. Makes for much faster IO. One last piece of advise, stay clear of EMC whenever possible. Just my opinion of course.... ~F |
#4
|
|||
|
|||
I have used both EMC and NetApp NAS solution. The NetApp solution beats EMC
hands down for its simplicity, features and performance. EMC NAS is a combination of many individual h/w units (at least the ones I have used so far). To get it all work is a challenge by it self. You need a data mover (couple if you need failover) a control station (again a couple if you need redundancy) , a FC switch (OEM from brocade/mcdata) to connect to the back end storage and then the backend storage itself (clariion or symmetrix). The OS on the dmovers (DART) is a piece of crap. The CS runs redhat linux with many modified binaries/scripts to talk to the dmovers. They sell all this to you and professional services to get it setup, which would be a few months project. last year we spent 3 months to get 6 TB of NAS installed using their PS. Their dmovers have limitations on the number of volumes and size of each volumes it can host. so be ware. They will sell you more dmovers than you need. NetAppp is so simple. One filer head (2 if you need failover) a couple of disk shelves (depending on your capacity needs). Hook em up and power on. you are up and running in 20 minutes. This year we installed close to 18 TB in just under a week. Finally nothing beats Netapps on the snapshot feature. I don't think NetApp is expensive if you were to look at all the features it provides. but them EMC will drop their pants to match prices but can't do anything when it comes to performance. Good luck. -G "Paul" pgaljan_delete_between the wrote in message news:xmyjb.88985$a16.22600@lakeread01... I had some bad experiences in an NFS environment posted to this group with EMC posted to this newsgroup (google IP4700+galjan). I've since done installs Netapp in multiple types of environments, with no problems. It simply works. It's expensive stuff though. "darren" wrote in message ... Hi all, I am looking for a NAS server to work over NFS for 15-20 pieces of my linux servers. These servers will be involved in heavy reading and writing (about 30Mbps up and down each server) of experimental data collected from a network of probes. I am thinking of using a NAS solution (I have got a GigE LAN already) from either NetApp (F825) or EMC (Celerra) as centralized storaged for all the servers. I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files 3) Reliability...should not crash or go down when under consistent heavy load. Any comments about their performance? p.s. sorry if this has been asked very often... |
#5
|
|||
|
|||
Just a comment on item 2.
The Netapp WILL store the data fine. Getting it there and back is going to be another issue. Depending on how fast you want this to occur, block data requests (like SAN/FCP) are far more efficient with small file transfers. In my environment we have Billions of 1K files. Network transfers are no fun, even with extreme performance mod's. Disk to disk transfers are the only way. We're all SAN for file storeage. No NAS good perform to the level we requested. However that was a year and a half ago so there may be some improvements since our last tests. Ohh and yes, we did test Netapp filers, really nice machines but no the solution for us. Robert "darren" wrote in message ... Hi all, I am looking for a NAS server to work over NFS for 15-20 pieces of my linux servers. These servers will be involved in heavy reading and writing (about 30Mbps up and down each server) of experimental data collected from a network of probes. I am thinking of using a NAS solution (I have got a GigE LAN already) from either NetApp (F825) or EMC (Celerra) as centralized storaged for all the servers. I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files 3) Reliability...should not crash or go down when under consistent heavy load. Any comments about their performance? p.s. sorry if this has been asked very often... |
#6
|
|||
|
|||
"Faeandar" wrote in message ... On Thu, 16 Oct 2003 22:22:13 +0800, "darren" wrote: .... I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files For both 1 and 2 you need to understand that there is *no* server or filesystem that can speedily do what your talking about. That's kind of a subjective matter, isn't it? Unless you expect miracles, that is. IIUC, NetApp's file system should do a pretty good job of handling bulk small-file writes: it certainly captures relevant meta-data updates in stable RAM, and may capture the data itself there as well - such that it can be written to disk in large, efficient chunks. And fast access to large numbers of small files is first a matter of having the directory tree mostly in RAM to make look-ups fast (at least assuming a competent directory structure that doesn't require linear searches to find files within a directory): one would expect a NetApp server to do this if it was configured with sufficient memory, and you might pick up a few cache hits on recently-accessed file contents as well. Beyond that, experimental file systems have tried shuffling files on disk based on access patterns to allow a single physical disk access to pick up the file(s) that are likely to be needed next, but I don't know of any commercial file system that does this (though Windows - gasp! - reportedly has some such mechanism to expedite application loading). - bill |
#7
|
|||
|
|||
to be needed next, but I don't know of any commercial file system that does
this (though Windows - gasp! - reportedly has some such mechanism to expedite application loading). Starting with XP, Windows uses this prefetch for both app binaries and for kernel modules - the latter speeds up booting a lot. -- Maxim Shatskih, Windows DDK MVP StorageCraft Corporation http://www.storagecraft.com |
#8
|
|||
|
|||
In article ,
Maxim S. Shatskih wrote: to be needed next, but I don't know of any commercial file system that does this (though Windows - gasp! - reportedly has some such mechanism to expedite application loading). Starting with XP, Windows uses this prefetch for both app binaries and for kernel modules - the latter speeds up booting a lot. There's lots of file systems that prefetch blocks and objects... UNIX has prefetched blocks for sequentially accessed files since the late '70s at least. That doesn't sound like what Bill was talking about... could you go into more detail, perhaps? -- I've seen things you people can't imagine. Chimneysweeps on fire over the roofs of London. I've watched kite-strings glitter in the sun at Hyde Park Gate. All these things will be lost in time, like chalk-paintings in the rain. `-_-' Time for your nap. | Peter da Silva | Har du kramat din varg, idag? 'U` |
#9
|
|||
|
|||
Purchasing a new unit from NetApp can be expensive, however you can
purchase a unit with transferable protocol licenses. Do a search on google for ' used netapp' there are some very reputable dealers that have completely refurbished units with transferable licensing. "Sto RageŠ" wrote in message y.com... I have used both EMC and NetApp NAS solution. The NetApp solution beats EMC hands down for its simplicity, features and performance. EMC NAS is a combination of many individual h/w units (at least the ones I have used so far). To get it all work is a challenge by it self. You need a data mover (couple if you need failover) a control station (again a couple if you need redundancy) , a FC switch (OEM from brocade/mcdata) to connect to the back end storage and then the backend storage itself (clariion or symmetrix). The OS on the dmovers (DART) is a piece of crap. The CS runs redhat linux with many modified binaries/scripts to talk to the dmovers. They sell all this to you and professional services to get it setup, which would be a few months project. last year we spent 3 months to get 6 TB of NAS installed using their PS. Their dmovers have limitations on the number of volumes and size of each volumes it can host. so be ware. They will sell you more dmovers than you need. NetAppp is so simple. One filer head (2 if you need failover) a couple of disk shelves (depending on your capacity needs). Hook em up and power on. you are up and running in 20 minutes. This year we installed close to 18 TB in just under a week. Finally nothing beats Netapps on the snapshot feature. I don't think NetApp is expensive if you were to look at all the features it provides. but them EMC will drop their pants to match prices but can't do anything when it comes to performance. Good luck. -G "Paul" pgaljan_delete_between the wrote in message news:xmyjb.88985$a16.22600@lakeread01... I had some bad experiences in an NFS environment posted to this group with EMC posted to this newsgroup (google IP4700+galjan). I've since done installs Netapp in multiple types of environments, with no problems. It simply works. It's expensive stuff though. "darren" wrote in message ... Hi all, I am looking for a NAS server to work over NFS for 15-20 pieces of my linux servers. These servers will be involved in heavy reading and writing (about 30Mbps up and down each server) of experimental data collected from a network of probes. I am thinking of using a NAS solution (I have got a GigE LAN already) from either NetApp (F825) or EMC (Celerra) as centralized storaged for all the servers. I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files 3) Reliability...should not crash or go down when under consistent heavy load. Any comments about their performance? p.s. sorry if this has been asked very often... |
#10
|
|||
|
|||
I am concerned about the following: 1) Fast read and write performance for consistant stream of small files 2) Ability to store LARGE ( 10million per 200GB) amount of small files For both 1 and 2 you need to understand that there is *no* server or filesystem that can speedily do what your talking about. That's kind of a subjective matter, isn't it? Unless you expect miracles, that is. IIUC, NetApp's file system should do a pretty good job of handling bulk small-file writes: it certainly captures relevant meta-data updates in stable RAM, and may capture the data itself there as well - such that it can be written to disk in large, efficient chunks. And fast access to large numbers of small files is first a matter of having the directory tree mostly in RAM to make look-ups fast (at least assuming a competent directory structure that doesn't require linear searches to find files within a directory): one would expect a NetApp server to do this if it was configured with sufficient memory, and you might pick up a few cache hits on recently-accessed file contents as well. Beyond that, experimental file systems have tried shuffling files on disk based on access patterns to allow a single physical disk access to pick up the file(s) that are likely to be needed next, but I don't know of any commercial file system that does this (though Windows - gasp! - reportedly has some such mechanism to expedite application loading). Let me clarify. For the majority of data access patterns WAFL handles small files very well, better than most in fact. The reason is that the clients generally are not accessing all bazillion files at the same time. My sentiment was primarily in regards to backups, which I did not mention so how would anyone know? Sorry about that. For general small file access for clients/apps WAFL is great. For full file system scans (like backup) nothing handles it well, except something like FlashBackup which converts the inode map into a bitmap. Kinda cool actually but I digress. ~F |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
WL 400 Tuning and fix ( Solution customer care N° 4203 ) | Free | Compaq Computers | 0 | November 12th 04 02:01 AM |
Howto install the drivers for mobility Radeon 9200 on Compaq X1000 with Windows Server 2003 !!! (solution) | Razvan | Ati Videocards | 1 | August 12th 04 07:13 AM |
Headphone Solution for High End Rig | Gamer | Ati Videocards | 2 | December 18th 03 03:29 AM |
Soyo and ATI Compatiblity Problem in XP ----- SOLUTION FOUND!!!! | Hoss.N.Pfeffer | Ati Videocards | 1 | October 9th 03 09:43 PM |
Artec or Memorex 48U Scanner Solution | BMac | Scanners | 0 | September 25th 03 07:06 PM |