A computer components & hardware forum. HardwareBanter

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

Go Back   Home » HardwareBanter forum » General Hardware & Peripherals » Storage & Hardrives
Site Map Home Register Authors List Search Today's Posts Mark Forums Read Web Partners

NAS solution, Netapp or EMC



 
 
Thread Tools Display Modes
  #1  
Old October 16th 03, 03:22 PM
darren
external usenet poster
 
Posts: n/a
Default NAS solution, Netapp or EMC

Hi all,

I am looking for a NAS server to work over NFS for 15-20 pieces of my linux
servers.

These servers will be involved in heavy reading and writing (about 30Mbps up
and down each server) of experimental data collected from a network of
probes.

I am thinking of using a NAS solution (I have got a GigE LAN already) from
either NetApp (F825) or EMC (Celerra) as centralized storaged for all the
servers.

I am concerned about the following:

1) Fast read and write performance for consistant stream of small files

2) Ability to store LARGE ( 10million per 200GB) amount of small files

3) Reliability...should not crash or go down when under consistent

heavy load.

Any comments about their performance?

p.s. sorry if this has been asked very often...


  #2  
Old October 16th 03, 03:57 PM
Paul
external usenet poster
 
Posts: n/a
Default

I had some bad experiences in an NFS environment posted to this group with
EMC posted to this newsgroup (google IP4700+galjan). I've since done
installs Netapp in multiple types of environments, with no problems. It
simply works. It's expensive stuff though.

"darren" wrote in message
...
Hi all,

I am looking for a NAS server to work over NFS for 15-20 pieces of my

linux
servers.

These servers will be involved in heavy reading and writing (about 30Mbps

up
and down each server) of experimental data collected from a network of
probes.

I am thinking of using a NAS solution (I have got a GigE LAN already) from
either NetApp (F825) or EMC (Celerra) as centralized storaged for all the
servers.

I am concerned about the following:

1) Fast read and write performance for consistant stream of small files

2) Ability to store LARGE ( 10million per 200GB) amount of small files

3) Reliability...should not crash or go down when under consistent

heavy load.

Any comments about their performance?

p.s. sorry if this has been asked very often...




  #3  
Old October 16th 03, 07:55 PM
Faeandar
external usenet poster
 
Posts: n/a
Default

On Thu, 16 Oct 2003 22:22:13 +0800, "darren"
wrote:

Hi all,

I am looking for a NAS server to work over NFS for 15-20 pieces of my linux
servers.


Linux has some NFS issues natively so take those into account. Run
your mounts over tcp not up. huge performance difference.


These servers will be involved in heavy reading and writing (about 30Mbps up
and down each server) of experimental data collected from a network of
probes.

I am thinking of using a NAS solution (I have got a GigE LAN already) from
either NetApp (F825) or EMC (Celerra) as centralized storaged for all the
servers.


In a NAS environment nothing beats NetApp overall.


I am concerned about the following:

1) Fast read and write performance for consistant stream of small files

2) Ability to store LARGE ( 10million per 200GB) amount of small files


For both 1 and 2 you need to understand that there is *no* server or
filesystem that can speedily do what your talking about. The filer
can do it but if you compare performance on any type of system this
traffic pattern will always be behind. And backing up 20mil+ files is
going to suck no matter what you do, unless you use something like
Veritas' FlashBackup.


3) Reliability...should not crash or go down when under consistent

heavy load.


Filer's I've worked with (30+) have never crashed due to load. Other
issues yes, but never load.


Any comments about their performance?

If performance is all your after you may want to look at BlueArc as
well. I am a NetApp fan so this hurts, but for pure speed nothing so
far is able to beat their box. They have asic's that handle the
network stripping and creation of packets. Makes for much faster IO.

One last piece of advise, stay clear of EMC whenever possible. Just
my opinion of course....

~F

  #4  
Old October 16th 03, 09:12 PM
Sto RageŠ
external usenet poster
 
Posts: n/a
Default

I have used both EMC and NetApp NAS solution. The NetApp solution beats EMC
hands down for its simplicity, features and performance.

EMC NAS is a combination of many individual h/w units (at least the ones I
have used so far). To get it all work is a challenge by it self.
You need a data mover (couple if you need failover) a control station (again
a couple if you need redundancy) , a FC switch (OEM from brocade/mcdata) to
connect to the back end storage and then the backend storage itself
(clariion or symmetrix). The OS on the dmovers (DART) is a piece of crap.
The CS runs redhat linux with many modified binaries/scripts to talk to the
dmovers.
They sell all this to you and professional services to get it setup, which
would be a few months project. last year we spent 3 months to get 6 TB of
NAS installed using their PS. Their dmovers have limitations on the number
of volumes and size of each volumes it can host. so be ware. They will sell
you more dmovers than you need.

NetAppp is so simple. One filer head (2 if you need failover) a couple of
disk shelves (depending on your capacity needs). Hook em up and power on.
you are up and running in 20 minutes. This year we installed close to 18 TB
in just under a week.

Finally nothing beats Netapps on the snapshot feature.

I don't think NetApp is expensive if you were to look at all the features it
provides. but them EMC will drop their pants to match prices but can't do
anything when it comes to performance.

Good luck.
-G


"Paul" pgaljan_delete_between the wrote
in message news:xmyjb.88985$a16.22600@lakeread01...
I had some bad experiences in an NFS environment posted to this group with
EMC posted to this newsgroup (google IP4700+galjan). I've since done
installs Netapp in multiple types of environments, with no problems. It
simply works. It's expensive stuff though.

"darren" wrote in message
...
Hi all,

I am looking for a NAS server to work over NFS for 15-20 pieces of my

linux
servers.

These servers will be involved in heavy reading and writing (about

30Mbps
up
and down each server) of experimental data collected from a network of
probes.

I am thinking of using a NAS solution (I have got a GigE LAN already)

from
either NetApp (F825) or EMC (Celerra) as centralized storaged for all

the
servers.

I am concerned about the following:

1) Fast read and write performance for consistant stream of small files

2) Ability to store LARGE ( 10million per 200GB) amount of small files

3) Reliability...should not crash or go down when under consistent

heavy load.

Any comments about their performance?

p.s. sorry if this has been asked very often...






  #5  
Old October 17th 03, 12:22 AM
Robert
external usenet poster
 
Posts: n/a
Default

Just a comment on item 2.
The Netapp WILL store the data fine. Getting it there and back is
going to be another issue. Depending on how fast you want this to
occur, block data requests (like SAN/FCP) are far more efficient with
small file transfers. In my environment we have Billions of 1K files.
Network transfers are no fun, even with extreme performance mod's.
Disk to disk transfers are the only way. We're all SAN for file
storeage. No NAS good perform to the level we requested. However that
was a year and a half ago so there may be some improvements since our
last tests.

Ohh and yes, we did test Netapp filers, really nice machines but no
the solution for us.

Robert


"darren" wrote in message ...
Hi all,

I am looking for a NAS server to work over NFS for 15-20 pieces of my linux
servers.

These servers will be involved in heavy reading and writing (about 30Mbps up
and down each server) of experimental data collected from a network of
probes.

I am thinking of using a NAS solution (I have got a GigE LAN already) from
either NetApp (F825) or EMC (Celerra) as centralized storaged for all the
servers.

I am concerned about the following:

1) Fast read and write performance for consistant stream of small files

2) Ability to store LARGE ( 10million per 200GB) amount of small files

3) Reliability...should not crash or go down when under consistent

heavy load.

Any comments about their performance?

p.s. sorry if this has been asked very often...

  #6  
Old October 17th 03, 11:52 AM
Bill Todd
external usenet poster
 
Posts: n/a
Default


"Faeandar" wrote in message
...
On Thu, 16 Oct 2003 22:22:13 +0800, "darren"
wrote:


....

I am concerned about the following:

1) Fast read and write performance for consistant stream of small files

2) Ability to store LARGE ( 10million per 200GB) amount of small files


For both 1 and 2 you need to understand that there is *no* server or
filesystem that can speedily do what your talking about.


That's kind of a subjective matter, isn't it? Unless you expect miracles,
that is.

IIUC, NetApp's file system should do a pretty good job of handling bulk
small-file writes: it certainly captures relevant meta-data updates in
stable RAM, and may capture the data itself there as well - such that it can
be written to disk in large, efficient chunks.

And fast access to large numbers of small files is first a matter of having
the directory tree mostly in RAM to make look-ups fast (at least assuming a
competent directory structure that doesn't require linear searches to find
files within a directory): one would expect a NetApp server to do this if
it was configured with sufficient memory, and you might pick up a few cache
hits on recently-accessed file contents as well. Beyond that, experimental
file systems have tried shuffling files on disk based on access patterns to
allow a single physical disk access to pick up the file(s) that are likely
to be needed next, but I don't know of any commercial file system that does
this (though Windows - gasp! - reportedly has some such mechanism to
expedite application loading).

- bill



  #7  
Old October 17th 03, 12:57 PM
Maxim S. Shatskih
external usenet poster
 
Posts: n/a
Default

to be needed next, but I don't know of any commercial file system that does
this (though Windows - gasp! - reportedly has some such mechanism to
expedite application loading).


Starting with XP, Windows uses this prefetch for both app binaries and for
kernel modules - the latter speeds up booting a lot.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation

http://www.storagecraft.com


  #8  
Old October 17th 03, 08:02 PM
Peter da Silva
external usenet poster
 
Posts: n/a
Default

In article ,
Maxim S. Shatskih wrote:
to be needed next, but I don't know of any commercial file system that does
this (though Windows - gasp! - reportedly has some such mechanism to
expedite application loading).


Starting with XP, Windows uses this prefetch for both app binaries and for
kernel modules - the latter speeds up booting a lot.


There's lots of file systems that prefetch blocks and objects... UNIX has
prefetched blocks for sequentially accessed files since the late '70s at
least. That doesn't sound like what Bill was talking about... could you go
into more detail, perhaps?

--
I've seen things you people can't imagine. Chimneysweeps on fire over the roofs
of London. I've watched kite-strings glitter in the sun at Hyde Park Gate. All
these things will be lost in time, like chalk-paintings in the rain. `-_-'
Time for your nap. | Peter da Silva | Har du kramat din varg, idag? 'U`
  #9  
Old October 17th 03, 09:16 PM
grey
external usenet poster
 
Posts: n/a
Default

Purchasing a new unit from NetApp can be expensive, however you can
purchase a unit with transferable protocol licenses. Do a search on
google for ' used netapp' there are some very reputable dealers that
have completely refurbished units with transferable licensing.



"Sto RageŠ" wrote in message y.com...
I have used both EMC and NetApp NAS solution. The NetApp solution beats EMC
hands down for its simplicity, features and performance.

EMC NAS is a combination of many individual h/w units (at least the ones I
have used so far). To get it all work is a challenge by it self.
You need a data mover (couple if you need failover) a control station (again
a couple if you need redundancy) , a FC switch (OEM from brocade/mcdata) to
connect to the back end storage and then the backend storage itself
(clariion or symmetrix). The OS on the dmovers (DART) is a piece of crap.
The CS runs redhat linux with many modified binaries/scripts to talk to the
dmovers.
They sell all this to you and professional services to get it setup, which
would be a few months project. last year we spent 3 months to get 6 TB of
NAS installed using their PS. Their dmovers have limitations on the number
of volumes and size of each volumes it can host. so be ware. They will sell
you more dmovers than you need.

NetAppp is so simple. One filer head (2 if you need failover) a couple of
disk shelves (depending on your capacity needs). Hook em up and power on.
you are up and running in 20 minutes. This year we installed close to 18 TB
in just under a week.

Finally nothing beats Netapps on the snapshot feature.

I don't think NetApp is expensive if you were to look at all the features it
provides. but them EMC will drop their pants to match prices but can't do
anything when it comes to performance.

Good luck.
-G


"Paul" pgaljan_delete_between the wrote
in message news:xmyjb.88985$a16.22600@lakeread01...
I had some bad experiences in an NFS environment posted to this group with
EMC posted to this newsgroup (google IP4700+galjan). I've since done
installs Netapp in multiple types of environments, with no problems. It
simply works. It's expensive stuff though.

"darren" wrote in message
...
Hi all,

I am looking for a NAS server to work over NFS for 15-20 pieces of my

linux
servers.

These servers will be involved in heavy reading and writing (about

30Mbps
up
and down each server) of experimental data collected from a network of
probes.

I am thinking of using a NAS solution (I have got a GigE LAN already)

from
either NetApp (F825) or EMC (Celerra) as centralized storaged for all

the
servers.

I am concerned about the following:

1) Fast read and write performance for consistant stream of small files

2) Ability to store LARGE ( 10million per 200GB) amount of small files

3) Reliability...should not crash or go down when under consistent

heavy load.

Any comments about their performance?

p.s. sorry if this has been asked very often...




  #10  
Old October 17th 03, 10:05 PM
Faeandar
external usenet poster
 
Posts: n/a
Default


I am concerned about the following:

1) Fast read and write performance for consistant stream of small files

2) Ability to store LARGE ( 10million per 200GB) amount of small files


For both 1 and 2 you need to understand that there is *no* server or
filesystem that can speedily do what your talking about.


That's kind of a subjective matter, isn't it? Unless you expect miracles,
that is.

IIUC, NetApp's file system should do a pretty good job of handling bulk
small-file writes: it certainly captures relevant meta-data updates in
stable RAM, and may capture the data itself there as well - such that it can
be written to disk in large, efficient chunks.

And fast access to large numbers of small files is first a matter of having
the directory tree mostly in RAM to make look-ups fast (at least assuming a
competent directory structure that doesn't require linear searches to find
files within a directory): one would expect a NetApp server to do this if
it was configured with sufficient memory, and you might pick up a few cache
hits on recently-accessed file contents as well. Beyond that, experimental
file systems have tried shuffling files on disk based on access patterns to
allow a single physical disk access to pick up the file(s) that are likely
to be needed next, but I don't know of any commercial file system that does
this (though Windows - gasp! - reportedly has some such mechanism to
expedite application loading).



Let me clarify. For the majority of data access patterns WAFL
handles small files very well, better than most in fact. The reason
is that the clients generally are not accessing all bazillion files at
the same time. My sentiment was primarily in regards to backups,
which I did not mention so how would anyone know? Sorry about that.
For general small file access for clients/apps WAFL is great. For
full file system scans (like backup) nothing handles it well, except
something like FlashBackup which converts the inode map into a bitmap.
Kinda cool actually but I digress.

~F
 




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
WL 400 Tuning and fix ( Solution customer care N° 4203 ) Free Compaq Computers 0 November 12th 04 02:01 AM
Howto install the drivers for mobility Radeon 9200 on Compaq X1000 with Windows Server 2003 !!! (solution) Razvan Ati Videocards 1 August 12th 04 07:13 AM
Headphone Solution for High End Rig Gamer Ati Videocards 2 December 18th 03 03:29 AM
Soyo and ATI Compatiblity Problem in XP ----- SOLUTION FOUND!!!! Hoss.N.Pfeffer Ati Videocards 1 October 9th 03 09:43 PM
Artec or Memorex 48U Scanner Solution BMac Scanners 0 September 25th 03 07:06 PM


All times are GMT +1. The time now is 04:21 PM.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright Š2004-2024 HardwareBanter.
The comments are property of their posters.