If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
Low-end SAN performance: EMC AX100
We are needing to orient our data storage towards a SAN and we are
thinking of going with two 3 TB AX100's, and a 16 port switch. We have 6 Linux compute servers that currently need to access the two storage units, however not all servers need access to both AX100's. We plan to add at least two more Linux servers in the very near future. I've seen a report ( http://groups.google.com/groups?hl=e...r4 %404ax.com ) about poor performance with the CX400's and was wondering if the same sort of problem would be encountered with the AX100 since they both use ATA (SATA in the case of the AX100) drives. We would be using large volumes on each device that multiple servers would mount to. I'm just starting to think about SAN's. I don't know a whole lot about them. Would other devices like the Apple Xserve RAID or the NexSAN ATABoy/Beast be similar to the AX100? We use a combination of large files (between 80 MB and 1.5 GB) and small files. I also have a couple of basic questions: What protocol is used to mount these devices to our Linux machines? Is it NFS? SMB? Something else? NFS in Linux is better than it was but it still has a lot to be desired. Should I expect a big performance boost compared to what I'm getting now with NFS over Gigabit ethernet connected to a 2TB 3Ware RAID system that gets 200MB/s reads and 150MB/s writes (locally). That is, will I not get any improvement since NFS might be the bottleneck? I'm hoping that the Fibrechannel connection will help a lot. We also have some machines that don't have PCI-X slots. Are there straight PCI HBA's that can be used that still give good performance? I've seen some HBA's in the $300 range. Are the $900 ones dramatically better? Thanks for your help. If anyone has any other ideas about how we might set up our storage I'd be interested in hearing it. Steve -- __________________________________________________ ____________________ Steve Cousins, Ocean Modeling Group Email: Marine Sciences, 208 Libby Hall http://rocky.umeoce.maine.edu Univ. of Maine, Orono, ME 04469 Phone: (207) 581-4302 |
#2
|
|||
|
|||
Steve Cousins wrote:
I've seen a report ( http://groups.google.com/groups?hl=e...r4 %404ax.com ) about poor performance with the CX400's and was wondering if the same sort of problem would be encountered with the AX100 since they both use ATA (SATA in the case of the AX100) drives. The AX-100 and the CX-400 are completely different animals, so there is no reason to expect the AX-100 to have similar performance characteristics. Whether that is good or bad news is tough to tell since EMC hasn't published any comparitive information :-) .. I'm just starting to think about SAN's. I don't know a whole lot about them. Would other devices like the Apple Xserve RAID or the NexSAN ATABoy/Beast be similar to the AX100? We use a combination of large files (between 80 MB and 1.5 GB) and small files. Mostly read or mostly write or somewhere in between? Do the servers need to share a single filesystem? I also have a couple of basic questions: What protocol is used to mount these devices to our Linux machines? Today, you need a Fibre Channel host bus adapter in each LINUX machine and the drives will the same as locally attached SCSI disk, i.e. they will have /dev/hd entries a will need to formatted & mounted locally. Is it NFS? SMB? Something else? NFS in Linux is better than it was but it still has a lot to be desired. Should I expect a big performance boost compared to what I'm getting now with NFS over Gigabit ethernet connected to a 2TB 3Ware RAID system that gets 200MB/s reads and 150MB/s writes (locally). That is, will I not get any improvement since NFS might be the bottleneck? I'm hoping that the Fibrechannel connection will help a lot. If what you need is a shared filesystem, then if you go the SAN route you'll need to add some sort of filesystem clustering software to each of your servers, a SAN doesn't directly support anything similar to a shared filesystem. We also have some machines that don't have PCI-X slots. Are there straight PCI HBA's that can be used that still give good performance? I've seen some HBA's in the $300 range. Are the $900 ones dramatically better? Most PCI-X HBAs are smart enough to dumb down their operation when placed in a PCI slot. Thanks for your help. If anyone has any other ideas about how we might set up our storage I'd be interested in hearing it. You need to provide some addtional information about your requirements. -- Nik Simpson |
#3
|
|||
|
|||
Nik Simpson wrote:
I'm just starting to think about SAN's. I don't know a whole lot about them. Would other devices like the Apple Xserve RAID or the NexSAN ATABoy/Beast be similar to the AX100? We use a combination of large files (between 80 MB and 1.5 GB) and small files. Mostly read or mostly write or somewhere in between? Somewhere in between. When we run a model, it reads in about 50 MB of data per day and outputs about 80 MB per day. By "per day" I mean "per model day". If we are re-running our model for 2003 then it needs to read in about 18 GB and output about 30 GB. However, other programs just read through these model output files and then create a single 100 MB file. Do the servers need to share a single filesystem? Yes. Since all of these compute servers are Linux is there a prefered file system? GFS? Do these work well? I've heard of CXFS from SGI and we do have an SGI machine (Origin 3000) but I've heard this is quite expensive. I haven't priced any of these file systems out yet so maybe they are all expensive. I also have a couple of basic questions: What protocol is used to mount these devices to our Linux machines? Today, you need a Fibre Channel host bus adapter in each LINUX machine and the drives will the same as locally attached SCSI disk, i.e. they will have /dev/hd entries a will need to formatted & mounted locally. Ah. I see. That helps a lot. If what you need is a shared filesystem, then if you go the SAN route you'll need to add some sort of filesystem clustering software to each of your servers, a SAN doesn't directly support anything similar to a shared filesystem. OK. See above. We also have some machines that don't have PCI-X slots. Are there straight PCI HBA's that can be used that still give good performance? I've seen some HBA's in the $300 range. Are the $900 ones dramatically better? Most PCI-X HBAs are smart enough to dumb down their operation when placed in a PCI slot. Thanks for your help. If anyone has any other ideas about how we might set up our storage I'd be interested in hearing it. You need to provide some addtional information about your requirements. What other information do you need? Basically, if NFS with Linux worked better I'd be happy with that. Since it doesn't work as well as I'd like, I want to see what it would take to do the same thing with a SAN (or if there is some other way please let me know; I use SMB and am mixed about it for our purposes) . We want a bunch of machines to share the same data and be able to read/write to the same volume. By doing this I hope to increase performance and centralize the storage to make it easier to manage the data. Cheers, Steve -- __________________________________________________ ____________________ Steve Cousins, Ocean Modeling Group Email: Marine Sciences, 208 Libby Hall http://rocky.umeoce.maine.edu Univ. of Maine, Orono, ME 04469 Phone: (207) 581-4302 |
#4
|
|||
|
|||
Steve Cousins wrote:
Nik Simpson wrote: I'm just starting to think about SAN's. I don't know a whole lot about them. Would other devices like the Apple Xserve RAID or the NexSAN ATABoy/Beast be similar to the AX100? We use a combination of large files (between 80 MB and 1.5 GB) and small files. Mostly read or mostly write or somewhere in between? Somewhere in between. When we run a model, it reads in about 50 MB of data per day and outputs about 80 MB per day. By "per day" I mean "per model day". If we are re-running our model for 2003 then it needs to read in about 18 GB and output about 30 GB. However, other programs just read through these model output files and then create a single 100 MB file. Any idea what sort of sustained rate you need, i.e. MB/s that you like to make available to the client. Is it a workflow type operation, i.e. server "a" reads the file does some processing creates an output file, server "b" reads outhput, does some processing an so on. Or do all the servers open the same file at the same time & read it. Do the servers need to share a single filesystem? Yes. Since all of these compute servers are Linux is there a prefered file system? GFS? Do these work well? I've heard of CXFS from SGI and we do have an SGI machine (Origin 3000) but I've heard this is quite expensive. I haven't priced any of these file systems out yet so maybe they are all expensive. You might want to ask about SAN filesystems in one of the LINUX groups as well, I'll be homest, other than knowing the names of the various products I know little about their pros & cons. What other information do you need? Basically, if NFS with Linux worked better I'd be happy with that. It might be useful to tell us what you are unhappy with vis NFS, performance? If it's performance, then how much performance do you need, there might be NAS solutions that would be worth investigating. It would also help to have some idea of the budget, i.e. is this a project where everything has to be done for the absolute minimum bucks, or is this a problem that you can spend money to solve? -- Nik Simpson |
#5
|
|||
|
|||
Nik Simpson wrote:
Any idea what sort of sustained rate you need, i.e. MB/s that you like to make available to the client. It doesn't have to be blazing but it would be nice to have it as fast as possible. If each client could sustain 40 to 50 MB/sec with large files that would be great. Is it a workflow type operation, i.e. server "a" reads the file does some processing creates an output file, server "b" reads outhput, does some processing an so on. Or do all the servers open the same file at the same time & read it. The programs are independent of one another. There is no work flow like what you outline. Generally we have a bunch of researchers running different ocean models who need access to the same initialization files and we have some compute servers that they can run these models on. I want all of the compute servers to look pretty much the same so the researchers don't need to worry about which machine they are using. What other information do you need? Basically, if NFS with Linux worked better I'd be happy with that. It might be useful to tell us what you are unhappy with vis NFS, performance? The main problem has been server load. Currently one of our compute nodes (dual Opteron) also has the 2 TB RAID array on it. If a bunch of the compute servers are hitting it at the same time that it is also doing some work then the load can get quite high which reduces performance on everything. For instance I just ran a process that would read 1.2 GB (thirty 40MB files), process it and output about 11 MB. The file server is also currently running a long term model and another machine is also running a long term model. The load on the file server went from a little over 2 to above 11 and everything slowed down quite a bit. Even if this server is not running a model, the load still can go up pretty high if one machine is continually reading through the output files like this. I believe it is a Linux NFS client issue. With the need to bring more servers on-line in the future, I see NFS as being even more of a bottleneck (at least with Linux clients). I don't want to increase the problem when a compute server is added. If it's performance, then how much performance do you need, there might be NAS solutions that would be worth investigating. It would also help to have some idea of the budget, i.e. is this a project where everything has to be done for the absolute minimum bucks, or is this a problem that you can spend money to solve? A NAS solution might work and be more affordable, although the Linux client issue might be a problem no matter what filer we use. In any case, we do have some money to add storage and make things more efficient. Thanks for your help with this Nik. Steve -- __________________________________________________ ____________________ Steve Cousins, Ocean Modeling Group Email: Marine Sciences, 208 Libby Hall http://rocky.umeoce.maine.edu Univ. of Maine, Orono, ME 04469 Phone: (207) 581-4302 |
#6
|
|||
|
|||
Hi,
"Nik Simpson" wrote in message ... Steve Cousins wrote: Nik Simpson wrote: I'm just starting to think about SAN's. I don't know a whole lot about them. Would other devices like the Apple Xserve RAID or the NexSAN ATABoy/Beast be similar to the AX100? We use a combination of large files (between 80 MB and 1.5 GB) and small files. Mostly read or mostly write or somewhere in between? Somewhere in between. When we run a model, it reads in about 50 MB of data per day and outputs about 80 MB per day. By "per day" I mean "per model day". If we are re-running our model for 2003 then it needs to read in about 18 GB and output about 30 GB. However, other programs just read through these model output files and then create a single 100 MB file. Any idea what sort of sustained rate you need, i.e. MB/s that you like to make available to the client. Is it a workflow type operation, i.e. server "a" reads the file does some processing creates an output file, server "b" reads outhput, does some processing an so on. Or do all the servers open the same file at the same time & read it. Do the servers need to share a single filesystem? Yes. Since all of these compute servers are Linux is there a prefered file system? GFS? Do these work well? I've heard of CXFS from SGI and we do have an SGI machine (Origin 3000) but I've heard this is quite expensive. I haven't priced any of these file systems out yet so maybe they are all expensive. You might want to ask about SAN filesystems in one of the LINUX groups as well, I'll be homest, other than knowing the names of the various products I know little about their pros & cons. What other information do you need? Basically, if NFS with Linux worked better I'd be happy with that. It might be useful to tell us what you are unhappy with vis NFS, performance? If it's performance, then how much performance do you need, there might be NAS solutions that would be worth investigating. It would also help to have some idea of the budget, i.e. is this a project where everything has to be done for the absolute minimum bucks, or is this a problem that you can spend money to solve? I would say that your requirements are pretty lightweight for cxfs :-) It becomes cost effective for large i/o rates. But i have a simple solution to your problem. You have a Origin 3000 which is quite an i/o monster. You could put the latest version of irix on it and turn it into an NFS server free of cost. The features supported by IRIX NFS server are : (a) ~1 GB/s i/o rates !!! Not many can touch this. These scalability fixes were added to 6.5.22f and later versions of IRIX. (b) Support for some nice direct i/o on the clients and some buffer copy avoidance techniques. Please dont be fooled by the high iops rate glorified by other vendors because looking at your workload,it looks like you will benefit from I/O throughput more than IOPS. Just my 2 cents. Cheers, Arun |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Poll (please): Time-shifting Performance | Bryan Hoover | Ati Videocards | 1 | December 15th 04 11:56 PM |
Question about performance | The Berzerker | Ati Videocards | 1 | September 27th 04 09:25 PM |
G400 & G-series RR performance question. | Kevin Lawton | Matrox Videocards | 6 | May 20th 04 09:51 PM |
Maximum System Bus Speed | David Maynard | Overclocking | 41 | April 14th 04 10:47 PM |
Geforce 4 2D/desktop performance in WinXP | zmike6 | Nvidia Videocards | 2 | August 29th 03 07:41 AM |