If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
IOPS benchmarks - looking for opinions
Hi everyone,
I recently read about the heated contest between two Storage vendors regarding some IOPS tests one did with the other's gear. The latter's claim is that the IOPS benchmark is biased and irrelevant, and therefore they don't run it with their own products. Not being neither a HPC or a high IO expert, I can't really decide whether or how relevant the claims are - whilst I found some arguments compelling. Being curious, I'll be interested in reading what some of the knowledgeable minds haunting this group would have to say regarding the IOPS benchmark validity and how relevant they are to the "real world" - whether or not they are a good proxy to assess how a given piece of hardware is able to sustain an heavy IO load. I don't want to discuss the validation process used in by one of the vendor, I have my own opinion on this matter, and I'd rather prefer to avoid replicating the heated exchanges I've read in the respective blogs of those two vendors :-) Thanks in advance. Fx -- "Ah, the beauty of OSS. Hundreds of volunteers worldwide volunteering their time inventing and implementing new, exciting ways for software to suck." -- Toni L. in alt.sysadmin.recovery |
#2
|
|||
|
|||
IOPS benchmarks - looking for opinions
fx [François-Xavier Peretmere] wrote:
Hi everyone, I recently read about the heated contest between two Storage vendors regarding some IOPS tests one did with the other's gear. The latter's claim is that the IOPS benchmark is biased and irrelevant, and therefore they don't run it with their own products. Not being neither a HPC or a high IO expert, I can't really decide whether or how relevant the claims are - whilst I found some arguments compelling. Being curious, I'll be interested in reading what some of the knowledgeable minds haunting this group would have to say regarding the IOPS benchmark validity and how relevant they are to the "real world" - whether or not they are a good proxy to assess how a given piece of hardware is able to sustain an heavy IO load. I don't want to discuss the validation process used in by one of the vendor, I have my own opinion on this matter, and I'd rather prefer to avoid replicating the heated exchanges I've read in the respective blogs of those two vendors :-) IOPs are much like having a speedometer that goes to 200 miles per hour, but an engine that may only stay put together at speeds up to 65 miles per hour. Put a set of file systems or luns on half of the storage pool on a set of arrays. Then benchmark. Then put two file systems or luns on the same set of arrays such that you fill the arrays. Crank up benchmarks on both. Some arrays will do very well, some won't. What have you demonstrated as far as your actual application performance? Very little if nothing at all. There are better tools, most are application specific and some of those can be coaxed into producing results predictive of performance on that application. |
#3
|
|||
|
|||
IOPS benchmarks - looking for opinions
moojit wrote:
""fx [François-Xavier Peretmere]"" wrote in message ... Hi everyone, I recently read about the heated contest between two Storage vendors regarding some IOPS tests one did with the other's gear. The latter's claim is that the IOPS benchmark is biased and irrelevant, and therefore they don't run it with their own products. Not being neither a HPC or a high IO expert, I can't really decide whether or how relevant the claims are - whilst I found some arguments compelling. Being curious, I'll be interested in reading what some of the knowledgeable minds haunting this group would have to say regarding the IOPS benchmark validity and how relevant they are to the "real world" - whether or not they are a good proxy to assess how a given piece of hardware is able to sustain an heavy IO load. IOPS are a valid test. All IOPS calcuations (atleast the ones advertised by a vendor) are performed using 512B transfer sizes which is the typical sector size used by most block SCSI devices. It gives you an indication of how well the vendors HBA driver and firmware perform. IOPS are also dependent on the setup because this value is not dependent on just the HBA driver and firmware, but is dependent on the system itself. Although 512 is the scsi size, it isnt the size used for IOPS calculations. Nor frankly is it even relevant for arrays that use a much larger transfer size and still have the same IOP. Some arrays may have a lower IOP count for 512 byte transfers, or not. Nor is 512 byte transfers overly useful for predicting application performance with apps that regularly use multi-megabyte transfers. e.g. databases. |
#4
|
|||
|
|||
IOPS benchmarks - looking for opinions
Lon wrote:
moojit wrote: ""fx [Fran?ois-Xavier Peretmere]"" wrote in message ... Hi everyone, I recently read about the heated contest between two Storage vendors regarding some IOPS tests one did with the other's gear. The latter's claim is that the IOPS benchmark is biased and irrelevant, and therefore they don't run it with their own products. Not being neither a HPC or a high IO expert, I can't really decide whether or how relevant the claims are - whilst I found some arguments compelling. Being curious, I'll be interested in reading what some of the knowledgeable minds haunting this group would have to say regarding the IOPS benchmark validity and how relevant they are to the "real world" - whether or not they are a good proxy to assess how a given piece of hardware is able to sustain an heavy IO load. IOPS are a valid test. All IOPS calcuations (atleast the ones advertised by a vendor) are performed using 512B transfer sizes which is the typical sector size used by most block SCSI devices. It gives you an indication of how well the vendors HBA driver and firmware perform. IOPS are also dependent on the setup because this value is not dependent on just the HBA driver and firmware, but is dependent on the system itself. Although 512 is the scsi size, it isnt the size used for IOPS calculations. Nor frankly is it even relevant for arrays that use a much larger transfer size and still have the same IOP. Some arrays may have a lower IOP count for 512 byte transfers, or not. small tranfers are pretty taxing and a good way to see how a system performs. Reading and writing large files and getting decent speeds isn't hard. Nor is 512 byte transfers overly useful for predicting application performance with apps that regularly use multi-megabyte transfers. e.g. databases. Not everything is a database with tranfer to and from the storage unit being nearly ideal in size. weak performance for small tranfers can mean horrible backup and restore times as well. even untarring a bunch of small files is tedious on a crappy array. |
#5
|
|||
|
|||
IOPS benchmarks - looking for opinions
fx [François-Xavier Peretmere] wrote:
Hi everyone, I recently read about the heated contest between two Storage vendors regarding some IOPS tests one did with the other's gear. The latter's claim is that the IOPS benchmark is biased and irrelevant, and therefore they don't run it with their own products. Not being neither a HPC or a high IO expert, I can't really decide whether or how relevant the claims are - whilst I found some arguments compelling. Being curious, I'll be interested in reading what some of the knowledgeable minds haunting this group would have to say regarding the IOPS benchmark validity and how relevant they are to the "real world" - whether or not they are a good proxy to assess how a given piece of hardware is able to sustain an heavy IO load. I don't want to discuss the validation process used in by one of the vendor, I have my own opinion on this matter, and I'd rather prefer to avoid replicating the heated exchanges I've read in the respective blogs of those two vendors :-) For the record, I assume you are talking about the Netapp/EMC SPC-1 results that Netapp "kindly" published on EMC's behalf. This is much more complex than simply running 512KB IoMeter type loads. -- Nik Simpson |
#6
|
|||
|
|||
IOPS benchmarks - looking for opinions
On Fri, 28 Mar 2008 12:58:16 -0500, nik Simpson
wrote: fx [François-Xavier Peretmere] wrote: Hi everyone, I recently read about the heated contest between two Storage vendors regarding some IOPS tests one did with the other's gear. The latter's claim is that the IOPS benchmark is biased and irrelevant, and therefore they don't run it with their own products. Not being neither a HPC or a high IO expert, I can't really decide whether or how relevant the claims are - whilst I found some arguments compelling. Being curious, I'll be interested in reading what some of the knowledgeable minds haunting this group would have to say regarding the IOPS benchmark validity and how relevant they are to the "real world" - whether or not they are a good proxy to assess how a given piece of hardware is able to sustain an heavy IO load. I don't want to discuss the validation process used in by one of the vendor, I have my own opinion on this matter, and I'd rather prefer to avoid replicating the heated exchanges I've read in the respective blogs of those two vendors :-) For the record, I assume you are talking about the Netapp/EMC SPC-1 results that Netapp "kindly" published on EMC's behalf. This is much more complex than simply running 512KB IoMeter type loads. I thought Dave Hitz explained the situation very well and had good points. But I hate EMC so my bias is unfettered. But, to the point. Synthetic tests are just that, synthetic. They can give you great insight into worst case or specific case IO scenario's. Some of those may or may not apply to your workload. Real World is so diverse that there's no Real World validation other than each users own application. Oracle data warehousing v. OLTP v. EDA grid apps v. web server v. ....... You get the picture. The truth of it all is that you HAVE to know what your target is, or at least what your current IO is. You can't just say "build me a house" and expect to be happy with the outcome. There are details involved. ~F |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
IOPS from RAID units | davidhoffer | Storage & Hardrives | 11 | March 24th 08 08:38 PM |
IOPS calculation for SAN Design | [email protected] | Storage & Hardrives | 4 | October 21st 07 02:41 PM |
IOPS from RAID units | [email protected] | Storage & Hardrives | 8 | July 21st 07 04:20 PM |
CPU Benchmarks | confused | General | 1 | September 30th 06 05:58 PM |
nfs ops to iops formula? | Faeandar | Storage & Hardrives | 7 | February 23rd 06 05:18 AM |