Comments on: Benchmarks – “Lies and the Lying Liars Who Tell Them” http://thesantechnologist.com/?p=10 A Storage Technology Blog by a Technologist Mon, 26 Oct 2009 04:20:58 +0000 hourly 1 http://wordpress.org/?v=3.0.1 By: Mr. Beaker with a fire in the lab. NetApp’s Alex McDonald with Lab tests and a flame war. | The SAN Technologist http://thesantechnologist.com/?p=10&cpage=1#comment-279 Mr. Beaker with a fire in the lab. NetApp’s Alex McDonald with Lab tests and a flame war. | The SAN Technologist Thu, 18 Sep 2008 21:48:45 +0000 http://thesantechnologist.com/?p=10#comment-279 [...] As interesting as funny puppets are, and if someone actually took the time to read my Blog entries as a cumulative collection of posts and not just what is on the front page, they will see I’m usually un-biased.  In fact I specfiically call out NetApp as one of the lead NAS vendors in several areas, including having the SPEC-SFS results that showed the OnTap GX cluster as by far the IOPs leader for clustered file systems. [...] [...] As interesting as funny puppets are, and if someone actually took the time to read my Blog entries as a cumulative collection of posts and not just what is on the front page, they will see I’m usually un-biased.  In fact I specfiically call out NetApp as one of the lead NAS vendors in several areas, including having the SPEC-SFS results that showed the OnTap GX cluster as by far the IOPs leader for clustered file systems. [...]

]]> By: The Storage Area Network (SAN) Technologist http://thesantechnologist.com/?p=10&cpage=1#comment-16 The Storage Area Network (SAN) Technologist Fri, 21 Sep 2007 19:42:05 +0000 http://thesantechnologist.com/?p=10#comment-16 [...] Benchmarks - “Lies and the Lying Liars Who Tell Them” September 6th, 2007 by Steven J. Schwartz [...] [...] Benchmarks – “Lies and the Lying Liars Who Tell Them” September 6th, 2007 by Steven J. Schwartz [...]

]]> By: Steven J. Schwartz http://thesantechnologist.com/?p=10&cpage=1#comment-13 Steven J. Schwartz Mon, 17 Sep 2007 20:33:30 +0000 http://thesantechnologist.com/?p=10#comment-13 I received this from Exanet in response to my Benchmarks post. I actaully considered these factors in my write up if you read through the details and not just the summary table. Hi Steven! I'm amazed that you spent so much time investigating this! I noticed that you used our 6-nodes test from 2004 for this comparison. While this was indeed our highest SpecSFS result published, this was not our best price/performance one. We ran the same benchmark again in 2005, and got to 143,380 NFSOps with a configuration priced at $338,544 - which was only $2.35/IOPs. This actually shows a major benefit of our open approach - the sheer economies of scale of the PC market and RAID subsystems allow us to offer the best price/performance. This is while our competitors offer solutions based on proprietary hardware - with vendor lock-in and inflated hardware prices. Now - what about single file system vs. single name space with actually multiple file systems? The SpecSFS test does actually ask specifically for Number of Filesystems to be publicized. In fact - in the NetApp GX test (http://www.spec.org/sfs97r1/results/res2006q2/sfs97r1-2006052200263.html) NetApp was careful enough and answered "single namespace (see notes)". This is because the system was configured with 24 file systems - as NetApp GX in its standard form is a set of separate file systems aggregated into a single namespace, and not a distributed clustered file system per-se. So - if you consider this, the IOPs/FS column clearly shows that Exanet is providing by far the fastest IOPs/FS ratio: NetApp 43,019 EMC 45,723 Panasas 5,096 Exanet 178,156 BlueArc 12,218 Yours, Eyal Traitel Product Marketing Manager Exanet, Inc. I received this from Exanet in response to my Benchmarks post. I actaully considered these factors in my write up if you read through the details and not just the summary table.

Hi Steven!

I’m amazed that you spent so much time investigating this!
I noticed that you used our 6-nodes test from 2004 for this comparison. While this was indeed our highest SpecSFS result published, this was not our best price/performance one. We ran the same benchmark again in 2005, and got to 143,380 NFSOps with a configuration priced at $338,544 – which was only $2.35/IOPs.
This actually shows a major benefit of our open approach – the sheer economies of scale of the PC market and RAID subsystems allow us to offer the best price/performance. This is while our competitors offer solutions based on proprietary hardware – with vendor lock-in and inflated hardware prices.

Now – what about single file system vs. single name space with actually multiple file systems?
The SpecSFS test does actually ask specifically for Number of Filesystems to be publicized. In fact – in the NetApp GX test (http://www.spec.org/sfs97r1/results/res2006q2/sfs97r1-2006052200263.html) NetApp was careful enough and answered “single namespace (see notes)”. This is because the system was configured with 24 file systems – as NetApp GX in its standard form is a set of separate file systems aggregated into a single namespace, and not a distributed clustered file system per-se.

So – if you consider this, the IOPs/FS column clearly shows that Exanet is providing by far the fastest IOPs/FS ratio:

NetApp 43,019
EMC 45,723
Panasas 5,096
Exanet 178,156
BlueArc 12,218

Yours,
Eyal Traitel
Product Marketing Manager
Exanet, Inc.

]]>