I regularly get asked about storage protocols, especially when it comes to the right protocol to use to virtualization.  There is not a single right answer here.  However, there are several less efficient ways to do it.

Boston – Copley Square: The Tortoise and the Hare (Photo credit: wallyg)

NFS vs. iSCSI vs. FCoE vs. CIFS vs. ATAoE

Ethernet based storage protocols all get lumped together for comparison and they shouldn’t!  You’ll find below a table of common storage protocols and some basics around them.  As it pertains to Ethernet based protocols, there are two basic types, and they can be divided into SAN(block) and NAS(file) protocols.  There is a very important distinction between the two, and this distinction becomes even more apparent based on the storage vendor and application.  Storage vendors really have two products (there are a very few exceptions to that rule), NAS devices and SAN devices.  There are SAN devices that have the ability to be front ended by a NAS gateway (i.e. Puts a file system on a block device and presents it out as NAS device), and NAS devices that have an ability to emulate a block device by manipulating a large file on its file system.

*Caveats Apply

Which is faster?

There is a difference between speed and throughput!  I’ve commonly used the garden hose vs. water main comparison for clients.  While at Equallogic the conversation of iSCSI vs. FC was the hottest topic, and the speed difference between GigE and 10GbE.  The common misunderstanding was that because 10 > 1 that 10GbE was of course faster!  So back to the water model.  If you need to pass a small pebble (one with a diameter smaller than the diameter of a garden hose), or even a consistent line of small pebbles (transactional data profile) a garden hose can move those pebbles at the same “speed” as a water main.  It is only when data sets that are boulders and multiple concurrent data sets, that would need to be broken down into smaller checks to fit in a garden hose that the water main exceeds the “throughput” of the garden hose.  As a physicist pointed out to me, light travels at the same speed regardless of it being a tiny laser, or a giant one.  So in many cases, we proved that GigE could actually compete with 2Gb & 4Gb Fibre Channel (iSCSI vs. FCP).  With today’s technologies we are comparing 10GbE to 8Gb FC, so the “pipe” is less of an issue.

VMWare – iSCSI, FCP, FCoE, or NFS?

The ongoing debate becomes what storage protocol to use with VMWare.  The history of feature support to storage protocol is important, because in the earlier days of VMWare, features were released for FCP based storage first, then iSCSI, and last NFS.  This is no longer the case with VMWare’s releases.  If you refer to the above table, FCoE, FCP and iSCSI require a file system on top of the protocol.  In the case of VMWare, it is typically VMFS, VMWare’s clustered/shared file system.  Keep in the back of your head that the most difficult thing to develop at scale is a clustered file system (usually because of distributed lock management which doesn’t really apply with vmdks).  NFS however, is already a POSIX compliant shared file system as a storage protocol.  This means that there is no mapping of LUNs to ESX hosts, no management of the VMFS, or multiple VMFS instances, and less overall management required.  NFS doesn’t require specialized networks beyond tuned traditional route/switch.  It doesn’t require any special commands or configuration to grow in size, and it is fully supported within ESX and VMWare!  So given the option of SAN vs. NAS, with the current state of support, I would choose NAS (NFS) for VMWare, however, make sure you choose an enterprise NAS storage solution!!!!

Related articles

• iSCSI SAN/NAS SOLUTION FOR ESXi SERVER (Tutorial No 5) (netwanlan.wordpress.com)
• Storage QoS – How Hard Can It Be? (architecting.it)
• A thought experiment – The Universal Fabric (brasstacksblog.typepad.com)
• VMworld 2012 – Psst Want to see the future of storage with VMware and EMC? (virtualgeek.typepad.com)
• Point, Shoot, Solution! (thesantechnologist.com)