In the past, when designing enterprise storage solutions for virtual infrastructures as I have, we were basically provided two primary considerations to complete a valid design.
One, you must have enough capacity to store your VMs.
Two, you must have enough I/O to meet your virtual workloads requirements.
A third consideration, though not strictly a necessity, was software integration with the hypervisor and storage array.
As disk sizes have increased, we no longer have to throw shelves and shelves of disks at the design to meet capacity needs. Likewise, with the introduction of Flash (EFDs, SSDs etc.) into our storage solutions, we no longer have to throw shelves and shelves of disks at the solution to meet our I/O requirement (a la “short stroking”). Now, customer demands are pushing for software defined storage solutions as their number one priority. The demand for “VM aware” storage is greater now than it ever has been.
ENTER the Host
We can argue what network protocol and medium in which to use for as virtual storage solution (Fibre Channel, Ethernet, NFS, iSCSI etc) until we are blue in the face, what’s not up for debate is this: if we move the active data closer to the CPU we are going to get decreased latency and increased response time.
To realize this we must first accept two conditions.
- Capacity Storage is becoming a commodity
- I/O requirements can and will be met with Host based Flash solutions
With these assumptions, we can begin to focus on storage solutions that truly provide “value add” at the software layer. Let’s focus on the second condition.
Solutions for utilizing host based flash aren’t exactly new. Product offerings from companies like Fusion IO, allow us to insert high quantities of flash storage into our virtual hosts. The real advantage of utilizing these types of peripheral interconnects comes out of the software that successfully manages them with efficiency and transparency to the data as it’s stored and retrieved.
The author thinks it takes the following attributes for a host based flash solution to be truly enterprise worthy:
- It must be truly clustered and compatible with cross-host functions like vMotion, DRS, HA, etc. And “being compatible” should mean that the product allows true network-based sharing and movement of a VM’s flash footprint as it goes through vMotion, DRS migrations, etc.
- It must support write acceleration, and in a manner that doesn’t incur data loss on host failures.
- It must not be a virtual appliance. I repeat, it should not be a virtual appliance. This often leads to performance, availability, scalability, and manageability problems. It must have seamless, simple, integration into the hypervisor management console.
- It must not require in-guest or application level changes. This is operationally un-scalable.
- It must be flash technology agnostic!
- It must work on per VM basis. We should have the ability to apply such a solution at the VM level, when we want to, and for how long. Such a solution should be “VM aware” with proper reporting. For example, we should be able, at any point in time, to see and understand the total number of IOPs being utilized from the solution by a particular VM. And thus make intelligent admin decisions based on this information.
I recently had the privilege of discussing such a solution with Satyam Vaghani. You may recall Satyam for his long tenure at VMware where he helped create ESXi, VMFS, VAAI and virtual volumes. He is now the CTO and cofounder of PernixData where he is working on his next game changing technology.
I asked about PernixData’s product offerings and how they may provide change that could impact the storage playing field?
His response: “At PernixData, we envision the need for a brand new yet seamless software defined data tier in front of the primary storage system and alongside the VMs. This tier provides IOPS for data-in-motion (“hot” data), and unburdens the storage system so it can focus on providing capacity and data services for your data-at-rest.”
More to come and full on follow up/interview with Satyam will be in the months ahead.
One thing is for certain: my designs for enterprise storage systems that support virtual workloads have changed. I now have to focus not only on capacity and I/O requirements but how the next generation software defined data center interacts with the data as it is being stored and fetched.