There is a fair amount of talk about NVMe lately as the leading non-volatile memory manufacturers focus their shift on creating products based on it and as more people – as he wave personally witnessed here at Thinkmate – Start adopting them.
Here we’ll try to explain briefly what NVMe is, how our customers are using it in the real world and how you can purchase the right drives and put them in the right server.
NVMe is an interface specification for PCI Express-based solid drives. This isn’t too different from the Intel-defined AHCI that enhanced the SATA bus by adding support for things like hot-plugging and Native Command Queuing. NVMe is the layer between the device driver and the PCIe device itself, creating a standardized version of rules with focus on low latency, scalability and security. Most recent high-end motherboards should see firmware updates that makes them NVMe-capable, allowing PCIe drives to be fully bootable.
NVMe is an interface specification developed for PCIe-based SSDs by a consortium of 13 members including Intel, Samsung and others. It was designed to be future-proof and to penetrate both client and enterprise markets, without inheriting any hard drive features.
As most of us know, The PCI Express bus offers extremely high bandwidth availability due to it being directly attached to the CPU. It was only natural that drive manufactures would want to take advantage of the increased bandwidth PCIe offers. As non-volatile memory technologies advanced, they found themselves limited by the bottlenecks of standards that were created for an era where drives where slow, mechanical spinning disks.
For a long time, many companies were creating their own PCIe drive solutions, each with their own proprietary register sets, feature sets and command sets. This created an inconsistency that threatened interoperability with hardware such as motherboards, CPUs and software such as drivers and OS support.
Still, the trend was clear: SAS and SATA SSDs were popular, but PCI Express SSDs would see significant growth.
There was a clear need for a standard, So NVMe was created. Developed by an open industry consortium that includes 13 promoter group members (listed below), NVMe was created as a standard, future-proof interface designed to penetrate both the client and enterprise markets.
Its focus was to create a specification made specifically for non-volatile memory (hence the name NVMe), without inheriting any hard drive legacy, supporting not only the most common NAND today but also any other technology that may appear in the future, such as phase-change memory (PCM), MRAM and others.
As discussed before, NVMe allows drives to use the PCI Express connection routes to offer many advantages that make it a more attractive option over standard SSDs, and in many cases, make it the only viable solution for many types of application and usage scenarios.Speed vs ATA/SATA
By sitting directly on top of PCI Express, NVMe drives are not affected by the bottlenecks of the ATA interface – delivering up to 4X better IOPs(Input/Output Operations Per Second, pronounced eye-ops) than the fastest SAS option available. With NVMe, the bottleneck in speed is now in the NAND itself, so we should expect even more performance gains as NAND technology advances and new technologies come to market.Multiple lanes = Scalable performance
NVMe also has the advantage of offering scalable performance, enabling drives to take advantage of the same “pool” of lanes that connect directly to the CPU. So just like our graphics cards are using x16 lanes, Gigabit network adapters are using x4 or x8, PCIe SSD drives can take advantage of extra lanes and go beyond the common x4 found in most PCIe SSD drives found today.Low Power Features
NVMe drives also offer exceptionally low power use in standby mode. Some have adopted the L1.2 low-power standby mode defined by the PCIe standards body, reducing consumption to under 2mW – A 97 percent decrease from the 50mW consumed using an L1 state, which is the most widely used low-power mode today. In addition to low power in idle, there are also different power states available. This could translate into real savings for many customers in the enterprise or data center.Security
The NVMe command set supports security container commands analogous to the security container commands found in the SCSI and ATA/ACS command sets, allowing NVMe-based SSDs to support industry standard security solutions such as the Opal SSC and Enterprise SSC specifications published by the Trusted Computing Group.
Here at Thinkmate, we are seeing a significant increase in NVMe adoption among customers who buy storage servers. Our customers are telling us that they’re really pleased with the performance and reliability of the drives, using them in database applications, big data analytics, as cache in front of a storage array, in a software defined storage solution or Hyper-V server.
Generally, NVMe is a good candidate to be considered anywhere traditional flash storage storage solutions are used such as in storage pools, for tiering or any application or usage scenario that demands high availability from the drive.
As with Hard Drives and SSDs, there are many different types of NVMe drives on the market with different specifications, sizes and form factors.
Before getting into the different form factors, I feel it’s important to stress the importance of buying quality memory. While there are many vendors selling non-volatile memory in the market, there are only three companies that currently have the capability to produce NAND: Intel/Micron, Samsung and Toshiba. Here at Thinkmate, we take data integrity and security very seriously. We do not build any servers with NAND that does not come from these companies directly. It is a company policy that stems from our commitment to building the most reliable servers in the industry.Add-In Cards
Add-In cards have the advantage of being the most supported way of using NVMe today. All it needs is a system with a supported motherboard with an available PCIe slot. Most servers sold by Thinkmate today support them out of the box. There are a few disadvantages, however: They can’t be hot-swapped, they utilize a higher power envelope and they take more physical space, making them harder to be deployed and serviced.U.2 NVMe 2.5 Inch drives
U.2 NVMe 2.5 Inch drives look like most SSDs sold today, However, they connect to an M.2 port on the motherboard using a cable. The form factor makes it similar to SSDs in terms of ease of deployment and serviceability, plus adding hotplug capability, making it suitable for the Data Center, Enterprise and among PC enthusiasts.M.2 Drives
M.2 Drives look like “sticks” of memory and they connect directly to the M.2 port on the motherboard. The advantage of M.2 drives is that they come in different lengths: 42mm, 80mm and 110mm, taking considerably less amount of physical space compared to U.2 and PCIe drives.
Servers with support for both U.2 and M.2 drives are available now at Thinkmate. To see a list of supported systems, simply type in “m.2” in our search box.
Choosing the right storage size seems like an easy task, but many people make the mistake of just looking at storage size and price, going for the drive with the lowest cost per gigabyte. However, not all memory is created equal, and you need to factor in how hard your application will be pushing your drive. This is a very important step that no one should overlook when looking for drives. Generally, manufacturers split their drive solutions into three different use tiers: Read Intensive, Mixed Use and Endurance:
As you can see above, the amount of writes per day can differ dramatically from one tier to another, and it’s important to take the time to assess how you will be using your drive in order to avoid major headaches like outages or even data loss.
As you saw above, NVMe drives can use different connections and come in different form factors. Some servers might have the available PCIe slots to accommodate PCIe SSD drives, but no motherboard support, making things even more confusing.
As of right now, the only manufacturer offering servers with true hot-swap capability is Supermicro, who has been aggressive in its support for NVMe with its X10 line of servers. Supermicro just seems to have this impressive ability to engineer a diverse array of servers in different sizes with new technologies like NVMe… However, none of this matters if your systems integrator is lazy to bring these products into your hands. We honor our position of being the world’s #1 Supermicro systems configurator by not only making these systems available as soon as possible, but providing a better browsing experience that encourages exploring, eliminates confusion and doesn't hide pricing.
With that said, here are the systems available with true NVMe hot-swap capability available immediately, directly from our Supermicro Store: