VMware ESXi 5 and Drobo

I have a larger-scoped post coming up sometime soon detailing our new infrastructure setup leveraging ESXi 5, Drobo, Veeam, and other best of breed technologies, but for now, I thought I’d bring some (more) attention to a pairing that is growing in number: VMware and Drobo, specifically, ESXi 5 and the Drobo B1200i.

Drobo recently introduced the B1200i as an entry-level iSCSI SAN for the enterprise. At around $12,000 for 12TB of storage, you’re talking serious bang for your buck if value is at the top of your priorities list. It also rates very high in the idiot test. If you have these things spread all over the world (which we do), any set of hands can be instructed to just plug a new drive in where the red light is.

Drobo has some pretty decent docs for setting up your B1200i for ESXi 5. This document outlines the Drobo-sanctioned way of setting up your array for VMware iSCSI connectivity. However, a recent Drobo KB article adds some wrinkles, and B1200i owners will be keen to pay attention to the differences.

• Firmware updates. As of right now, the B1200i is at 1.0.2, and at a minimum, you “must” [official Drobo word] use 1.0.1 to run on VMware. When I started this project, I got it working with 1.0.0, but apparently the performance improvements are vast starting at the 1.0.1 level.
• Use a dedicated iSCSI VLAN for storage traffic. Kind of a duh point, but still worth mentioning. I enabled jumbo frames in my environment, from ESXi to the Cisco to the Drobo, but I don’t see any difference in general use, and haven’t done performance metric testing.
• Create multiple 1- or 2TB LUNs. This I don’t yet understand, and I plan on bringing it up with a Drobo support engineer. Maybe it has to do with the way Drobo protects and optimizes access to these specifically-sized LUNs across the platters.
• If possible, allocate one VM per LUN. Yuck! Right after they recommend at least a 1TB volume, they say, put no more than one VM in it! Yeah, my 40GB Domain Controller is going to be sitting all alone in that 1TB volume. Boo hiss to this. Now, with just a handful of VMs and a fully loaded Drobo, this is no problem capacity-wise. But I still plan on raising this issue with Drobo as well, if for nothing else other than better understanding what makes the Drobo tick.
• Set NMP (Native Multi-pathing) to Round Robin on all iSCSI LUNs. The first Drobo doc I linked to has you setting LUNs to MRU (Most Recently Used), so this is an important change.

One we roll more of these out, and I get some free time at the office (ha!), I plan on putting these B1200i’s through their paces with some load testing to get some real numbers on performance. For now, they serve up our (admittedly basic) apps admirably. VM Guest boot time and RDP logins seem to lag a bit, but once the systems are up they’re plenty snappy. Again, watch this post for updates as I plan to flesh out the areas where my understanding is lacking.

SnapManager for SQL and SMVI “Recent” Snapshot Naming Bug

NetApp pushes its SnapManager software as the answer for backing up Exchange (SME), SQL (SMSQL), Oracle, SharePoint, and VMware virtual infrastructure (SMVI). It’s not without its faults, however, and I came across a pretty nasty one when first implementing SMSQL with the objective of eventually sending these backups off to tape.

Coupled with the bug that I will get to soon, NetApp’s support team seems to have a communications problem. I’ve talked to various engineers on this, only to be told that this was the first they’ve heard of it. Anyway, here’s how the problem manifests itself:

When creating a backup using SMSQL, one of the backup options is naming convention. You can choose between generic (which appends a “_recent” suffix to the backup name) and a timestamp, which gives each backup a unique name based on the time it was taken. The logic in the generic choice is that you always know what your most recent backup’s name will be. Which is an absolute necessity when it comes to using NDMP as a means for backup; you need to know the exact path name in order to back it up to tape.

The problem exhibits itself when you run SMSQL in conjunction with SMVI – SMVI actually wraps the snapshot name in its own container. So while a SMSQL snapshot name *should* be something like “sqlsnap__servername__recent”, it ends up with this: “smvi__sqlsnap__servername__recent_20120213022637”. You can see that SMVI is prepending the SMSQL job with its own name, and appending the unique timestamp to the end, completely eliminating the purpose of the recent naming convention.

To work around this, I had to create a PowerShell script that NetBackup called when the backup job for that volume executes. The script greps the snapshot list for the “recent” string (since there is only one with that in there), renames it to it’s proper name, and then runs the NDMP backup. Once the backup is complete, it renames the backup back to its “improper” form. If you don’t rename it back, SMSQL will break as it loses track of its most recent backup.

All in all, a bit of a kludge, but the official word (?) out of NetApp is that this bug requires a re-write of the SMVI framework, so the current plan is to see that sometime after Q2-2012. We’ll see about that. For now, this solution works wonderfully.