Posts Tagged ‘Virtualisation’

If, like me, you’ve been using VMware Workstation for a long time, it is quite possible you’ll have dozens of “Workstation” virtual machines littering up your PC. Now you’ve just built a shiny new ESXi virtualisation server, is it possible to convert those ‘desktop’ VMs into something ESXi can handle?

The answer to that is a definite ‘yes’. The tool of choice is a freebie download from VMware themselves, called the VMware vCenter Converter Standalone 4.0.1. It’s a versatile tool, too: if you’ve got Microsoft Virtual PC VMs, it can process them; it will even convert a Symantec Ghost image of a (real, physical) PC into an ESXi-compatible VM. Actually, it can even convert live, running physical PCs into ESXi VMs, which seems devilishly clever to me!

It’s a big download (103MB or so). When you go to install it, you get a choice between a ‘local’ and a ‘client-server’ installation. I picked the local option, so that all VM conversions will take place directly on my Windows 7 desktop (as opposed to me managing them from my desktop but all the heavy lifting taking place on a remote server -definitely an option worth investigating at work!). The conversion process itself is very simple: point the Converter tool at the .vmx file which represents your “desktop” VM’s configuration file; connect the Converter to the ESXi server; choose which of the ESXi’s datastores you want the converted VM to end up on; and then sit back and wait.

In pictures, here’s how the process looks for an XP VM I had with a 30GB virtual hard disk. Once the program is running, select the Convert Machine option from the top toolbar:

You then select your source type (in my case, a VMware Virtual Machine) and point the tool at the .vmx file for that virtual machine:

You then attach to your ESXi server, providing the relevant log on details:

Probably the most important bit is then to specify whereabouts on that ESXi server (which ‘datastore’ in the jargon) you want your VM to be transported to:

You then get a summary screen of what’s about to be converted, and each section of the conversion process can be edited:

So, although my original desktop VM had been assigned 2GB of RAM, I might only want to assign 512MB (for example) to the converted VM when it migrates over to the ESXi server. Click ‘Next’ after that and you get one final confirmation of what’s about to happen. Click ‘Finish’ on that screen, and it all starts happening. My 30GB XP VM was converted and moved to my new ESXi server in 25 minutes flat, which seems pretty reasonable to me. The VMware Tools that were previously installed in the ‘desktop VM’ had to be upgraded manually once the machine was running on the ESXi server, but otherwise everything else was taken care of perfectly.

Another issue that arose with this specific VM Guest conversion was that the XP guest suddenly went silent on me: no sound device existed in the converted machine where one had existed and functioned just fine in the original ‘desktop’ version. That’s curable, however, by following these instructions in the VMware knowledge base, so no permanent harm done. (It raises the question, I suppose, of why one would ever actually want a VM running on a remote server to pipe sound back to your desktop PC via the network: you’re always going to have latency/contention/traffic volume issues, after all. Still it’s nice to know you can do it if you really want to.) Having followed those knowledge base instructions, you still won’t be able to get the XP VM to do sound from within the vSphere Client, claiming that no audi device exists in the XP VM at all; but if you use a Remote Desktop Connection to it, you’ll find the XP machine able to use the ‘Microsoft RDP Audio Driver’, and sound will then be piped across the network.

It’s all incredibly neat and works beautifully.

(Just one niggle: sometimes, when I’d filled in the ESXi logon details and clicked Next, I was told that my session had not been authenticated -and the conversion process won’t take place until it has been. The fact that I definitely supplied the correct username and password wasn’t, apparently, enough. Well, it turns out that this is quite a common problem that (apparently) arises from time differences between your client session and the ESXi server clock. A reboot of your client (in my case, of my Windows 7 desktop) was sufficient to allow me to re-start the wizard and this time authenticate perfectly.

One of the biggest niggles I had (and continue to have) with VMware’s ESXi hypervisor is that its management tool (the “vSphere Client”) won’t run on Windows 7. It installs well enough, but try and actually run it and you’ll simply be told:

Click the OK button there, and you’ll get told rather more mysteriously:

And that’s a clue (to the initiated, at least!) that it’s something wrong with the .NET framework on Windows 7 that is causing all the grief. The fix for this problem has been around for a while and is scattered on various websites in slightly different variations across the wide expanses of the Interweb, so I claim absolutely zero originality for what follows. I’m only repeating it here so I can find it easily in the future and in a version that worked for me!

You need to use a PC running some version of Windows which is not Windows 7 for what follows. I happened to have a VM running Windows XP kicking about the place: it had Service Pack 3 installed, but that was about the only update it had ever received. I didn’t fancy taking the usual ‘Windows Update’ route, because that would have taken for ever, so instead I visited Microsoft’s website and obtained the .NET Framework 3.5 downloader by itself. Run that once it’s downloaded, and the entire .NET Framework gets downloaded and installed (all 60MB of it). Then you visit Microsoft again and download the Service Pack 1 for the .NET 3.5 Framework bootstrapper. Run that, and after another 52MB, you’ve got the SP1 version of .NET 3.5 and you’re in business. (I realise this sounds a long-winded way of doing it: the alternative that several other sites take is to simply provide a link to download the relevant file that we need from their own private websites. But I prefer not to obtain my system dlls from non-authorised sources, and I suspect others might think likewise! Doing it this way, therefore, you can be sure that the files you’re about to copy to your Windows 7 system are kosher Microsoft ones).

Anyway, once you’ve got a non-Win7 .NET Framework 3.5 SP1 installed, find the C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727 directory and copy the system.dll file you find there to your Windows 7 box. It has to go in a very specific location (that will only exist after you’ve installed the vSphere client, so do that first if you haven’t already done so). That location is: C:\Program Files (x86)\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\lib. If you’re not running the 64-bit version of Windows 7, that first bit will be plain old ‘C:\Program Files’, without the ‘(x86)’ bit, but since I am running everything 64-bit these days, the x86-version of the path is what I used. If the “lib” directory doesn’t already exist inside the Launcher folder, just create it and then copy the .dll to there.

In that Launcher directory, too, you’ll find a file called VpxClient.exe.config. Right-click that and open it in Wordpad. Welcome to the wonderful world of XML editing! The rules here are simple: you add these three lines to the file, immediately after the </appSettings> one and immedately before the final </configuration> one:

<runtime>
<developmentMode developerInstallation="true"/>
</runtime>

Make sure the double quotes around the word ‘true’ there are proper ones, not some fancy curly version that assorted web browsers or word processors might add to the mix. Save the modified version of that file which should end up looking like this in its entirety:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.net>
       <connectionManagement>
           <clear/>
           <add address="*" maxconnection="8" />
       </connectionManagement>
  </system.net>
 <appSettings>
   <add key = "protocolports" value = "https:443"/>
 </appSettings>
<runtime>
<developmentMode developerInstallation="true"/>
</runtime>
</configuration>

The new additions are highlighted there in red.

Finally, you need to add a new DEVPATH environment variable to your system. Click the Start “orb” button in your Windows 7 PC, then right-click the Computer menu option and select the Properties item from the context menu. Finally, click the Advanced system settings option from the left-hand pane. On the Advanced tab of the resulting dialogue box, click the Environment Variables button:

In the System variables section of the resulting screen, click the New button. Type in a new variable name of DEVPATH, and set its value to be the lib directory where you stored the system.dll file copied from your non-Win7 box. In other words, in my case, that would be C:\Program Files (x86)\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\Lib:

Click OK as many times as it makes the various dialogues and their pop-up boxes disappear, and you should then be able to run the vSphere Client on your Windows 7 PC without a drama. As is apparently pretty standard no matter which version of Windows you’re running on, you’ll receive an initial ‘certificate warning’, but the correct response there is to select to install the certificate and then click the Ignore button:

Once that’s done, you’re definitely in the business of managing your shiny new ESXi server.

So I am given control of a dual quad core Xeon server (i.e., 8 cores in all) with 2TB + 500GB Hard Disk arrays, plus 32GB RAM. Not the best box we’ve got in the work server room, but pretty nice all the same. Out comes my newly-minted VMware ESXi Server installation CD, and within minutes, the installation is complete. It takes me a further four days (plus an all-nighter), but I finally ended up this afternoon with 7 perfectly-formed Windows 2008 x64 virtual machines running all our test and development Oracle environments. Those same seven databases had previously lived on at least three servers, but things got moved around so much it was hard to tell. The server copes admirably with the load (well, the developers and testers can’t be said to hammer the things hard, I suppose!), and a 7-to-1 consolidation (or even a 3-to-1 ratio, depending on the way you look at it) seems a good outcome all round.

A couple of niggles (inevitably). First, one of our arrays actually consists of 6 500GB disks. We use RAID-5 (and I’m prepared to argue strongly that doing so makes perfect sense in a lot of environments, so let the BAARF people commence battle if they must!), so that means there are approximately 2350GB of usable space on the array… yet ESXi reported it as a datastore of precisely 512 Bytes. What’s a few thousand missing Gigabytes between friends?! Turns out that ESXi 4.0 has a 2TB limit on any one datastore, which meant I had to re-configure the array as a 5-disk one with a hot spare: 1835GB and therefore under the magic number, and accordingly reported correctly (and entirely usable as a result).

Second, in order to make sure my home network resembles my work one as closely as possible for R&D purposes, I attempted to install ESXi on a dual-core box I have spare (only 4GB RAM, but a couple of terabytes of usable disk space, so nothing too bad). Sadly, the installation died an abrupt death with a Failed to load lvmdriver error and continued to do so, no matter which of my many spare Netgear cards I installed, or whether I stuck with the on-board Realtek network interface. The point being that ESXi 4.0 uses the network card to generate a unique ID for each host server -so if it doesn’t support the NIC you’re using, the installation can’t proceed. Nothing that an AU$40 purchase of a proper Intel PRO/1000GT network card wouldn’t fix, however -so things are running nicely here, too.

The whole thing is managed by the VSphere Client (free, and downloadable from the server itself, once it’s been installed properly). It’s a lovely GUI tool that makes creating, snapshotting, performance monitoring and generally managing multiple guest VMs a piece of cake.

I am very enthusiastic about the final outcome at work (especially considering the juggling I had to do to get there  -moving databases all over the place multiple times to free up the server, so it could be rebuilt, then moving them all back again… and all without disrupting important testing and development efforts going on at the moment, for example). I was, of course, enthusiastic about XenServer too, before I bumped into its own bunch of quirks and ‘features’ that proved show-stoppers. Time will tell if the same is true for ESXi… but it’s not looking too bad at the moment!

I’ve been doing virtual machines since 2000, which makes me a bit of an old-timer, I would think. I invented the laptop rac using a single virtual machine and taught it in Oracle University classrooms from about 2002 on, which was something of a world first. I’ve been banging on about the virtues of virtualisation for a long time… you’d think I’d therefore get it right when it comes to putting it into production, wouldn’t you? But no. I’ve stuffed up. In fact, I’ve behaved like I’d never met a virtual machine before. A virtualised virgin, if you like.

Beguiled by the swift, elegant simplicity of the Citrix XenServer installation and included management tools, I’ve persuaded the powers that be at work to dedicate a 32GB, 8 CPU monster server to running XenServer. I’ve even persuaded them to do something of a first this financial year and part with about AU$6000 so that I can upgrade the storage from a paltry 400GB to a more useful 3000GB, all with a view to combining every test and dev database we’ve got onto a single server, but running in discrete, virtual servers for the purpose of easy snapshotting, backing up, migration and so on.In the process, we would free up two physical servers, one of which could house a small database that all our other databases link to for ‘globally unique’ information. That would free up 6GB of RAM on one of our other servers, thus enabling the two databases running there to increase their buffer caches and get their web response times down further from their already pretty good 0.6 seconds. Wins all round.

Except I chose Citrix XenServer.

It’s not a bad virtualisation product. It’s just immature compared to the likes of VMware. A core requirement was the easy ability to create snapshots, to then navigate back and forth up and down the snashot tree as required. XenServer is bloody awful at snapshotting anything (promised to be fixed in the next release). Another core requirement was the ability to export a virtual machine for backup purposes: one of my 80GB VMs took 11 hours to export with Citrix. On VMware ESXi, it takes 36 minutes. God knows why, but the difference -functionally- is enormous. Finally, as reported in these parts yesterday, XenServer throws a complete mental fit when confronted with Centos 5.4; ESXi just took it in its stride. (Oh, and as another aside, XenServer refuses to run Windows in a VM unless hardware virtualisation extensions are present; ESXi doesn’t appear to give a stuff, and runs them just fine and with seemingly perfectly acceptable performance results).

The XenServer is going, in other words. And we’re switching to ESXi.

I have my problems with ESXi, too, which I’ll no doubt document before long. The fact that its GUI management tool doesn’t run on Windows 7 was a bit of a killer, for example, but I have found a way around that. The licensing is also a complete pig’s breakfast and I’m still trying to get my head around that. But, all in all, ESXi does what we needed virtualisation to do, today, whereas Citrix XenServer doesn’t and won’t until the proverbial “next release”.

I apologise to my reader(s) for having eulogised a fundamentally dud technology (at least as far as we are concerned at work) for far too long. Normal service will be resumed soon, I hope!

I mentioned last time that Citrix Xen Server was incredibly fast and easy to setup, but a bit of a pain to use, practically, because the Centos VM I created on it insisted on running in strictly command-line mode only. All true enough… but a GUI is do-able, and here’s how I did it.

First, I read the available documentation! The relevant bit is here, which references a bit about installing guest agent software, which is documented here.

The basic outcome is that, in console (text) mode, you install the Xen Agent software for VM Guests first (that’s into the Centos 5.3 x64 virtual machine I’d already built in text mode, of course). You do that by logging on as root at the console, switching  the DVD selector so that it reads xs-tools.iso and then typing the commandmount /dev/xvdd /mnt. You’ve just effectively inserted the Xen Tools CD into your virtual CD-ROM.So now you can launch the relevant installer off that virtual CD by typing the command /mnt/Linux/install.sh. Say ‘yes’ when prompted and then reboot the virtual PC.

Next, you need to edit the Centos virtual machine’s /etc/gdm/custom.conf file (remember, I only deal with Centos 5.3 these days: the file is different for Centos 3 and 4 versions). I edited that file to contain the following:

The [servers] bit is already in the file, but everything else has to be typed in, using your favourite text editor (nano in my case). The command= line continues without a break all the way through to the end -BlacklistTimeout 0 bit. The ‘flexible=true’ stuff is a new line in its own right.

With that file changed and saved, I then had to switch off the Centos firewall -something I’d normally do automatically in a GUI OS install, because the installer prompts you. But in a CLI environment, you have to take that step yourself. Logged in as root, type the command system-config-securitylevel-tui and -using your skill and judgement- press whatever combination of TAB, Space Bar and ENTER switches off the firewall in its entirety. You should also set SELinux to at least permissive if not outright disabled (because Oracle doesn’t interact with SELinux very well). Note that if you set SELinux to disabled, you should reboot the VM so that the new setting can take effect. If you set it to permissive, a reboot is not required.

Finally, you can now launch the Gnome Desktop Manager by simply typing the command gdm. If for some reason it’s already running. just ype  /usr/sbin/gdm-restart instead. In either case, you should see the Switch to Graphical Console button light up in the Xen Management tool on your Windows client PC. Click that, and you’ll be rewarded with  graphical Centos login screen -and a short while later, the glory that is this:

You will note from that screenshot that I’m busy running the Goal script for automated Oracle installations… which gives you a clue that Oracle on-Centos-on-Xen is a piece of cake. And which might, I suppose, be the subject of another post or several in the coming days! (It completes in 19 minutes, which as these things go is pretty darn’d quick, and very close to physical PC installation times. Impressive).

I have to say as a parting note that it’s a shame that Citrix Xen makes having a graphical Centos experience a matter of editing a couple of textfiles and keeping your fingers crossed. But I cannot fault the fact that it is a matter of typing just one or two commands and then the pain is behind you. Given the capability of this software, and its extraordinary price of US$0.00, I can live with it -especially as there are so many other redeeming features!

It’s been a “long weekend” here in Sydney: today is a public holiday (though the occasion for it escapes me… perhaps something to do with Labour Day? Who knows!)

I’ve spent most of the day finalising my server setups. Now that my new desktop has arrived, my old quad core becomes The Other Half’s PC, and TOH’s old PC becomes my new ’spare server’. It’s not bad as these things go: dual core Core 2 Duo, 3GHz, 4GB RAM, 1TB hard disk space. So I thought I’d try a bit of free (as in zero cost) bare-bones virtualisation and see how I got on.

I started with Citrix Xen Server 5, then Citrix Xen Server 5.5 (the current release). I moved on to Vmware’s ESXi 3.5 and then Vmware’s ESXi’s 4.0 (again, the current release). I tried Hyper-V (Microsoft’s offering in this virtual space). And I wrapped up with a dabble with Oracle’s own VM-Server, version 2.5.

I’ll save the gory details for a later post, because it’s worth doing a proper side-by-side comparison of these things. But the short story is: Xen Server 5.0 installed extremely well, but when trying to create a Centos 5.3 x86_64 virtual machine, the Centos installer lost contact with the Xen Server’s DVD drive immediately after booting, rendering further installation impossible. Call that the deadest of dead-ends, then! This didn’t happen with the more recent Xen Server 5.5, however, and I was therefore able to install the Xen Server, its Windows client tools and create a Centos 64-bit virtual machine in precisely 27 minutes flat… which is amazing, I think, as these things go (especially as I was cooking dinner at the time). The only slight problem was that the Centos installation refused to install graphically and subsequently refused to run X at all… so it was a command-line only affair. That’s not the end of the world, of course, since it’s possible to run an X server on your Windows machine and have a graphical experience of a console-only virtual machine. But it’s not ideal.

ESXi Server was an extremely quick install, but it’s another 100MB download to get hold of the Windows remote administration client which is mysteriously called V-Sphere …and which, as it turns out, doesn’t actually run on Windows 7! There are workarounds: they involve downloading a dll which could contain absolutely anything from a website whose trustworthiness you have to take on, er, trust. And then you have to hack around a few XML files and keep your fingers crossed. Eventually, it does all work, but the experience is deeply, deeply nasty and not one I’d recommend.

Hyper-V was a success, in the sense that I had a working Centos 5.3 virtual machine within an hour and a half… but the graphics were all messed up, with the Centos machine needing to be set to an 800×600 display before it would display correctly on an old 1280×1024 monitor. Set it actually to display at 1280×1024 and it looked more like 3856×1900… very, very large and unusable! If you didn’t mind working in a tiny screen, or forever scrolling around an enormous one, I suppose you could call it workable. But yet another download (this time, 200MB+) was needed to get the Remote Server Administration Tools for Windows 7 installed… only then to discover that no matter what one did, you just kept being told that you lacked the permissions necessary to administer the virtual server. Apparently, if you faff around with assorted Group Policies and DCOM permissions, you might get it all working, but I never did. So, if you don’t mind running everything direct off the virtual server itself, no problemo. But if you want to manage it from your Windows 7 desktop, forget it, basically.

Oracle’s own virtualisation server was slick enough… but incredibly fiddly to use, with the need to pre-provision everything (such as the Centos installation disks) made incredibly difficult. (Think of it like having to pre-declare all your variables before being allowed to use them!) It doesn’t help that there’s still no Windows client software for remote administration of the virtual machines: so you’re reduced to creating a virtual Linux machine on your Windows desktop so you can administer a virtual Linux machine on your Oracle VM Server! Pathetic, to be honest, and far more trouble than it’s worth. Especially since installing the ‘client’ software involves an incredibly adventurous and lengthy install of a complete Oracle XE database server, plus a mountain of Oracle Application Server componentry. I kept seeing visions of sledgehammers and nuts, to be honest. After all that, there’s not even the facilty to just say, ‘please install from the DVD what I’ve just installed in the Virtual Server’s physical DVD drive’. Oh no. Nothing that simple. Various import options are available to make an already-obtained ISO usable as a VM installation source, but none of them worked for me. My server just sat there, promising me mountains of virtual pleasure, but denying me any of it. Meanwhile, I’d been forced to create an additional amply-apportioned virtual server simply to act as the managing remote client for the server I’d intended to create. Bizarre in the exteme.

Short story, then: Windows Hyper-V works, but remote administration is a bugger. Xen Server works incredibly simply, swiftly and elegantly…but without X, it’s a more complex command-line only experience (update: but this can be fixed!). ESXi is OK, but seems fiddly, and doesn’t have a good Windows 7 client. And Oracle’s own Virtual Server offering is a complete load of unmanageable nonsense as far as I’m concerned.

A detailed write-up to follow, in other words. Meanwhile, a lot of fun, even more frustration… and some surprising outcomes. (I had expected Oracle’s offering to be a lot more elegant than it was, for example!)