Tag Archives: VMware

Add a dash of software…

In my last-but-one post, I explained how you could build a ‘Linux toolbox’: a RCSL server that acts as web server, DNS lookup server, NFS server and more. I also showed how you could do all of that with barely any manual intervention, thanks to the magic of Kickstart scripts.

Well, now I want to suggest a couple of additions to really round out your toolbox’s functionality. These can’t really be automated, because they involve agreeing to licences and export restrictions and so forth.

1. Add some useful binaries

Your toolbox server is lacking a GUI, so monitoring its CPU, disk I/O and memory usage might be tricky. Fortunately, nmon is a command-line utility that monitors all these things easily. So adding that to your toolbox server might be a good idea. It can be obtained as an rpm suitable for all of the RCSL 6.x, 64-bit distros from this site.

You may also feel you need a graphical way of re-configuring things like hostnames, IP addresses, DNS settings and so on. A useful tool for that is webmin, which is a browser-based way of doing precisely that. There’s an rpm suitable for all RCSL distros available on this page.

Once you install it, just fire up a browser in a client PC and point it at your toolbox server’s IP address, specifying port 10000. In my case, that means typing into a browser’s URL bar something like http://192.168.8.250:10000

After that, it’s just a matter of logging on (as root) and then clicking around seeing what is available.

I’m going to suggest two additional binaries you might want to download and store in the toolbox’s /var/www/html directory -not because they are useful to the toolbox itself, but because they might prove useful later on. First, the public domain korn shell is largely redundant these days, but happens to be a prerequisite for Oracle installs on RCSL distros, If you don’t install it, you can therefore get annoying warning messages from the Oracle installer. Fortunately, a version of it is available for download as an RCSL 64-bit compatible RPM from here.

The final binary I suggest you download is rlwrap. You can download it as a tarball, extract it and then compile it if you like, but there’s an RPM version available for download which is a bit simpler to get working, so I’m going to suggest you get that and store it in your toolbox server’s web document root directory instead.

Of course, if you’ve been following this series of posts, you’ll know that my toolbox server, by design, doesn’t have a network connection to the Internet, so downloading those things using the toolbox itself is not possible. That’s why I download them all on a different machine that does have an Internet connection and then burn them onto an ISO (a virtual DVD). (I use Brasero to do that, but any DVD burning software will do the job). Once the four binaries (nmon, webmin, pdksh and rlwrap) are stored on the virtual DVD, it’s trivial to get the toolbox server to mount the ISO and copy them all off to the /var/www/html directory. Once there, you can install nmon and webmin on the toolbox itself:

rpm -Uvh nmon-14f-1.el6.rf.x86_64.rpm
rpm -Uvh webmin-1.620-1.noarch.rpm

The other two will come in handy later on. If you can’t burn your own ISOs, here’s one I prepared earlier…

2. Add some Oracle files

The other software which it would be useful to copy to the toolbox, so it can be served up to clients later on, is the Oracle database software itself. I can’t give you a handy link to those: you’ll have to get them from either Oracle’s own otn or edelivery sites.

If you have paid-for support, you could obtain the 11.2.0.3 version of the Oracle database: you’ll have files called things like “p10404530_112030_Linux-x86-64_1of7.zip” and “p10404530_112030_Linux-x86-64_2of7.zip”.

If you download the freebie versions from OTN, you’ll have to make do with the 11.2.0.1 version, and you’ll end up with file names like “linux.x64_11gR2_database_1of2.zip” and “linux.x64_11gR2_database_2of2.zip”.

If you do your downloads from edelivery without a paid-for support contract, you’ll again be limited to the 11.2.0.1 version of the database, but this time you’ll end up with file names such as “V17530-01_1of2.zip” and “V17530-0_2of2.zip”.

Since all the versions and file name variations can get confusing, I’m going to suggest that whatever files you download, you rename them to be in the form: oradb-1120x-1of2.zip, where “x” is the relevant bit of the version number. So, having paid-for support, I’m going to end up with oradb-11203-1of2.zip; if I were only able to access OTN freebies, I’d end up with oradb-11201-1of2.zip, and so on.

Once you’ve download and re-named your Oracle files, again burn them onto a virtual DVD iso file, get your toolbox server to mount it, and copy all the zips it contains to the /var/www/html directory. Once there, the zip files will be able to be used by other servers you build that need them.

3. Conclusion

Having done all of the above, when I list the contents of my toolbox server’s /var/www/html directory, here’s what I see:

The “sl” directory (in blue) is there because, you’ll remember, my Kickstart script causes the toolbox to copy its own installation media to disk -and this happens to be a Scientific Linux box.

The four files in red are the little utilities that the toolbox itself, and any server it’s later used to help build, might find useful. Obviously, there are no Oracle database files there yet, but a bit of downloading and renaming as described earlier results in this:

Now you see the 2 Oracle database zip files sitting there (these are the freebie versions, so they’re 11.2.0.1).

The significance of all this? Well, you’ve now got yourself a toolbox that is almost ready to do completely hands-free Oracle installations on other RCSL servers. I’ll explain the extra ingredient needed next time…

Putting it all together

In the previous sequence of posts, I’ve documented how you can build a RCSL server that can act as a DHCP, DNS, NFS, Web and Time server.

But why do each one in isolation? Why not build a single server that is configured to be all of those things, so that when all is done, it acts as giant ‘toolbox’ of network services to your entire network? Actually, there is a good reason not to do that, called “single point of failure”. But in a home network/test lab environment, I don’t think that’s an important consideration. So the all-in-one toolbox approach sounds good to me!

You’ll notice from all those previous posts, though, that setting up each of these network services often takes a lot of typing -and thus has a high risk of getting something wrong.

So can we automatically build a single server that can provide all these services, configured correctly ‘out of the box’? Sure we can: and it’s our old friend Kickstart that will work this particular brand of magic. And here is a Kickstart script that will do everything discussed in the previous 6 posts, all in one, completely automatically:

#  =======================================================================
#  This Kickstart script written by Howard Rogers. 
#  Copyright (c) 2013 Howard Rogers,Dizwell Informatics
#  The script is supplied "as-is" with no warranties or guarantees of 
#  fitness of use or otherwise. Neither Dizwell Informatics nor 
#  Howard Rogers accepts any responsibility whatsoever for any 
#  damage caused by the use or misuse of this script.
#  =======================================================================

install
text
cdrom
lang en_US.UTF-8
keyboard us
network --onboot yes --device eth0 --bootproto static --ip 192.168.8.250 --netmask 255.255.255.0 --noipv6 --nameserver 192.168.8.250 --hostname marconi.dizwell.home
firewall --disabled
authconfig --enableshadow --passalgo=sha512
selinux --disabled
timezone Etc/GMT
bootloader --location=mbr --driveorder=sda --append="crashkernel=auto rhgb quiet"
clearpart --all
part / --fstype=ext4 --size 20000 --grow
part swap --size 1024

%packages
@base
@core
@nfs-file-server
@network-server
@system-admin-tools
@web-server
@server-platform
bind
dhcp
nfs-utils
ntp
crypto-utils
-vconfig
-setuptool
-kexec-tools
-sos
-wireless-tools
-system-config-network-tui
-mdadm
-dmraid
-hunspell-en
-tcsh
-abrt-cli
-scl-utils
-systemtap-runtime
-fprintd-pam
-dosfstools
-smartmontools
-rfkill
-blktrace
-pinfo
-system-config-firewall-tui
-words
-abrt-addon-kerneloops
-abrt-addon-ccpp
-bridge-utils
-mtr
-pcmciautils
-usbutils
-ledmon
-hunspell
-abrt-addon-python
-man-pages
-b43-fwcutter
%post
chkconfig iptables off

# Configure NFS Server
mkdir /griddata
mkdir /dbdata
echo "/dbdata 192.168.8.0/24(rw,sync,no_wdelay,insecure_locks,no_root_squash)" > /etc/exports
echo "/griddata 192.168.8.0/24(rw,sync,no_wdelay,insecure_locks,no_root_squash)" >> /etc/exports
chkconfig nfs on
service nfs start

#Configure DHCP Server
echo "option domain-name            \"dizwell.home\";" > /etc/dhcp/dhcpd.conf
echo "option domain-name-servers    192.168.8.250;" >> /etc/dhcp/dhcpd.conf
echo "default-lease-time   600;" >> /etc/dhcp/dhcpd.conf
echo "max-lease-time 6000;" >> /etc/dhcp/dhcpd.conf
echo "authoritative;" >> /etc/dhcp/dhcpd.conf
echo "subnet 192.168.8.0 netmask 255.255.255.0 {" >> /etc/dhcp/dhcpd.conf
echo "range dynamic-bootp 192.168.8.200 192.168.8.240;" >> /etc/dhcp/dhcpd.conf
echo "option broadcast-address 192.168.8.255;" >> /etc/dhcp/dhcpd.conf
echo "}" >> /etc/dhcp/dhcpd.conf
chkconfig dhcpd on
service dhcpd start

#Configure DNS Server
echo "options {" > /etc/named.conf
echo "    directory \"/var/named\";" >> /etc/named.conf
echo "    dump-file \"/var/named/data/cache_dump.db\";" >> /etc/named.conf
echo "    statistics-file \"/var/named/data/named_stats.txt\";" >> /etc/named.conf
echo "    memstatistics-file \"/var/named/data/named_mem_stats.txt\";" >> /etc/named.conf
echo "    allow-query { localhost; 192.168.8.0/24; };" >> /etc/named.conf
echo "    recursion yes;" >> /etc/named.conf
echo "    dnssec-enable no;" >> /etc/named.conf
echo "    dnssec-validation no;" >> /etc/named.conf
echo "    dnssec-lookaside auto;" >> /etc/named.conf
echo "    /* Path to ISC DLV key */" >> /etc/named.conf
echo "    bindkeys-file \"/etc/named.iscdlv.key\";" >> /etc/named.conf
echo "    managed-keys-directory \"/var/named/dynamic\";" >> /etc/named.conf
echo "    };" >> /etc/named.conf
echo "    logging {" >> /etc/named.conf
echo "        channel default_debug {" >> /etc/named.conf
echo "       file \"data/named.run\";" >> /etc/named.conf
echo "       severity dynamic;" >> /etc/named.conf
echo "         };" >> /etc/named.conf
echo "    };" >> /etc/named.conf
echo "    zone \".\" IN {" >> /etc/named.conf
echo "         type hint;" >> /etc/named.conf
echo "         file \"named.ca\";" >> /etc/named.conf
echo "    };" >> /etc/named.conf
echo "    zone \"dizwell.home\" IN {" >> /etc/named.conf
echo "        type master;" >> /etc/named.conf
echo "        file \"dizwell.hosts\";" >> /etc/named.conf
echo "    };" >> /etc/named.conf
echo "    include \"/etc/named.rfc1912.zones\";" >> /etc/named.conf
echo "    include \"/etc/named.root.key\";" >> /etc/named.conf

echo "\$TTL 86400" > /var/named/dizwell.hosts
echo "@ IN SOA marconi.dizwell.home. hjr.dizwell.com. (" >> /var/named/dizwell.hosts
echo "2013030501 ;Serial" >> /var/named/dizwell.hosts
echo "3600 ;Refresh" >> /var/named/dizwell.hosts
echo "1800 ;Retry" >> /var/named/dizwell.hosts
echo "604800 ;Expire" >> /var/named/dizwell.hosts
echo "86400 ;Minimum TTL" >> /var/named/dizwell.hosts
echo ")" >> /var/named/dizwell.hosts
echo "" >> /var/named/dizwell.hosts
echo "        IN NS   marconi.dizwell.home." >> /var/named/dizwell.hosts
echo "        IN A    192.168.8.250" >> /var/named/dizwell.hosts
echo "" >> /var/named/dizwell.hosts
echo "marconi         IN A    192.168.8.250" >> /var/named/dizwell.hosts
echo "alpher          IN A    192.168.8.101" >> /var/named/dizwell.hosts
echo "bethe           IN A    192.168.8.102" >> /var/named/dizwell.hosts
echo "alpher-priv    IN A    10.0.0.1" >> /var/named/dizwell.hosts
echo "bethe-priv    IN A    10.0.0.2" >> /var/named/dizwell.hosts
echo "alpher-vip      IN A    192.168.8.111" >> /var/named/dizwell.hosts
echo "bethe-vip       IN A    192.168.8.112" >> /var/named/dizwell.hosts
echo "racscan         IN A    192.168.8.200" >> /var/named/dizwell.hosts
echo "racscan         IN A    192.168.8.201" >> /var/named/dizwell.hosts
echo "racscan         IN A    192.168.8.202" >> /var/named/dizwell.hosts
echo ";; end of zone" >> /var/named/dizwell.hosts
echo ";;" >> /var/named/dizwell.hosts

echo "search dizwell.home" > /etc/resolv.conf
echo "nameserver 192.168.8.250" >> /etc/resolv.conf
chown named:named /var/named/dizwell.hosts
#service named start
chkconfig named on

#Configure NTP Server
echo "server 127.127.1.0" > /etc/ntp.conf
echo "fudge 127.127.1.0 stratum 2" >> /etc/ntp.conf
echo "driftfile /var/ntp/ntp.drift" >> /etc/ntp.conf
echo "statsdir /var/ntp/ntpstats/" >> /etc/ntp.conf 
echo "filegen peerstats file peerstats type day enable" >> /etc/ntp.conf
echo "filegen loopstats file loopstats type day enable" >> /etc/ntp.conf
service ntpd start
chkconfig ntpd on

#Configure Web Server

if [ `lsb_release -d | grep 6.4 | wc -l` -eq 1 ]; then
  VERSION=64
elif [ `lsb_release -d | grep 6.3 | wc -l` -eq 1 ]; then
  VERSION=63
else
  VERSION=6
fi

DISTRO=$(lsb_release -is)
if [ $DISTRO = "CentOS" ]; then
  mkdir -p /var/www/html/centos/$VERSION
  DISTRO=centos
elif [ $DISTRO = "OracleServer" ]; then
  mkdir -p /var/www/html/oel/$VERSION
  DISTRO=oel
elif [ $DISTRO = "Scientific" ]; then
  mkdir -p /var/www/html/sl/$VERSION
  DISTRO=sl
else
  mkdir -p /var/www/html/unknown/$VERSION  
  DISTRO=unknown
fi

mount /dev/sr0 /media
cp -rvf /media/* /var/www/html/$DISTRO/$VERSION
umount /media
service httpd start
chkconfig httpd on

%end

This script works with any of the mainstream 64-bit RCSL distros: it’s been tested on 64-bit CentOS 6.3 and 6.4, Scientific Linux 6.3 and 6.4 and Oracle Enterprise Linux 6.3 and 6.4,

To use it, obviously you first need appropriate O/S installation media. Use the full DVD media that allows fully-functional installs from a single DVD, because the script can’t cope with changing DVDs half-way through an installation. Here are where I sourced my test ISOs from:

Centos 6.3 http://mirror.aarnet.edu.au/pub/centos/6.3/isos/x86_64/CentOS-6.3-x86_64-bin-DVD1.iso
Centos 6.4 http://mirror.aarnet.edu.au/pub/centos/6.4/isos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso
Scientific 6.3 http://mirror.aarnet.edu.au/pub/scientific/6.3/x86_64/iso/SL-63-x86_64-2012-08-02-Install-DVD.iso
Scientific 6.4 http://mirror.aarnet.edu.au/pub/scientific/6.4/x86_64/iso/SL-64-x86_64-2013-03-18-Install-DVD.iso
Oracle 6.3 V33411-01.iso from edelivery or http://mirror.aarnet.edu.au/pub/oraclelinux/OL6/U3/x86_64/OracleLinux-R6-U3-Server-x86_64-dvd.iso
Oracle 6.4 V37084-01.iso from edelivery

(Red Hat installation media are only available to paying customers, so I figure you’ll already know where to go on the Red Hat portal to get the appropriate ISO!)

The Kickstart script itself then needs to be copied to somewhere that your “toolbox server” can find it during O/S installation: a floppy disk is good, but they are getting a bit thin on the ground, physically, these days! A USB stick might do the job, too. But, for my money, the simplest approach is to burn it onto a CD (or ISO CD image). If 700MB or 4.7GB for a single text file seems a bit extravagant, so be it: at least it works. :-)

Here is an ISO file containing the above script. Burn it to a physical CD/DVD if you like, or use it as an ISO file to help boot a virtual machine. Of course, the Kickstart file it contains is hard-wired to build a server called Marconi with an IP address of 192.168.8.250, as per my previous posts. If you want a different configuration, you’ll have to build your own ISO having edited the script, first.

This Kickstart-on-DVD arrangement would mean, of course, that your physical server would need two DVD drives installed: the first to hold the O/S DVD media and the second to hold the Kickstart file. That might be a tall order on a physical server (though USB external DVD drives could potentially come to the rescue), but it’s easy on a virtual machine: just add the extra hardware at the time you create the new virtual machine:

That’s in VirtualBox: note that the full Linux installation disk is listed before the toolbox.iso: that’s important, because the boot disk has to be the first in order of boot priority. If the toolbox disk is listed first, you’ll be trying to boot from a text file… and that won’t work! Just alter the “IDE Primary/Secondary Master” and “IDE Primary/Secondary Slave” drop box against each disk until your boot disk is primary master and your toolbox ISO is listed as primary slave or secondary master/slave.

After that, you just have to intervene briefly at boot time:

As for any Kickstart installation, you simply press TAB when the boot welcome menu appears. That displays some existing text (vmlinuz initrd=initrd.img) and all you have to do is, as you see here, append a bit of information to say where your Kickstart file is. Mostly, that just means correctly identifying the device it’s stored on.

If your Kickstart file was stored on a floppy disk, the device identifier would be hd:fd0 (that’s a zero, not an oh). Since mine is on a secondary CD drive, my device identifier is as you see it here: hd:sr1. In either case, the Kickstart file itself is called “toolbox.ks”, so you just stick that on the end of the appropriate device identifier. In my case, I end up with:

hd:sr1/toolbox.ks

The hardest part is working out your device names -and that might take a bit of trial and error. But once you get it right, your involvement in the rest of the installation will be absolutely minimal:

  • You may be prompted to agree to having your hard disk wiped, but only if it’s a completely new hard disk that has never been partitioned before. If you’re re-using a pre-loved hard disk, it will be wiped without any prompting.
  • You will be prompted once for a root password (typed in twice).

Other than that, the entire installation will complete automatically, and all the network services discussed over the past 6 posts (DHCP, DNS, NFS, NTP and Apache) will be auto-configured without any manual intervention on your part. At the end of the installation, therefore, you’ll be owner of a self-contained, highly functional Linux ‘toolbox’.

Remember, if you’re doing this in a Virtual Machine, that I’m expecting your toolbox to have just 512MB RAM, at least a 25GB hard disk and to be using Host-only networking.

To show you how painless it all is, here’s a video of me doing practically nothing! (I’ve cut out nothing, but I suggest you play it back at 2x or 3x speed, so that it’s not quite as boring as it was originally!).

Key features of the movie:

  • At ~0:38, I am asked to re-initialise the disk, because this one had never been used before. I select the Re-initialize all option.
  • At ~0:44, I supply a root password (twice)
  • At ~3:33, the installation process ends and the post-installation scripts are run. This involves copying the installation media itself from DVD to hard disk. Since about 4GB of data is involved, it takes a while to complete.
  • At ~5:00, I am asked to reboot. Once the machine does so, I log on as root and demonstate that all the network services are up and running -after precisely three bits of user input.

line2

Assuming you can view the toolbox.ks file in a text editor that can display line numbers, here are a few observations about the file, referenced to the various lines.

Line 11: The RCSL install that uses this Kickstart file will do a text-mode install. This is good, because a server providing all the network services I’ve mentioned only needs around 512MB of RAM -but that’s not enough memory to do a graphical installation.

Line 15: The Server will end up part of the dizwell.home domain (you can obviously edit this to suit), and have an IP address of 192.168.8.250.

Line 16: The Firewall is disabled. It’s a blunt approach: we should more properly open up ports 80, 53 and so on. But this is simpler: and it works!

Line 18: SELinux is disabled. SELinux afficionados will disapprove, but again, simply disabling it means everything works without fuss.

Lines 21 – 23: The server will expect a hard disk of at least ~25GB. Most of that will be assigned to a single root partition at least 20GB in size (but it can grow bigger than that if the disk space is there); a 1GB swap partition is also created. Note that this will wipe any hard disk it’s pointed at… making it lethal. If you don’t want a hard disk wiped, don’t use this script!

Lines 25 – 70: The bare minimum of software is installed. You’ll note that NFS, Apache, Bind (DNS), DHCP and NTP are all added to the mix, though.

Lines 73 – 79: Pretty much a re-hash of my recent ‘quickly build an NFS server’ post

Lines 81 – 92: Pretty much a re-hash of my recent ‘quickly build a DHCP server’ post

Lines 94 – 155: Pretty much a re-hash of my recent ‘quickly build a DNS server’ post

Lines 157 – 165: Pretty much a re-hash of my recent ‘quickly build an NTP server’ post

Lines 195 – 196: Getting the Apache webserver running and ensuring it runs at every server reboot thereafter, as explained in the ‘quickly build a webserver’ post.

Lines 192 – 194 Potentially the most important lines of the lot, because they ensure that the server copies its own installation DVD to hard disk -thus making it able to host future over-the-network installs of other RCSL servers. As future posts here will explain, that capability is going to become significant.

Splendid Isolation

Here is my home network (or a bit of it, anyway):

Diagram1

(I apologise in advance for lack of abilities when it comes to using Dia, the Linux equivalent of Visio!)

What you have there is not very exciting: a couple of Centos-running servers called NEWTON and GALILEO, one of which connects to the Internet, thanks to a wireless dongle and the magic of 3G. There’s a wireless access point, handling all the “devices”: Nexus 7, Kindles, Smartphones and the like. Then there’s FEYNMAN, ToH’s Windows 8 PC doing sterling duty as a Photoshop workstation. RUTHERFORD’s the media centre (also running Windows 8, but with Windows Media Centre added). And my new PC is DIRAC, running Fedora 18. Everything is running on a 192.168.0… subnet.

I probably should expand this diagram, though: there are 7 different virtual machines which run routinely on Dirac: they all share the 192.168.0 subnet thanks to ‘bridged networking’ in VMware Workstation and can therefore access Galileo and, from there, the Internet. Though they are merely virtual machines in fact, they behave exactly as real, physical machines would: by sharing the same subnet as the PC they run on, they look just like real machines to all the other real machines on the network. Bridged networking is fine for VMs that are allowed to function as though they were real.

But here’s the problem: what if I want a bunch of new virtual machines to be able to talk amongst themselves as though they were part of a normal network but without any of the existing, physical machines being aware of the fact? For example, I may want to build a DHCP server -but I don’t want any of my ‘real’ PCs accidentally picking up their IP address from it. Similarly, I  may want to build a new DNS server -but I wouldn’t want FEYNMAN trying to look up hostnames there.

The underlying reason for needing this ability is, in fact down to my laptop (which isn’t shown in the above diagram). I use it at work, so it needs to see proper, commercial DNS, DHCP, NTP and other servers -but if I want to run VMs on my laptop that are running their own DNS, DHCP, NTP and other servers, I have to be able to ensure that whatever I build can never, ever “leak” out onto the real network. I need, in fact, complete isolation of my network of virtual machines running on my laptop from the real corporate network my laptop is physically connected to.

Fortunately, this is easy to do with VMware Workstation -and VirtualBox: both employ a feature misleadingly called host-only networking. I say ‘misleadingly called’ there, because that name implies (to my mind, anyway) that virtual machines making use of it can only communicate with the physical host. If that were true, you could ‘network’ the host to the VM -but you’d be stuffed trying to network one VM with another VM. But happily, it’s the name that’s wrong, not the functionality: host-only networking in both virtualisation products actually means that any VM built to use the feature can communicate with the physical host and with every other host-only VM, too.

So here’s the network I want to build:

Diagram2

The virtual network is all the grey-coloured stuff at the bottom. My physical PC, DIRAC, will host six different virtual machines, all running in host-only mode, and using the 192.168.42 subnet. Note that DIRAC itself acquires a second IP address, allowing it and it alone both to connect to the rest of the world and to connect to any of the 192.168.42 virtual machines.

How exactly is that to be achieved, then? Well, if I persist with my usual desktop virtualisation software (VMware Workstation), I’d find the menu option for the Virtual Network Editor (in Linux, you can launch it from the command-line by typing /usr/bin/vmware-netcfg -you’ll be prompted to supply root’s password if you’re not root at the point of issuing that command):

When you install VMware Workstation, you get three different network interfaces created for you automatically: one of them is used for bridged networking, one for Network Address Translation …and one for host-only networking. Highlight that last one and you’ll be able to alter its properties in the lower part of the screen. Here you see that I’ve switched off the virtual DHCP server that VMware offers to run for you on this interface; I’ve also altered the default subnet IP so that it matches my desired ’192.168.42…’ subnet address range. Leave the last octet of that address set to zero and you will find that your physical host PC or laptop is automatically assigned a 192.168.42.1 address, which is fine.

You can create as many of these host-only network interfaces as you like: each one will create a new network adapter (if your host is running Windows) or a new network interface, visible via ifconfig, if your host runs Linux), meaning that if your host PC is powerful enough, there’s nothing to stop it running multiple, isolated virtual networks at the same time.

VirtualBox does it slightly differently: just use the File -> Preferences menu options to bring up a Settings dialog, then select the Network item:

By default (at least on Fedora 18!), you don’t have any host-only network adapters installed here, but if you click that little green ‘plus’ button, a new one will be created for you and will be automatically named virtualbox0. After creating it, click the third little icon on the right to configure its settings:

Again, just type in a ‘starting address’ for your new isolated network. With VirtualBox, I find it necessary to specify the ‘.1′ at the end of the IP address, otherwise it doesn’t work properly. By specifying the 192.168.42.1 address in full, therefore, I ensure my physical host is assigned that IP address on its new virtualbox0 interface. (As you can see from that last screenshot, too, I don’t have much use for IPv6 at the moment!)

Finish things off by switching to the DHCP Server tab and uncheck the ‘Enable Server’ option: again, VirtualBox proposes to run a virtual DHCP server for you, which you might find desirable, but which I definitely don’t!

Incidentally, if you happen to have VMware Workstation and VirtualBox installed on the one physical guest, make sure each uses a different isolated subnet, because they can’t both be configured to use the same one meaningfully! In other words, if I’ve created a 192.168.42.x host-only subnet in VMware, I’ll have to create a 192.168.43.x host-only subnet in VirtualBox. Networking won’t work reliably if you try assigning the same IP address to two different interfaces at the same time!

Once your network interfaces are configured properly, it’s simply a matter of choosing the right sort of networking for each new Virtual Machine that you build. In VMware, you do that as part of the ‘create new VM’ wizard:

By default, VMware threatens to create new VMs with a NAT interface, but it’s easy to switch to using a host-only one.

VirtualBox is not as easy, because the wizard you use to create a new VM doesn’t offer you any opportunity to specify anything about the network interface to use at all. It’s therefore necessary to create a new VM ‘blind’, and after it’s been created, right-click it and select Settings. Select the Network item on the left:

You’ll discover that, once again, a NAT interface has been assigned to your new VM by default. Just click that combo-box, though, and you’ll find an item for Host-only adapter. Select that, click OK, and you’re all done.

Once built, you’ll find that all your host-only VMs can ping each other and your physical PC on which they’re running -but they won’t be able to ping beyond your physical PC. That means, in my case, that ALPHER and BETHE can ping each other, or MARCONI, TESLA or DIRAC (my physical PC). But they won’t be able to ping GALILEO -and since GALILEO controls access to the Internet, that means my host-only VMs can’t reach the Internet, either.

For my purposes, that’s fine: why would I need an Oracle server (say) to be able to access the Internet? You wouldn’t normally do that in production (I hope!), so having a set of servers that behave similarly is not a problem for me. It does mean I have to have one of my VMs hosting a software repository from which my other VMs can download packages and so on necessary for running Oracle (or anything else), but that’s usually not difficult to do.

The main issue is easily solved, therefore: host-only networking allows your virtual machines and all the networking services they provide and consume to be completely isolated from the rest of the world -and any real corporate networks your physical PC might be attached to at the time! Splendid!!

(Oh, and as a side benefit: my laptop will be able to run a virtual network of six VMs even when I’m running it on a train and it’s stuck in a tunnel without a 3G signal. The virtual network will keep running just fine, even if the physical one decides to die -or even if you don’t have a physical network at all. Host-only VMs are a great way of virtualising networks on standalone PCs, in other words).

VMware Blues

VMware Workstation is annoying me. A recent Fedora 18 upgrade took the kernel to version 3.8.1-201, and a change in kernel necessitates a re-compilation of the bits and bobs that make VMware work (kernel modules and the like). Often, that is not a problem -but, far too often, it is, and such was the case this time. I suspect an upgrade to gcc version 4.7.2 is responsible.

So, when you launch VMware Workstation, you get this:

Which is fine: click the [Install] option, which brings up (after a prompt for the root password) this:

The yellow warning triangles are your first clue that this isn’t going as well as you hoped! The next screen just confirms it:

And if you check the log file mentioned there, you’ll see this sort of thing:

2013-03-10T11:19:29.891+10:00| vthread-3| I120: Building module with command 
       "/usr/bin/make -j8 -C /tmp/modconfig-eHRedb/vmci-only 
       auto-build HEADER_DIR=/lib/modules/3.8.1-201.fc18.x86_64/build/include 
       CC=/usr/lib64/ccache/gcc IS_GCC_3=no"
2013-03-10T11:19:30.562+10:00| vthread-3| W110: Failed to build vmci.  
       Failed to execute the build command.

So, that’s the end of that. :-(

It turns out that there’s a newer version of VMware available (9.0.2 instead of the 9.0.0.something I’ve been using), and installing that fixes everything. But it was only about three weeks ago that I downloaded the 9.0.0 version, and it’s 400MB-a-download, which I consider excessive. A product shouldn’t, I think, be so damned fragile as to break every time some libraries change a bit. And slimmer would be better, too.

Meanwhile, VirtualBox has passed through the same upgrade unscathed and fully-functional -and it’s only about 68MB to download. Productivity suggests it’s time for VMware to lift their game -and for me to look elsewhere for all my virtualization needs.

Toshiba P870 and Fedora 18

p870It hasn’t exactly been plain-sailing with my new laptop! It shipped with Windows 8 and I won’t touch that with the proverbial barge-pole, so it was clear from the outset that some sort of O/S re-installation would be required. I didn’t expect it to be such a trauma, though.

For starters, none of my distro CDs or DVDs would actually boot. Traditionally, this usually happens because the computer BIOS has a boot order set, and the hard disk often appears in that before the DVD drive. So, no problem: change boot order, make sure DVD is top of the list… and still nothing.

It turns out that this is a direct consequence of Microsoft’s insistence on “Secure Boot” (which you can read about here, for example). I’ve followed that particular saga on various websites for months, but never imagined I’d become one of its victims. But that’s exactly what I was: dig around in the P870′s UEFI settings long enough and you’ll find an option to disable secure boot… after which Linux distros will boot fine. I find it annoying that something trivially easy to do in the past has now become difficult and non-obvious to fix: if you didn’t know about Secure Boot and its consequences for Linux, how would you know to go looking for the option to disable it? You could well argue that someone wanting to boot Linux is likely to be technically clued-up enough to know about Secure Boot -but although I would consider myself to be in that bracket, it wasn’t the first thing that sprang to mind and I had no idea what I was looking for in the BIOS even after I suspected that it might be a secure boot problem. I call that irritating.

Anyway, I finally managed to boot Fedora 17 and install it on a partition I’d vacated from within Windows. Installation was smooth… except that Windows 8 was no longer bootable afterwards. I did everything I could think of to get WIndows 8 bootability back, but nothing worked. Fundamentally, I didn’t mind much, because I’d never expected to use the O/S much anyway -but having paid for the damn thing, it would have been nice to be able to at least burn some ‘rescue media’ to use later, if I wanted. I was, actually, a bit miffed that Toshiba supply absolutely nothing with their Prince of Laptops: no drivers disk, no O/S disk, nothing. Instead, it’s all on a ‘rescue partition’, and you’re supposed to burn off copies from there before you stuff around with anything. Being the gung-ho chap that I am, however, I didn’t do that. So, without Windows 8, I couldn’t access the rescue partition. And without that rescue partition, my license to use Windows 8 at some point in the future no longer existed.

In the end, I was reduced simply to wiping the whole thing and just kissing goodbye to Windows 8 entirely (my Technet subscription will get me a copy if I ever feel the need for it in the future, I guess). It disappoints me, though, that Toshiba don’t provide physical installation media for what is their their top-of-the-line laptop.(I’ve read that apparently they do… if you are prepared to pay them $66 for the privilege of them sending it to you. Seems a bit steep, to me).

On went Windows 7, with mercifully few dramas: Toshiba have a page-full of Windows drivers you can download and apply, so everything ended up working quite easily -except for the graphics. Before you can install the NVidia graphics driver (which Toshiba supply), you are told you have to install the Intel integrated graphics driver (which Toshiba doesn’t supply). A not-so-quick trip to Intel’s bewildering website later, however, and even that was sorted. So I had a fully-functioning Windows 7 laptop, finally… and only two days after I’d bought it!

Then, it was a new install -this time of Fedora 18, the latest and greatest from the Fedora fold. There are two fundamental problems with using Fedora on this laptop: the Ethernet port is not detected; and the Wireless Ethernet port is not detected either. So you can install the O/S perfectly well -but you’ll have zero connectivity, making it as useful as a chocolate teapot, basically.

Thankfully, this is fixable. First you will need to copy a bunch of rpms off your installation DVD to some directory or other (hunt around in the packages directory for each of them, being careful to match the names exactly):

cloog-ppl-0.15.11-4.fc18.1.x86_64.rpm
ppl-0.11.2-10.fc18.x86-64.rpm
ppl-pwl-0.11.2-10.fc18.x86_64.rpm
libmpc-0.9.3.fc18.2.x86_64.rpm
cpp-4.7.2-8.fc18.x86_64.rpm
gcc-4.7.2-8.fc18.x86_64.rpm
glibc-headers-2.6-24.fc18.x86_64.rpm
glibc-devel-2.16-24.fc18.x86_64.rpm
kernel-headers-3.6.0-4.fc18.x86_64.rpm
kernel-devel-3.6.10-4.fc18.x86_64.rpm

Install them all in one go with (as root) rpm -ivh *.rpm

Once those packages are installed, you can download compat-wireless-2012-03-12-p.tar.bz2 from here using someone else’s PC, transfer them to the laptop via a USB stick, and (still as root) issue these commands:

tar xvf compat-wireless-2012-03-12-p.tar.bz2
cd compat-wireless-2012-03-12-p
./scripts/driver-select alx
make install
modprobe alx

The network icon on the top-right of your screen should spring in to life -at which point, click it, select Network Settings and type in appropriate IP, Gateway and DNS addresses: Ethernet wired networking should now be properly functional.

To get wireless networking going, you’ll have to download a driver for the Realtek adapter, unzip it, cd to the new rtl_92ce… directory and then issue the commands:

make
make install
modprobe rtl8723e

As soon as you’ve done that, clicking the Networking icon in the system tray area at the top of your screen should display a list of nearby wireless networks you can now connect to.

Just be warned that if you use your new-found network connectivity to update your kernel at any time, the kernel modules for both network interfaces that you’ve just compiled will immediately stop working… and I’ve not yet been able to get them to re-compile, despite using newer compat-wireless downloads ad infinitum. For the moment, at least, I’m therefore trapped using a 3.6 kernel, instead of the latest 3.7.x variety… but I can live with that.

On the whole, it’s a painless process getting both networking interfaces working -and as far as I could tell, pretty much everything else on the laptop works as advertised (speakers, webcam and so on). I was worried that the graphics wouldn’t be right (as mentioned above, the laptop uses a curious combination of Intel integrated graphics and NVidia GT 630M), but they appear to work fine. My standard test is the framerate displayed once Stellarium has been installed and run:

Screenshot from 2013-02-12 13:52:44

That screenshot shows I’m getting about 45 frames per second, which is a bit on the low side, but entirely usable. In the Windows 7 installation, Stellarium manages ~70 frames per second, so clearly there’s some graphical optimisation I could do on the Fedora side of things if I was so inclined… but really, it’s perfectly usable as it is, so I probably won’t bother.

The only other bother I had with Fedora on this laptop was a biggie: VMware Workstation 9.0 produced a kernel panic immediately it was installed …and repeated the feat routinely at every subsequent startup. This turns out to be a reasonably well-documented problem that VMware has with Linux kernels 3.5 and above, generally: it affects VMware Player, too, for example. Happily, a slightly more up-to-date 9.0.1 version cures the problem -though at 395MB, it’s a regrettably large download.

Of course, before you can install that new version, you have to uninstall the old version -which is a bit tricky to do when the presence of the old version causes your O/S to keel over before you get a chance to uninstall it. In my case, from the black screen full of dire warning messages that results when the crash occurs, I was able to press [Ctrl]+[Alt]+[F2] to get to a command-line login prompt. Logging in as root, I was then able to issue the command

vmware-installer -u vmware-workstation

…to get the old version removed. After a reboot, the graphical desktop starts perfectly, so installing the new version was trivial. Fiddlier than I’d have liked, for sure; but fortunately, on this occasion, not a show-stopper after all.

So, apart from a lack of networking and an incompatibility with VMware, Fedora 18 runs nicely on this laptop. Bizarrely, too, I find that Gnome 3 is nowhere near as ghastly as I remembered it: improvements have been made, and the thing now seems to run slickly, looking good as it does so. I found the original Gnome Shell hopeless for a multi-monitor setup, but this newer versions seems a much better fit on a single screen laptop. I had been intending to install Mate, but quite honestly (and much to my surprise) I think I’ll give this particular slick implementation of Gnome 3 a good long run first.

A cautious thumbs up, then: Fedora and the Toshiba P870 work quite well together, with a modest amount of fiddling first. I’ll be happier if and when I can upgrade my kernel, but there’s no functional deficiency in the meantime.

VirtualBox Installations on Scientific Linux

I’ve mentioned previously that my preferred virtualization platform is VMware Workstation. That remains true …but VirtualBox does have the distinct advantage of being free-of-charge. So unless I want to insist that my readers find AU$291.50 for VMware’s offering, it behooves me instead, from time to time, to use the virtualization platform I know we can all afford.

So here is my two-minute recipe guide to getting the latest VirtualBox product installed (for zero dollars!) on Scientific Linux 6.1. (You could always just download the relevant rpm and install it directly, but I prefer to do all my package management via yum wherever possible, so that’s the approach described here).

1. Get the gpg key

VirtualBox is supplied as a bunch of rpm packages which have been digitally signed. By checking the signature, you know no-one’s messed about with the packages before they reached you. It therefore makes sense to obtain and install the digital key needed to do that signature check. It’s easy to do, as root, at a command prompt, by issuing these two commands:

wget http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc
rpm --import oracle_vbox.asc

2. Create the yum repository

Again as root, issue this command to create a new, blank repository file:

gedit /etc/yum.repos.d/virtualbox.repo

Now paste in these contents to the empty file:

[virtualbox]
name=RHEL/CentOS-$releasever / $basearch - VirtualBox
baseurl=http://download.virtualbox.org/virtualbox/rpm/rhel/6.0/$basearch
enabled=1
gpgcheck=1
gpgkey=http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc

Save the file changes and close down gedit. Just in passing, you might note that I’ve changed this file from the version which Oracle themselves makes available on the VirtualBox website. Specifically, their baseurl uses an environment variable, called $releasever, where I have hard-coded in the number 6.0. The trouble, of course, is that if you are using the latest versions of Scientific Linux or Centos, you’ll be picking up a releasever of 6.1 or 6.2 …and no such directory exists on the VirtualBox servers. You’d need to manually check out those server’s directory structure to see if that situation changes over time.

3. Install the Software

As root once more, the following one-liner will display all the different versions of VirtualBox that are available for installation:

yum search VirtualBox

You might see this sort of output in return:

Loaded plugins: refresh-packagekit
virtualbox | 951 B 00:00
virtualbox/primary | 4.4 kB 00:00
virtualbox 17/17
============== N/S Matched: VirtualBox ==============
VirtualBox-3.2.x86_64 : Oracle VM VirtualBox
VirtualBox-4.0.x86_64 : Oracle VM VirtualBox
VirtualBox-4.1.x86_64 : Oracle VM VirtualBox

This shows that Oracle keeps a couple of older versions of the software alive and available, should you need to use them. Most people, though, will really only need the latest version, so pick that from the list and issue an appropriate “yum install” command. In my case, given the above output, this command will do the right thing:

yum -y install VirtualBox-4.1.x86_64

It’s a 58MB download or so, and as it’s installed you might see this message appear:

Running Transaction
 Installing : VirtualBox-4.1-4.1.8_75467_rhel6-1.x86_64 1/1

Creating group 'vboxusers'. VM users must be member of that group!

This gives you the clue to the last stage of the installation process…

4. Assigning Group Privileges

The software installation has created a new O/S group, called vboxusers, but it won’t have made your user account a member of that group. That needs to be fixed.

From the Gnome top panel, clieck System > Administration > Users and Groups. Find your user details on the Users tab and double-click the entry. Switch to the Groups tab, scroll down and check the vboxusers group name:

Click [OK] to save the change, and you’re done, though you’ll need to log off and back on before the group membership changes take practical effect.

If you prefer doing everything at the command line, just edit /etc/group (as root, of course) and add a :your-username entry to the end of the vboxusers line, which will probably be the last line of the file. In my case, for example, the line ended up reading vboxusers:x:501:hjr -which simply means that user ‘hjr’ is now a member of the vboxusers group. (Again, it’ll take a log off and fresh log on before the new group membership actually takes effect).

Either way, you’re now done and can run the VirtualBox program successfully, with the program launcher being found in Applications > System Tools.

5. USB Support

The version of VirtualBox installed by the above procedure will be unable to access USB 2.0 devices that might be plugged into your physical host. However, this shortcoming can be fixed by installing the “Oracle/VirtualBox Extension Pack”. Download it from the VirtualBox website and then just double-click the file when the download completes. You should see the following appear:

Click the [Install] button there, agree to the license, authenticate as root and all should be done in a matter of seconds. You’re now ready to build and run fully-functional virtual machines.

VMware at the Command Line

I use VMware Workstation for all my virtualisation needs. It runs (IMHO) more reliably than VirtualBox. With version 8, too, you can do neat things like close the virtual machine but have it continue to run in the background: perfect for a little web server ‘virtual appliance’, say, which needs to be providing a service but definitely doesn’t need to be in your face whilst doing so.

Generally, I do all my VM maintenance with the GUI, but it’s nice to know that there are command line alternatives -especially since the command line often lets you do things which are not exposed in the GUI.

For example, I recently created a virtual machine without noticing I’d accepted the default option of having its hard drive split into multiple 2GB files. Once I’d loaded data into the thing, I was the proud owner of over 100 separate files making up the virtual disk! Is there a way to consolidate all those files back into one, single giant file? Not in the GUI there isn’t (update: actually, there is in Version 8, but regardless…), but this command did the trick:

vmware-vdiskmanager -r /path/to/disk/filename.vmdk -t 0 /path/to/disk/newfilename.vmdk

The “-t 0″ means ‘create a single, growable virtual disk’. Obviously, there are multiple files which make up the original virtual disk: you only specify the ‘parent file’ in this command. So, instead of mentioning vdisk-s001.vmdk, vdisk-s002.vmdk and so on, you just specify the plain “vdisk.vmdk”.

This doesn’t do anything to make the virtual machine aware of the new disk, or to get it to boot up from the new disk rather than the old: you still have to edit the machine details (using the GUI) to achieve that, but at least I don’t have hundreds of files littering my physical hard disk any more!

That same utility, vmware-vdiskmanager, can additionally be used to create new virtual disks in the first place (with the -c /path/filename.vmdk option). It also allows you to resize a virtual disk if you created it inappropriately small (using the -x option, plus a size…  so -x 300GB /path/filename.vmdk would do the trick). Disks that were created inappropriately large can be shrunk with the -k /path/filename.vmdk option (no size is specified… the thing shrinks to the smallest size that can contain all existing data). There’s also the option to defragment a virtual disk (-d /path/filename.vmdk).

To see the complete list of options, just type the command vmware-vdiskmanager without any options at all. Very handy!

The Case of VMware and the missing SCSI ID

When you’re setting up Oracle’s Automatic Storage Management feature (ASM), you have to ensure that the ‘raw’ devices that you have added to your server for ASM’s use are assigned the correct device names and usable permissions every time the server bounces.

In the dark days of Red Hat 3 and 4, we generally arranged for that to happen by creating raw device mappings -but that’s no longer supported on RH5 or RH6. Instead, you’re supposed to create new udev rules which do the job of declaring which devices exist and which permissions they should at every server boot.

And one of the first things you have to do to write a decent udev rule is to correctly identify the hard disks that exist: you can’t apply a permissions rule to something which you can’t uniquely identify in the first place, after all.

So when this happens, you have a bit of a problem:

That’s four SCSI hard disks, previously added to my VMware Workstation virtual machine and partitioned, resolutely failing to respond to the scsi_id command, which is what you’re supposed to use to get a unique ‘id string’ returned for a device. This happens when I use virtual machines built on VMware’s ESXi 4.1 server, too.

But, funnily enough, it doesn’t happen if you use VirtualBox as your virtualization platform:

Now, I’m not going to say this proves VirtualBox is better than VMware (because it’s not), but I am going to point out that, by default, all VMware virtual machines exhibit this behaviour, which will stop you dead in your tracks if you’re trying to build a virtual ASM or RAC machine. Without that ID string, you can’t identify your ASM disks uniquely -and that means you can’t get those disks correctly discovered by the operating system …and it’s downhill all the way after that.

Lucky, then, that this is all fixable with a modest bit of re-configuration!

If you were running ESXi 4.1, that bit of re-configuration consists of

  • shut down your virtual machine
  • right-click the VM’s entry in the left-hand panel and select Edit Settings
  • click the Options tab
  • Select the Advanced -> General item on the left and click the Configuration Parameters… button you then see displayed on the right
  • Click the Add Row button
  • Add disk.EnableUUID as the name of the new row, and the word TRUE as its value (don’t use quotation marks around either of these entries).
  • Click OK to make the new parameter addition ‘stick’.

You can then reboot your virtual machine.

Sadly, VMware Workstation has no interface like this that allows you to add this new configuration key to your VMs. Instead, you are reduced to having to do it yourself, by hand, using a text editor. It’s easy enough, however.

First, find the directory where the files representing your virtual machine are stored. One of them will be called the name of your VM, with an extension of .vmx. In my case, for example, the file is called sl6.vmx, because when I created my VM, I called it “SL6″. The file will be 3 or 4KB in size. Open it in the text editor of your choice and at the very end of the file, add this line:

disk.EnableUUID = "TRUE"

The quotation marks around the word “TRUE” here are important and must be typed. Now save the edited file and reboot your virtual machine. You should find that scsi_id is now capable of returning perfectly usable values:

As you can see from this screenshot, my VMware virtual machine is now displaying SCSI IDs for my hard disks just fine. The values being returned are quite different from those shown earlier in my VirtualBox VM, of course, because VMware and VirtualBox handle virtual disk identification quite differently. Whatever the specific values might be, though, the important point is to be able to see a value of any sort!

VMware Workstation Glitch

I opted to install various updates today to my Debian “Squeeze” PC. I’m afraid I don’t know precisely what updates they were, because (like a lot of people, I suspect!) I didn’t bother to pay a lot of attention to the list of what was about to be done to my system. Whatever it was, where I was happily VMware-ing before the update (and for hours afterwards), after I’d rebooted the PC, I couldn’t start the virtual machine any more: the program kept complaining about “couldn’t find /dev/vmmon”, and when I manually tried to modprobe it, it came up with all sorts of errors indicating kernel troubles.

There being no obvious way to fix things (Google doesn’t help much on this, with all suggestions I saw involving re-running vmware-config.pl, which simply doesn’t exist on my system for some reason!), I resorted to the crowbar method of software maintenance: I ran /usr/bin/vmware-installer -u vmware-workstation and got the software uninstalled. Then I re-ran sh VMware-Workstation-Full-7.1.0-261024.x86_64.bundle to reinstall it… and after the re-install, it all worked as fine as before.

God knows what the trouble was, but let that be a lesson to you: don’t update (especially on a testing branch of Debian!) unless you really need to!

Before I decided on the re-install route, I did dabble with the idea of giving up altogether and using VirtualBox once more. The open source version is available in the Squeeze repositories, and I figured that if it was there, then any updates affecting those repositories is likely to leave VirtualBox at least in working order. Of course, I wouldn’t want to have to rebuild my VMware Workstation virtual machine… a lot of software and OS updates have gone on in it.

So is it possible to open a VMware Workstation VM in VirtualBox?

The answer to that is, ‘yes, very easily’:

  • Make sure your VMware virtual machine has no snapshots. Delete them if it does.
  • Start VirtualBox OSE (Applications -> System Tools -> VirtualBox OSE)
  • In File -> Virtual Media Manager, click Add. Point to the .vmdk file of your VMware virtual machine. Double-click to select and add it, then click OK to close the media manager window.
  • Click the New button to create a new Virtual Machine, fill out the memory details as you think appropriate. Select the Use existing hard disk option when shown, and select the .vmdk file from the combo box.
  • Boot the new VirtualBox virtual machine as normal. Windows will, of course, have an absolute fit at the amount of hardware you’ve changed, but if you give it long enough, it should come good as it detects it all and re-configures itself accordingly. You may need a couple of reboots before everything is detected and installed perfectly.
  • Don’t forget to install the VirtualBox Guest Additions so that you get proper video/mouse handing for the new environment.

When I first did this, the machine would simply boot to a blank screen, sit there doing nothing and otherwise just die. Rebooting a couple of times, the same thing happened each time. When I checked the logs for the VM (found in my /home/hjr/.VirtualBox/Machines/Windows XP/Logs directory), I saw I was getting the error message: BIOS: int13_harddisk: function 15, unmapped device for ELDL=81. I had a bit of a guess at this one: powering down the VM, I went into Settings -> System and switched on the option to Enable IO APIC. Once I booted the VM once more, it worked perfectly.

One other gotcha: the networking in the VMware machine was fine, but the guest OS when running in VirtualBox declared that no network was available. The reason for that is that my VMware machine had been built with Bridged networking, but VirtualBox creates new VMs with Network Address Translation (NAT) networking instead. Power the guest down and edit the Settings -> Network to switch to the Bridged Adapter option.

One quirk that is probably peculiar to my setup: my VM has Checkpoint Secure Remote installed on it (it’s the VPN I use to connect to work, since they don’t seem to be able to run to a VPN that has a Linux client!). That prevented my new network card from functioning properly -but once I’d uninstalled Checkpoint and rebooted, the network card was detected fine. Then I was able to re-install Checkpoint and connect to work as normal… yet another instance of crowbar software maintenance, I’m afraid, but at least it worked (again)!

Not too painful, all things considered -though I’d suggest trying it out on a copy of your VMware virtual machine before you let rip on the real thing! Getting networking, er, working was probably the hardest part of the entire affair.

The extent of the hardware changes the guest OS has to deal with are enormous, so it was frankly a bit of a surprise that Windows XP coped… whether later versions of Windows would is a matter for speculation, I guess. I would expect such a degree of hardware change to prompt re-activation of Windows, though (the reason I tend to stick with XP in my VMs: no activation required, at least for my ancient installation disk!).

There is an alternative to this approach, which involves using qemu and stuffing around actually transforming a .vmdk into a native .vdi file via a .bin intermediate file, but there’s really no need, so that’s not an option I’ve experimented with.

I’ll probably stick with using VMware Workstation (having paid for it, after all!), but there’s something to be said for having your virtualisation technology available from the repositories… so we’ll see!

ESXi Hard Disk Boo-Boo… and fix!

Running ESXi 4.1 in freebie mode, I accidentally created an XP virtual machine with an 8GB hard disk (instead of the intended 80GB). Inevitably, I failed to notice this until after I’d spent the best part of a day upgrading the thing to Kingdom Come and configuring large amounts of software. Starting from scratch with a new VM and a correctly-sized disk was, then, out of the question. But how, then, do you increase the size of an existing virtual hard disk?

With great difficulty, in a nutshell!

If you Google around a bit, you’ll see various people suggesting you can get yourself into the host machine’s “Tech Support Console” and run the command vmkfstools -X 80G ./name-of-disk-file.vmdk. However, when I tried this, I was told that the relevant file was not a valid virtual disk file… so, end of experiment with that technique. (I believe you can get this error message if the virtual machine has snapshots. Also if the file is corrupt!)

Anyway, here’s what I ended up doing instead to resolve the issue (and it’s not exactly pretty!)

First, shut down the virtual machine if it’s not already. Then right-click and Edit Settings. Add a second hard drive, this time sized correctly. Once that’s added, reboot the virtual machine.

Next, from within the virtual machine, visit http://www.miray.de/download/sat.hdclone.html and download the zero-cost HD Clone utility for Windows (the .exe is fine, no need for the .zip). Run that and install the software. If necessary, reboot once more.

Third, run the HD Clone utility. Select the Cloning option, and allow it to detect the two hard disks. Follow the prompts to clone the small original disk to the larger new one. Let it do its stuff (it took about 7 minutes to clone an 8GB drive). When prompted, allow it to automatically resize the finished clone, and do whatever else it feels needs to be automatically adjusted.

When the clone is complete, shut down the virtual machine. Go back to the right-click-edit-settings option and assign the first hard drive (the original boot disk) to IDE controller 1:1 (it started off on 0:0, which is what made it the first -and boot- hard disk). This frees up the 0:0 location for the second, larger hard disk you’ve just created, so now you click that and assign it to the just-freed-up 0:0 IDE interface (making it the new boot disk). Lastly, you click the original hard disk and click the Remove button (if you’re not feeling brave, select the option to remove the disk from the virtual machine without deleting the files off the physical host!).

This little game of deckchair shuffling means the new, larger hard disk ends up occupying the same position in the IDE chain of command as the original, smaller, one did. So when you now power on the virtual machine, it should correctly boot from the new hard disk. Windows XP certainly did, first time of asking -and then promptly went into that ‘new hardware has been detected, you must reboot for it to work properly’ routine. One further reboot later, therefore, and I finally ended up with an XP machine running on an 80GB virtual hard disk rather than an 8GB one.

I have read that you might need to jiggle the boot flags on the new hard disk, but I didn’t have to. If you do, you’ll have to alter the VM so that its CD/DVD Drive is pointing to an ISO of Gparted. That will let you set the new hard disk to be bootable, and then you should be good to go. Happily for me, HD Clone seems to take care of this by itself, so as I say, I never had to do this further bit of fiddling.

If all has gone well, you can use the ESXi Browse Datastore option to remove the physical file associated with the original virtual hard disk if you weren’t brave enough to physically delete it originally. You can also uninstall HD Clone from within the virtual machine, since you won’t (presumably) be needing it again. HD Clone works fine in Windows 7, by the way, so the same basic technique should apply to any vaguely-recent version of Windows, not just XP.