Saturday, February 9, 2019

Deploying ESXi 6.0u3 via PXE in a lab

In my never ending quest for automation, I thought I'd experiment with what is required to deploy ESXi over the network to new servers.

In order to script this, and know that I hit every checkmark, I decided to write it first using Vagrant.

I wrote this because other posts on the subject are close, but not exactly spot-on. This post, using Vagrant, is guaranteed because it's built from scratch every time I deploy it. I destroyed and rebuilt the entire environment multiple times while creating this post.

It uses these packages:


dhcp                 x86_64    12:4.1.1-63.P1.el6.centos
syslinux             x86_64    4.04-3.el6
tftp-server          x86_64    0.49-8.el6

If those packages change, this tutorial might become out of date. You should be able to force a version of each package to install in the bootstrap.sh file.

This tutorial uses CentOS 6.4 64-bit as a DHCP server and Vagrant.

All the code needed for this tutorial is available at https://github.com/volvo64/VMwarePXE

You'll also need an ESXi ISO. I used VMware-VMvisor-Installer-6.0.0.update03-5050593.x86_64.iso. This should be in the same directory as your Vagrantfile and bootstrap.sh. Make sure to change references to this ISO in bootstrap.sh.

Sorry for the weird formatting, Blogger isn't the best platform to write about code on. Review the code on Github for that.

First, in Vagrantfile, we're going to specify Centos64:
config.vm.box = "forumone/centos64-64"
We also need to specify a bootstrap file:
config.vm.provision :shell, path: "bootstrap.sh"
Lastly, in the Vagrantfile we need to specify a private network to perform DHCP on:
config.vm.network "private_network", ip: "192.168.33.10"
192.168.33.0/24 works for me, you should use what works for you.

That's all for the Vagrantfile.

bootstrap.sh actually builds the services once the Vagrant box has been created. We need a few programs installed. Let's start with dhcp, tftp-server and syslinux:
#!/usr/bin/env bash

yum -y install dhcp tftp-server syslinux
Next we need the contents of the ESXi ISO available in our tftpboot directory. We can mount it from the /vagrant/ directory that's automatically shared from your host:
mount -o loop /vagrant/VMware-VMvisor-Installer-6.0.0.update03-5050593.x86_64.iso /mnt/
Copy the files:
cp -rf /mnt /var/lib/tftpboot/esxi60u3
And finally unmount the ISO:
umount /mnt/
Now that we have DHCP installed and the ISO copied we need to create the DHCP options. This block writes the options to /etc/dhcp/dhcpd.conf:
echo '# dhcpd.conf
#
# Sample configuration file for ISC dhcpd
#

# option definitions common to all supported networks...
option domain-name "damon.local";
option domain-name-servers localhost.localhost;

default-lease-time 600;
max-lease-time 7200;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
authoritative;

# Use this to send dhcp log messages to a different log file (you also
# have to hack syslog.conf to complete the redirection).
log-facility local7;

allow booting;
allow bootp;
option client-system-arch code 93 = unsigned integer 16;

# This is a very basic subnet declaration.

subnet 192.168.33.0 netmask 255.255.255.0 {
range 192.168.33.100 192.168.33.110;
option routers 192.168.33.10;
}

class "pxeclients" {
match if substring(option vendor-class-identifier, 0, 9) = "PXEClient";
# specifies the TFTP Server
next-server 192.168.33.10;
if option client-system-arch = 00:07 or option client-system-arch = 00:09 {
# PXE over EFI firmware
filename = "esxi60u3/mboot.efi";
} else {
# PXE over BIOS firmware
filename = "pxelinux.0";
}
}
' > /etc/dhcp/dhcpd.conf
 We need to do the same for the TFTP options in /etc/xinetd.d/tftp:
echo 'service tftp
{
socket_type = dgram
protocol = udp
wait = yes
user = root
server = /usr/sbin/in.tftpd
server_args = -s /var/lib/tftpboot
disable = no
per_source = 11
cps = 100 2
flags = IPv4
}' > /etc/xinetd.d/tftp
 The original boot.cfg that we copied from the ESXi ISO contains references to "/" all over the place that are no longer valid. We need to remove them. To do that:
sed -i 's/\///g' /var/lib/tftpboot/esxi60u3/boot.cfg
We need to create some directories and move some files to make PXE boot:
mkdir -p /var/lib/tftpboot/pxelinux.cfg
cp /usr/share/syslinux/pxelinux.0 /var/lib/tftpboot/
echo y | cp /usr/share/syslinux/menu.c32 /var/lib/tftpboot/esxi60u3/ 
 The last change we need is to write the menu.c32 file:
echo 'DEFAULT esxi60u3/menu.c32
MENU TITLE ESXi-6.0 Boot Menu
NOHALT 1
PROMPT 0
TIMEOUT 300
LABEL install
KERNEL esxi60u3/mboot.c32
APPEND -c esxi60u3/boot.cfg
MENU LABEL ESXi-6.0U3 ^Installer
LABEL hddboot
LOCALBOOT 0x80
MENU LABEL ^Boot from local disk' > /var/lib/tftpboot/pxelinux.cfg/default
And finally restart some services:
/etc/init.d/xinetd restart

/etc/init.d/dhcpd restart

/etc/init.d/iptables stop

 /etc/init.d/ip6tables stop
Run 'vagrant up' and you should have a server after a few minutes!

To verify the network interfaces, check out its network config. You should see two NICs attached:

Adapter 1 is your management interface, Adapter 2 is your internal interface.

Create a new VM in Virtualbox. Give it at least 4GB of RAM and a little storage. Make sure the network is attached to the same network as Adapter 2:

In order to boot it you need to assign 2 CPUs:
Unfortunately since Virtualbox doesn't allow for Nested Virtualization (Nested VT-x/AMD-V) we won't actually be able to run anything on these hypervisors.

Start up your test VM, use F12 and boot it from the LAN and it will show you the ESXi installer:

After this point just install ESXi as normal and it should load up fine!


Thanks to:
http://www.vstellar.com/2017/07/25/automating-esxi-deployment-using-pxe-boot-and-kickstart/

Will investigate stateless ESXi/running a real config directly from PXE soon, as well as upgrading to newer ESXi releases via PXE.

No comments:

Post a Comment