The servers hosting the openstack-platforms also need to be provisioned, but they cannot be provisioned by openstack. The provisioning of the hardware-resources are thus controlled by Shiftleader, which is a simple tool helping us to configure PXE/DHCP/DNS/Puppet sufficiently.

Commisioning of a new server

New servers (both physical and VMs running on our infra-nodes) are provisioned the same way:

  1. Collect the servers MAC-adresses (for the primary NIC and for its iDRAC/BMC interface)
  2. Register the server in shiftleader, first with the primary NIC before adding a second NIC for the iDRAC.
    1. If the server need addresses from other networks, like the storage-network, register more interfaces for the server.
    2. Make sure to select a suitable environment, role and boot-template for the server.
    3. This step results in the server being registered in DHCP/DNS, and the PXE-environment will be configured to netboot this server.
  3. Configure the servers boot-order to be pxe before local disk. This way Shiftleader can control if a server should be reinstalled or not when its rebooted. A server that should not be reinstalled will be instructed to boot from its disks when it tires PXE.
  4. Boot the server, and se that it gets installed.
    1. The installation-process also installs the puppet-agent and generates a TLS-certificate for the host used to authenticate itself for the puppetservers.
  5. Our Zabbix-servers will automatically discover the new server, and add general monitoring-templates.
    1. Remember to add any extra templates or macros if they are needed. 

Reinstallation of a server

To reinstall a server you should simply:

  1. Mark the server for reinstallation in shiftleader
    1. Consider to switch the server-role to «base» if you dont want the server to be reinstalled right into the same role
  2. Reboot the server

Decommisioning of a server

When a server is going to be decommissioned we need to:

  1. Make sure the server is not hosting services for anyone anymore.
    1. If a compute-node: migrate VMs elsewhere
    2. If a storage-node: Move data elsewhere
    3. If a controller-node: Make sure all services is also hosted from another controller.
  2. Shut down the server
  3. Delete the server from shiftleader to revoke the puppet-certificate, clean DNS and free the IPs.
  4. Tell puppetDB that the server is gone by logging in to a puppetserver and run the command:
    1. puppet node deactivate <SERVER-FQDN>
  5. Delete the server from zabbix.