Upgrade

Overview of the Upgrade Process

Example Pusher Upgrade

For upgrades, HeadSpin will provide the latest keys-customer-red and keys-customer along with a versioned installer image. This guide will go through preparing the software on a USB drive and running the upgrade installer script.

Untar the installer archive on the Unified Controller or to a USB drive. If using a drive, mount the drive and change into the installer directory on the USB drive. The usb drive must be written in a format readable by Ubuntu 16.04 server by default. HFS+ is an acceptable format, it is readable, but not writable on Ubuntu by default. ext4 is also a good format to use but is not readable on macOS by default. A virtual machine can be used on macOS to write to ext4 format.

Once your USB drive is mounted on the Unified Controller, simply navigate to the installer directory and run the deploy script.

cd /media/onprem/installer-{version}
sudo ./onprem_deploy.sh $(pwd)/keys

Onprem Software Upgrade

First, untar the update installer archive:

tar -zxvf customer-installer-0.X.X.tar.gz

A Headspin onprem installer should contain the following file structure.

The upgrade is intended to be deployed from the onprem Unified Controller but can be used to deploy from any machine on the network.

Next, untar the keys archive:

tar -zxvf customer-keys.tar.gz

The extracted contents of the archive will contain a keys folder with two sub-folders:

Put the file contents of the installer on a USB drive along with <code class="dcode">keys-customer-red</code> and <code class="dcode">keys-customer</code>. Plug your USB drive into the Unified Controller and then and mount the drive in the Unified Controller.

Use <code class="dcode">lsblk</code> to determine which drive is appropriate to pass to the mount command. In the example below you may have to replace <code class="dcode">/dev/sda2</code> with a different path depending on which port the USB drive is plugged into.

sudo mkdir -p /media/onprem/
sudo mount /dev/sda2 /media/onprem

The final layout of the USB drive should look like below.

Run <code class="dcode">onprem_deploy.sh</code>

cd /media/onprem/installer-0.X.X
sudo ./onprem_deploy.sh

The deploy scripts will run and push updates to the hosts in the onprem unit. Once the script concludes, your onprem unit has finished updating.

Onprem Upgrade Components

<code class="dcode">onprem_deploy.sh</code> loads up mirrors necessary for deployments and then runs a Docker container named pusher to deploy to all hosts listed in the <code class="dcode">keys-env/deploy_type/host_push.yml</code> configs.

Pusher

Pusher runs pre-push-checks on the keys and target hosts to make sure they are reachable and keys are configured properly. Post push logs will be collected in /var/log/pusher directory and play recaps will be displayed.

Leave the USB drive plugged into the Unified Controller. For full mirrors, package mirror data will be served from it.

If a push fails, it is usually a config or networking error. If just pusher failed for the onprem deploy, then you can re-run just pusher without re-loading mirrors data with:

cd /media/onprem/installer-0.X.X/pusher
sudo ./pusher.py -k /path/to/keys/directory

<code class="dcode">pusher.py</code> also takes a comma separated target argument so you can specify specific deployment types to upgrade.

sudo ./pusher.py -k /path/to/keys/directory -t unifiedcontroller,pboxagent

Mirrors

Mirrors are used for onprem deployments to provide required packages to the network. Mirrors are loaded and ran as part of the onprem deployment and are essential for distributing required packages for pushes.

There are 6 types of mirrors. One for apt, pip, npm, bower, brew, and the docker registry

Mirrors are ran in docker containers. The mirrors and mirrors-resources repositories are required to load up and run mirrors on a machine. mirrors-resources/load_everything.sh will load the docker images and volumes required for mirrors to work. Then mirrors can be started and stopped from the mirrors repo.

(cd mirrors-resources && ./load_everything.sh)
(cd mirrors && ./mirrors.sh run all)
(cd mirrors && ./mirrors.sh stop all)

Common Problems

mysql

If an error occurs trying to upgrade mysql during onprem deployment, then <code class="dcode">fix_mysql.py</code> can be ran to fix it.

python /media/onprem/installer-0.X.X/pusher/devtools/tools/fix_mysql.py