Cisco announced their release of UCS Director 5.4 back in November. As I’m currently running 5.3 and ran into an issue with a workflow Cisco support recommended upgrading to 5.4. I had a look over the Cisco UCS Director 5.4 Release Notes and there’s a new version of Java and the CentOS operating system are newer in the latest version. Due to this the upgrade procedure for 5.4 is different from previous version. In earlier versions it was possible to upload a patch via shelladmin and it would upgrade the software and database schema in place. 5.4 however requires new appliances to be deployed and a migration of database files etc. to be done between the 5.3 and 5.4 versions.
I really think that Cisco needs to look at using a HTML 5 console in the future as this upgrade path is overly complicated. Considering a lot of companies want you to be on the latest version when opening support calls, including Cisco, it would make sense for them to make it easier to perform the required upgrades.
The primary changes that have caused the modification to the upgrade path are:
- CentOS version 5.4 to version 6.6
- Java version 1.6 to version 1.8
Another thing to note is that version 5.54 requires 12GB RAM.
Cisco recommend standing up the new appliances beside your current UCS Director and Bare-Metal Appliances and performing a migration. In my case there’s a few firewall rule etc already been created for the existing environment so I wanted to keep the same IP addresses and machine names. I changed the IP addresses of the current appliances to be something else within the same subnet and gave the new appliances temporary names but the existing IP addresses. Once everything had been migrated and the changes confirmed I was able to rename the appliances to be the existing ones and removed the older appliances from the infrastructure. Before commencing the upgrade I also had a sold read over the UCS Director Upgrade 5.4 Guide and the UCS Director Bare-Metal Agent 5.4 Upgrade Guide
Download the software form the Cisco Download Software portal. Download the UCS Director 5.4 appliance and the BareMetal Agent files.
Once it’s been downloaded extract the zip to access the OVF required for a new appliance.
UCS Director Upgrade Preparation Steps:
The first step that is always recommended is to take a backup of the database for UCS Director. This can be done via the console under the shelladmin login. Take a snapshot within vCenter also.
Open a console into vCenter and login.
Log in using the shelladmin account.
Select option 3 to stop all services and select Y at the prompt.
Select option 2 to ensure that all the services have been stopped.
Then exit the shelladmin login and select Configure Network and give the current UCSD appliances a new IP address so the current one can be re-used.
Select n for a DHCP address and enter the new IP address. I’ve had to mask the actual addresses here but you can see that the final octet has changed from .54 to .235. Enter the DNS servers etc and commit the change of IP address. Make sure you can ping the appliance on its new IP address.
UCS Director Deployment Steps:
In vCenter select File -> Deploy OVF Template
Select the extracted OVF file from the earlier download of the appliance.
Have a look over the requirements and click Next.
Accept the license and click Next
Select the Host and Cluster to deploy the new appliance to. Enter a name for the appliance and click Next.
Select the relevant resource pool and click Next
Select the datastore location for where you want to place the appliance.
Select which disk format you want to use. In this instance I left the disk format defaults. Click Next.
Select the network to assign to the appliance and click Next.
Enter the IP address address and put in the root and shelladmin passwords. The IP address used here is the one from the old appliance. It is the one that ends in .54. Once all the information has been entered click Next.
Lastly, verify all the information and click Finish
Once the appliance has finished installing power it on. You’ll see the database creating the required schemas
Log into the new appliance using the root account created during the OVF deployment. As the IP address is the one used previously by the 5.3 version it is recommended to run the /opt/scripts/configStaticIP.sh. Fill in the details as required. In the screenshot below it shows I also ran the configStaticIP.sh on the old appliance to ensure that it also retained its static address. The same entries will be needed for the new appliance. This is a recommendation from the Cisco documentation.
Once the IP has been set log out of the console and you’ll see an Emerald Bay UCSD 5.4.0 title for the appliance as well as the required web console login.
Open a browser and log in using the admin account. It will appear as a blank system. Now that the new appliance is installed the files needed can be migrated/copied from the old appliance before the old one is decommissioned.
Migrate Data between UCS Director versions:
Log back into the new appliance using the root account. Run the script /opt/infra/migration/performMigration.sh to migrate the database and configuration files from the 5.3 appliance to the 5.4 release. The script backs up the database on the old appliance and restores it to the new one. It also executes the scripts required to upgrade the database.
When prompted, provide the IP address of the old appliance.
The services on the old appliance will be requested to be shut down during the migration process.
Once the migration process has been completed you’ll be prompted to start the services using shelladmin on the new appliance.
Select option 4 to restart all of the services.
Run option 2 to verify that the services have started.
Once the migration script has completed you may need to copy some files manually from the old appliance. You’ll need to review the files to move. Enable the SSHD service on both the old and new UCS Director appliances. Check the old version against the new version to see if any of the following files need to be copied. If not, ignore them. Open WinSCP sessions to both and copy these files:
- mysql tuning in /etc/my.cnf
In my case the ntp.conf was different but I just edited the new file to point to internal NTP servers
Changes in the following properties files:
Migrate the contents from the/opt/infra/uploads folder. This folder contains any files and OVFs uploaded from the Cisco UCS Director user interface. To migrate the files I ran a WinSCP session to download the required files locally and upload them to the new appliance via another WinSCP session. When prompted to overwrite existing files click Yes.
At this point I restarted the entire appliance to just start with a fresh slate. While doing this I powered down the old appliance so there were no potential clashes. I recommend to rename the old appliances to _temp and modify the VM names of the deployed VMs to the existing VM names. After the reboot log into UCS Director via the web console.
Click About and verify the version you are running.
You have now successfully migrated to 5.4.0. The last thing to do is to configure a weekly backup of the database. Once the script has been configured just run the job manually and ensure that the backup reaches your FTP server.
BareMetal Agent Upgrade Steps:
Once the main UCSD is done log into UCSD BMA using the root account and run the /opt/infra/configureInterface.sh script for eth0 so that its a different IP address. The new appliance being deployed will take over the current IP address assigned to BMA.
Then run the command ./stopInfraAll.sh to shut down all services.
Next we can begin the deployment of the new appliance. Go to File -> Deploy OVF template
Select the BMA OVF that was downloaded earlier.
Click Next to continue the deployment process
Review the details of the appliance and click Next
Accept the license and click Next
Enter a name for the appliance and click Next
Select the resource pool to deploy the appliance to.
Select the relevant datastore and click Next.
Check the disk format and select whichever format fits to your environment.
At this point there’s a need to select the relevant networks. One is required for management and one is required for PXE boot. Click Next.
Enter details for the passwords for the accounts and the IP address of the old appliance. Click Next to continue.
Review the details of the appliance and click Finish to complete the deployment.
Open the console session and verify the correct IP address is assigned.
Open a web browser and verify that you can at least see the 22.214.171.124 version showing up.
Log into the new appliance with the root account.
Once logged in run /opt/infra/migration/performBMAMigration.sh script to migrate the images, templates, and configuration files from the 5.3 BMA appliance to the 5.4 BMA appliance.
The migration script will then go and copy all the required files.
Once it has been completed log in toCisco UCS Director as an admin and verify that all the images from the 5.3 BMA has been migrated successfully. You may need to refresh the view to make sure it shows up correctly.
As with the main UCS Director appliance you’ll need to copy some files with WinSCP. Copy the following files from the old appliance using WinSCP
- mysql tuning in /etc/my.cnf
Reboot the new appliance and shutdown the old version
For any Windows images migrated from the 5.3 BMA update the WinPE file with the new IP address of the 5.4 BMA. You do not need to edit WinPE if the Baremetal Agent IP address was not changed.
If you use a separate interface for PXE Traffic you’ll need to configure the PXE interface IP using the Cisco UCS Director UI if you haven’t reused the 5.3 BMA appliance PXE IP address. Otherwise it will be fine.
Using the Configure Interface option, select the BMA account and copy the /etc/dhcpd.conf file from the 5.3 BMA appliance and reconfigure it based on the new PXE Interface IP address.
The last steps involve renaming any of the appliances as needed and then once the new appliance has been confirmed as working correctly delete the old appliances from the environment. And that’s it, everything has been updated. It’s a real pain to have to deploy new appliances and hopefully this will be addressed in upcoming versions.
Note: I just noticed as I was writing this post that a new 5.4.2 patch has been released. I’ll have to start looking at upgrading that patch also.