Upgrade CSM and additional products with IUF

This procedure is used when performing an upgrade of Cray System Management (CSM) along with additional HPE Cray EX software products at the same time. This procedure would be used when upgrading from one HPC CSM Software Recipe release to another.

This procedure is not used to perform an initial install or upgrade of HPE Cray EX software products when CSM itself is not being upgraded. See Install or upgrade additional products with IUF for that procedure.

This procedure streamlines the rollout of new images to management nodes. These images are based on the new images provided by the CSM product and customized by the additional HPE Cray EX software products, including the Cray Operating System (COS) and Slingshot Host Software (SHS).

The steps in this procedure alternate between CSM upgrade instructions that do not utilize the IUF and instructions for upgrading additional HPE Cray EX software products whose installation is managed by the IUF.

All stages of iuf are executed in this procedure. All of the new product software provided in the recipe release is deployed and all management NCNs and managed compute nodes and application nodes are rebooted to new images and Configuration Framework Service (CFS) configurations. Manual operations are documented for procedures that are not currently managed by IUF.

The upgrade workflow comprises the following procedures. The diagram shows the workflow and the steps below it provide detailed instructions which must be executed in the order shown.

Upgrade CSM and additional products with IUF

  1. Prepare for Upgrade to Next CSM Major Version in the CSM 1.3 documentation.

  2. CSM preparation, Stage 0.1, and Stage 0.2

    Read the Important Notes section of the CSM 1.3.0 or later to 1.4.0 Upgrade Process documentation and then follow only these CSM instructions in order:

    1. Stage 0.1 - Prepare assets
    2. Stage 0.2 - Prerequisites

  3. Prepare for the upgrade procedure and download product media

    1. Follow the IUF Prepare for the install or upgrade instructions to set environment variables used during the upgrade process.

    2. Download the desired HPE product media defined by the HPC CSM Software Recipe to ${MEDIA_DIR}, which was defined in the previous step.

  4. Product delivery

    Follow the IUF Product delivery instructions.

  5. Configuration

    Follow the IUF Configuration instructions.

  6. Image preparation

    Follow the IUF Image preparation instructions.

  7. CSM Stage 0.4

    Follow the CSM Stage 0.4 - Backup workload manager data instructions.

  8. CSM Stage 0.5

    Follow the CSM Stage 0.5 - Upgrade Ceph and stop local Docker registries instructions.

  9. CSM Stage 0.6

    Follow the CSM Stage 0.6 - Enable Smartmon Metrics on Storage NCNs instructions.

  10. Backup

    Follow the IUF Backup instructions.

  11. Management rollout

    Follow the IUF Management rollout instructions.

  12. CSM Stage 2 and CSM health validation

    Follow these CSM instructions in order:

    1. Stage 2 - CSM Service Upgrades
    2. Validate CSM Health During Upgrade

  13. Deploy product

    Follow these IUF instructions in order:

    1. Deploy product
    2. Validate deployment

  14. Managed rollout

    Follow the IUF Managed rollout instructions.

The IUF upgrade workflow is now complete. Exit any typescript sessions created during the upgrade procedure and remove any installation artifacts, if desired.