HPE PDU Admin Procedures

The following procedures are used to manage the HPE Power Distribution Unit (PDU):

IMPORTANT: Because of the polling method used to process sensor data from the HPE PDU, telemetry data may take up to six minutes to refresh; this includes the outlet status reported by the Hardware State Manager (HSM).

Verify PDU vendor

(ncn-mw#) If the PDU is accessible over the network, the following can be used to determine the vendor of the PDU.

PDU=x3000m0
curl -k -s --compressed  https://$PDU -i | grep Server:
  • Example ServerTech output:

    Server: ServerTech-AWS/v8.0v
    
  • Example HPE output:

    Server: HPE/1.4.0
    

This document covers HPE PDU procedures.

Connect to HPE PDU web interface

Connect and log in to the HPE PDU web interface. Access to the HPE PDU web interface is required for the other administrative procedures in this section.

Prerequisites

The following is needed before running this procedure:

  • IP address or domain name of ncn-m001
  • Component name (xname) of the HPE PDU
  • root password for ncn-m001
  • admin password for HPE PDU (default: 12345678)
  1. Create an SSH tunnel from a local PC/MAC/Linux machine:

    ssh -L 8443:{PDU_xname}:443 -N root@{ncn-m001_ip}
    

    In this example, {PDU_xname} is the component name (xname) of the PDU and {ncn-m001_ip} is the IP address of ncn-m001.

    Enter the root password for ncn-m001 when prompted.

    This command will not complete. It should be left running until the SSH tunnel is no longer needed. At that point, it can be exited with control-C.

  2. Connect to https://localhost:8443 using a web browser.

    This must be done on the system where the SSH tunnel was created in the previous step.

  3. Log in with the admin username.

    Enter the admin password. If the admin password has never been changed, then there will be a prompt to change it.

HPE PDU initial set-up

Set up an HPE PDU for administrative use by completing the following tasks:

  1. Connect to the HPE PDU Web Interface and log in as admin.

    See Connect to HPE PDU web interface.

Ensure that Redfish is enabled

  1. Use the Settings icon (gear in computer monitor in top right corner) to navigate to Network Settings.

  2. Verify that there is a check next to RESTapi Access.

    If there is not, then click the Edit icon (pencil) and enable.

Add the default user

  1. Use the admin menu (top right corner) to navigate to User Accounts.
  2. Click on the Add User button.
  3. Use the form to add the username and password for the default River user. Assign the role Administrator to that user.

Enable outlet control

  1. Using the Home icon (house in top right corner) navigate to Control & Manage.
  2. Verify that the Outlet Control Enable switch on the top of the page is selected (green).

Update HPE PDU firmware

Verify that the firmware version for the HPE PDU is 2.0.0.L. If it is not, then a firmware update is required.

Check firmware version

  1. Connect to the HPE PDU Web Interface and log in as admin.

    See Connect to HPE PDU web interface.

  2. Select the Home icon (house in the top right corner) and navigate to Identification.

    The version will be displayed. If the version is not 2.0.0.L, then Update firmware.

Update firmware

  1. Download version 2.0.0.L firmware from HPE Support.

    This will download an .exe file, which is a self-extracting zip archive.

  2. Extract the firmware files.

    If using a Windows system, run the .exe file to extract the files. Otherwise use an unzip program on the file.

    One of the files extracted will be named HPE.FW. That is the firmware file needed for uploading.

  3. Connect to the HPE PDU Web Interface and log in as admin.

    See Connect to HPE PDU web interface.

  4. Use the Settings icon (gear in computer monitor in top right corner) to navigate to System Management.

  5. Click the Update Firmware button.

  6. Click Choose File and select the HPE.FW file that was downloaded.

  7. Click Upload button.

    The firmware will be updated and the PDU management processor will restart.

Change HPE PDU user passwords

Change the password of any existing user account using the HPE PDU web interface.

  1. Connect to the HPE PDU Web Interface and log in as admin.

    See Connect to HPE PDU web interface.

  2. Use the admin menu (top right corner) to navigate to User Accounts.

  3. Click on the Edit icon (pencil) next to the user.

  4. Enter the new password and make any other desired changes for that user account.

  5. Click the Save button.

Discover HPE PDU after upgrading CSM

Use the following procedure to ensure that the hms-discovery job and Redfish Translation Service (RTS) correctly discover HPE PDUs when upgrading to CSM 1.2 from an earlier release.

IMPORTANT: This procedure is only needed when upgrading CSM, not performing a fresh install. Run this procedure after CSM has been fully upgraded, including the discovery job.

  1. In CSM 1.0 and earlier releases, the hms-discovery job and RTS treated all PDUs as if were made by ServerTech.

    After the upgrade to CSM 1.2, RTS will still think that the HPE PDUs in the system are ServerTech PDUs. Remove these erroneous HPE PDU entries for RTS from Vault.

    1. (ncn-mw#) Get Vault password and create Vault alias.

      VAULT_PASSWD=$(kubectl -n vault get secrets cray-vault-unseal-keys -o json | jq -r '.data["vault-root"]' |  base64 -d)
      alias vault='kubectl -n vault exec -i cray-vault-0 -c vault -- env VAULT_TOKEN=$VAULT_PASSWD VAULT_ADDR=http://127.0.0.1:8200 VAULT_FORMAT=json vault'
      
    2. (ncn-mw#) Identify HPE PDUs known by RTS.

      vault kv list secret/pdu-creds
      

      Example output:

      [
        "global/",
        "x3000m0",
        "x3000m1"
      ]
      
    3. (ncn-mw#) Remove each identified HPE PDU from Vault.

      Repeat the following command for each HPE PDU identified in the output of the previous sub-step.

      PDU=x3000m0
      vault kv delete secret/pdu-creds/$PDU
      
    4. (ncn-mw#) Restart the Redfish Translation Service (RTS).

      kubectl -n services rollout restart deployment cray-hms-rts
      kubectl -n services rollout status deployment cray-hms-rts
      
  2. (ncn-mw#) Find the list of PDU MAC addresses.

    The ID field in each element is the normalized MAC address of each PDU:

    cray hsm inventory ethernetInterfaces list --type CabinetPDUController
    
  3. (ncn-mw#) Use the returned ID from the previous step to delete each HPE PDU MAC address from HSM.

    cray hsm inventory ethernetInterfaces delete {ID}
    

    On the next hms-discovery job run, it should relocate the deleted PDUs and discover them correctly as HPE PDUs.

  4. (ncn-mw#) After waiting five minutes, verify that the Ethernet interfaces that were previously deleted are now present:

    sleep 300
    cray hsm inventory ethernetInterfaces list --type CabinetPDUController
    
  5. (ncn-mw#) Verify that the Redfish endpoints for the PDUs exist and are DiscoverOK.

    cray hsm inventory redfishEndpoints list --type CabinetPDUController