Change Credentials on ServerTech PDUs

This procedure changes password used by the admn user on ServerTech PDUs. Either a single PDU can be updated to a new credential, or all ServerTech PDUs in the system can be updated to the same global credentials.

NOTES:

Prerequisites

  • All of the commands in these procedures should be run from a master or worker node, unless otherwise indicated.

  • The Cray command line interface (CLI) is initialized and configured on the system. See Configure the Cray CLI.

  • The PDU is accessible over the network. A PDU can be reachable by its component name (xname) hostname, but may not yet be discovered by HSM.

  • PDUs are manufactured by ServerTech. This can be verified by the following command

    ncn-mw# PDU=x3000m0
    ncn-mw# curl -k https://$PDU -i | grep Server
    

    Expected output for a ServerTech PDU:

    Server: ServerTech-AWS/v8.0v
    

Procedure

  1. List the ServerTech PDUs currently discovered in the system.

    ncn-mw# cray hsm inventory redfishEndpoints list --type CabinetPDUController --format json |
        jq -r '.RedfishEndpoints[] | select(.FQDN | contains("rts")).ID'
    

    Example output:

    x3000m0
    
  2. Set up Vault password variable and command alias.

    ncn-mw# VAULT_PASSWD=$(kubectl -n vault get secrets cray-vault-unseal-keys -o json | jq -r '.data["vault-root"]' |  base64 -d)
    ncn-mw# alias vault='kubectl -n vault exec -i cray-vault-0 -c vault -- env VAULT_TOKEN=$VAULT_PASSWD VAULT_ADDR=http://127.0.0.1:8200 VAULT_FORMAT=json vault'
    
  3. Look up the existing password for the admn user.

    • To extract the global credentials from Vault for the PDUs:

      ncn-mw# vault kv get secret/pdu-creds/global/pdu
      
    • To extract the credentials from Vault for a single PDU:

      ncn-mw# PDU=x3000m0
      ncn-mw# vault kv get secret/pdu-creds/$PDU
      
  4. Store the existing password for the admn user.

    ncn-mw# read -s OLD_PDU_PASSWORD
    
  5. Specify the new desired password for the admn user. The new password must follow the following criteria:

    • Minimum of 8 characters
    • At least 1 uppercase letter
    • At least 1 lowercase letter
    • At least 1 number character
    ncn-mw# read -s NEW_PDU_PASSWORD
    
  6. Change and update the password for ServerTech PDUs.

    Either change the credentials on a single PDU or change all ServerTech PDUs to the same global default value:

    • Update the password on a single ServerTech PDU

      NOTE: To change the password on a single PDU, that PDU must be successfully discovered by HSM.

      1. Set the PDU hostname to change the admn credentials:

        ncn-mw# PDU=x3000m0
        
      2. Verify that the PDU is reachable:

        ncn-mw# ping $PDU
        
      3. Change password for the admn user on the ServerTech PDU.

        ncn-mw# curl -i -k -u "admn:$OLD_PDU_PASSWORD" -X PATCH https://$PDU/jaws/config/users/local/admn \
            -d $(jq --arg PASSWORD "$NEW_PDU_PASSWORD" -nc '{password: $PASSWORD}')
        

        Expected output upon a successful password change:

        HTTP/1.1 204 No Content
        Content-Type: text/html
        Transfer-Encoding: chunked
        Server: ServerTech-AWS/v8.0p
        Set-Cookie: C5=1883488164; path=/
        Connection: close
        Pragma: JAWS v1.01
        
      4. Update the PDU credentials stored in Vault.

        ncn-mw# vault kv get secret/pdu-creds/$PDU |
                jq --arg PASSWORD "$NEW_PDU_PASSWORD" '.data | .Password=$PASSWORD' |
                vault kv put secret/pdu-creds/$PDU -
        
    • Update all ServerTech PDUs in the system to the same password.

      1. Change password for the admn user on the ServerTech PDUs currently discovered in the system.

        ncn-mw# for PDU in $(cray hsm inventory redfishEndpoints list --type CabinetPDUController --format json |
                  jq -r '.RedfishEndpoints[] | select(.FQDN | contains("rts")).ID'); do
                      echo "Updating password on $PDU"
                      curl -i -k -u "admn:$OLD_PDU_PASSWORD" -X PATCH https://$PDU/jaws/config/users/local/admn \
                            -d $(jq --arg PASSWORD "$NEW_PDU_PASSWORD" -nc '{password: $PASSWORD}')
                done
        

        Expected output upon a successful password change:

        Updating password on x3000m0
        HTTP/1.1 204 No Content
        Content-Type: text/html
        Transfer-Encoding: chunked
        Server: ServerTech-AWS/v8.0p
        Set-Cookie: C5=1883488164; path=/
        Connection: close
        Pragma: JAWS v1.01
        Updating password on x3001m0
        HTTP/1.1 204 No Content
        Content-Type: text/html
        Transfer-Encoding: chunked
        Server: ServerTech-AWS/v8.0p
        Set-Cookie: C5=1883488164; path=/
        Connection: close
        Pragma: JAWS v1.01
        
      2. Update Vault for all ServerTech PDUs in the system to the same password:

        ncn-mw# for PDU in $(cray hsm inventory redfishEndpoints list --type CabinetPDUController --format json |
                  jq -r '.RedfishEndpoints[] | select(.FQDN | contains("rts")).ID'); do
                      echo "Updating password on $PDU"
                      vault kv get secret/pdu-creds/$PDU |
                        jq --arg PASSWORD "$NEW_PDU_PASSWORD" '.data | .Password=$PASSWORD' |
                        vault kv put secret/pdu-creds/$PDU -
                done
        

    NOTE: After five minutes, the previous credential should stop working as the existing sessions time out.

  7. Restart the Redfish Translation Service (RTS) to pickup the new PDU credentials.

    ncn-mw# kubectl -n services rollout restart deployment cray-hms-rts
    ncn-mw# kubectl -n services rollout status deployment cray-hms-rts
    
  8. Wait for RTS to initialize itself.

    ncn-mw# sleep 3m
    
  9. Verify that RTS was able to communicate with the PDUs with the updated credentials.

    ncn-mw# kubectl -n services exec -it deployment/cray-hms-rts -c cray-hms-rts-redis -- redis-cli keys '*/redfish/v1/Managers'
    

    Expected output for a system with two PDUs.

    1) "x3000m0/redfish/v1/Managers"
    2) "x3001m0/redfish/v1/Managers"