Update Default Air-Cooled BMC and Leaf-BMC Switch SNMP Credentials

This procedure updates the default credentials used when new air-cooled hardware is discovered for the first time. This includes the default Redfish credentials used for new air-cooled NodeBMCs and Slingshot switch BMCs (RouterBMCs), and SNMP credentials for new management leaf-BMC switches.

IMPORTANT: After this procedure is completed, all future air-cooled hardware added to the system will be assumed to be configured with the new global default credential.

NOTE: This procedure will not update the Redfish or SNMP credentials for existing air-cooled devices. To change the credentials on existing air-cooled hardware follow the Change Air-Cooled Node BMC Credentials and Change SNMP Credentials on Leaf-BMC Switches procedures.

Limitation

The default global credentials used for liquid-cooled BMCs in the Change Cray EX Liquid-Cooled Cabinet Global Default Password procedure needs to be the same as the one used in this procedure for air-cooled BMCs (River hardware).

Procedure

The River Endpoint Discovery Service (REDS) sealed secret contains the default global credential used by REDS.

Follow the Redeploying a Chart procedure with the following specifications:

  • Chart name: cray-hms-reds

  • Base manifest name: core-services

  • When reaching the step to update the customizations, perform the following steps:

    Only follow these steps as part of the previously linked chart redeploy procedure.

    1. Clone the CSM repository.

      ncn-mw# git clone https://github.com/Cray-HPE/csm.git

    2. Acquire sealed secret keys.

      ncn-mw# mkdir -pv certs &&
        kubectl -n kube-system get secret sealed-secrets-key -o jsonpath='{.data.tls\.crt}' | base64 -d > certs/sealed_secrets.crt &&
        kubectl -n kube-system get secret sealed-secrets-key -o jsonpath='{.data.tls\.key}' | base64 -d > certs/sealed_secrets.key

    3. Modify REDS sealed secret to use new global default credentials.

      1. Inspect the original default Redfish credentials used by REDS and HMS discovery.

        ncn-mw# ./utils/secrets-decrypt.sh cray_reds_credentials ./certs/sealed_secrets.key ./customizations.yaml | jq .data.vault_redfish_defaults -r | base64 -d | jq

        Expected output looks similar to the following:

        {
    "Cray": {
        "Username": "root",
        "Password": "foo"
    }
}

      2. Inspect the original default switch SNMP credentials used by REDS and HMS discovery.

        ncn-mw# ./utils/secrets-decrypt.sh cray_reds_credentials ./certs/sealed_secrets.key ./customizations.yaml | jq .data.vault_switch_defaults -r | base64 -d | jq

        Expected output looks similar to the following:

        {
    "SNMPUsername": "testuser",
    "SNMPAuthPassword": "foo",
    "SNMPPrivPassword": "bar"
}

      3. Update the default credentials in customizations.yaml for REDS and HMS discovery to use.

        1. Specify the desired default Redfish credentials.

          ncn-mw# echo '{"Cray":{"Username":"root","Password":"foobar"}}' | base64 > reds.redfish.creds.json.b64

        2. Specify the desired default SNMP credentials.

          ncn-mw# echo '{"SNMPUsername":"testuser","SNMPAuthPassword":"foo1","SNMPPrivPassword":"bar2"}' | base64 > reds.switch.creds.json.b64

      4. Update and regenerate the cray_reds_credentials sealed secret.

        ncn-mw# cat << EOF | yq w - 'data.vault_redfish_defaults' "$(<reds.redfish.creds.json.b64)" | yq w - 'data.vault_switch_defaults' "$(<reds.switch.creds.json.b64)" | yq r -j - | ./utils/secrets-encrypt.sh | yq w -f - -i ./customizations.yaml 'spec.kubernetes.sealed_secrets.cray_reds_credentials'
{
    "kind": "Secret",
    "apiVersion": "v1",
    "metadata": {
        "name": "cray-reds-credentials",
        "namespace": "services",
        "creationTimestamp": null
    },
    "data": {}
}
EOF

      5. Decrypt generated secret for review.

        1. Review the default Redfish credentials.

          ncn-mw# ./utils/secrets-decrypt.sh cray_reds_credentials ./certs/sealed_secrets.key ./customizations.yaml | jq .data.vault_redfish_defaults -r | base64 -d | jq

          Expected output looks similar to the following:

          {
    "Username": "root",
    "Password": "foobar"
}

        2. Review the default switch SNMP credentials.

          ncn-mw# ./utils/secrets-decrypt.sh cray_reds_credentials ./certs/sealed_secrets.key ./customizations.yaml | jq .data.vault_switch_defaults -r | base64 -d | jq

          Expected output looks similar to the following:

          {
    "SNMPUsername": "testuser",
    "SNMPAuthPassword": "foo1",
    "SNMPPrivPassword": "bar2"
}

  • When reaching the step to validate the redeployed chart, perform the following steps:

    Only follow these steps as part of the previously linked chart redeploy procedure.

    1. Wait for the REDS Vault loader job to run to completion.

      ncn-mw# kubectl -n services wait job cray-reds-vault-loader --for=condition=complete --timeout=5m

    2. Verify that the default Redfish credentials have updated in Vault.

      ncn-mw# VAULT_PASSWD=$(kubectl -n vault get secrets cray-vault-unseal-keys -o json | jq -r '.data["vault-root"]' |  base64 -d)
ncn-mw# kubectl -n vault exec -it cray-vault-0 -c vault -- env VAULT_TOKEN=$VAULT_PASSWD VAULT_ADDR=http://127.0.0.1:8200 vault kv get secret/reds-creds/defaults

      Expected output:

      ==== Data ====
Key     Value
---     -----
Cray    map[password:foobar username:root]

    3. Verify that the default SNMP credentials have updated in Vault.

      ncn-mw# kubectl -n vault exec -it cray-vault-0 -c vault -- env VAULT_TOKEN=$VAULT_PASSWD VAULT_ADDR=http://127.0.0.1:8200 vault kv get secret/reds-creds/switch_defaults

      Expected output:

      ========== Data ==========
Key                 Value
---                 -----
SNMPAuthPassword    foo1
SNMPPrivPassword    bar2
SNMPUsername        testuser

  • Make sure to perform the entire linked procedure, including the step to save the updated customizations.