Non-compute node (NCN) personalization applies post-boot configuration to the
HPE Cray EX management nodes. Several HPE Cray EX product environments outside
of CSM require NCN personalization to function. Consult the manual for each
product to configure them on NCNs by referring to the
1.5 HPE Cray EX System Software Getting Started Guide S-8000
on the HPE Customer Support Center.
This procedure defines the NCN personalization process for the CSM product using the Configuration Framework Service (CFS).
During a fresh install, carry out these procedures in order. Later, individual procedures may be re-run as needed.
This procedure should be run during CSM installation and any later time when the SSH keys need to be changed per site requirements.
The goal of passwordless SSH is to enable an easy way for interactive passwordless SSH from and between CSM product environments (management nodes) to downstream managed product environments (COS, UAN, etc), without requiring each downstream environment to create and apply individual changes to NCNs. Passwordless SSH from downstream nodes into CSM management nodes is not intended or supported.
Passwordless SSH keypairs for the Cray System Management (CSM) are created
automatically and maintained with a Kubernetes deployment and staged into
Kubernetes secrets (csm-private-key) and ConfigMaps (csm-public-key) in the
services namespace. Administrators can use these provided keys, provide their
own keys, or use their own solution for authentication.
The management of keys on NCNs is achieved by the trust-csm-ssh-keys and
passwordless-ssh Ansible roles in the CSM configuration management repository.
The SSH keypair is applied to management nodes using NCN personalization.
NOTE: CFS itself does not use the CSM-provided (or user-supplied) SSH keys to make connections between nodes. CFS will continue to function if passwordless SSH is disabled between CSM and other product environments.
The default CSM Ansible plays are already configured to enable Passwordless SSH
by default. No further action is necessary before proceeding to
Configure the root password in Vault.
Administrators may elect to replace the CSM-provided keys with their own custom keys.
Set variables to the locations of the public and private SSH key files.
Replace the values in the examples below with the paths to the desired key files on the system.
ncn-mw# PUBLIC_KEY_FILE=/path/to/id_rsa-csm.pub
ncn-mw# PRIVATE_KEY_FILE=/path/to/id_rsa-csm
Provide the custom keys by script or manually.
There are two options for providing the keys.
Provide custom SSH keys by script.
The replace_ssh_keys.sh script can be used to replace the keys from files.
The
docs-csmRPM must be installed in order to use this script. See Check for Latest Documentation
ncn-mw# /usr/share/doc/csm/scripts/operations/configuration/replace_ssh_keys.sh \
--public-key-file "${PUBLIC_KEY_FILE}" --private-key-file "${PRIVATE_KEY_FILE}"
Manually provide custom SSH keys
The keys stored in Kubernetes can be updated directly.
Replace the private key half:
ncn-mw# KEY64=$(cat "${PRIVATE_KEY_FILE}" | base64) &&
kubectl get secret -n services csm-private-key -o json | \
jq --arg value "$KEY64" '.data["value"]=$value' | kubectl apply -f - &&
unset KEY64
Replace the public key half:
ncn-mw# kubectl delete configmap -n services csm-public-key &&
cat "${PUBLIC_KEY_FILE}" | base64 > ./value &&
kubectl create configmap --from-file value csm-public-key --namespace services &&
rm ./value
Passwordless SSH with the provided keys will be setup once NCN personalization runs on the NCNs.
Proceed Configure the root password in Vault.
Local site security requirements may preclude use of passwordless SSH access between
management nodes. If this is the case, remove or comment out the invocation of the
trust-csm-public-keys role in Ansible plays in the configuration repositories
of the environments where it is configured. By default, the HPE Cray Operating System (COS) and User Access Node (UAN) configurations enable passwordless SSH.
Refer to the following in the documentation for each product stream to change
the default configuration:
COS: Refer to the VCS Configuration section in the Install or Upgrade COS
procedure.UAN: Refer to Create UAN Boot Images and UAN Ansible Roles procedures.Modifying Ansible plays in a configuration repository will require a new commit and subsequent update of the configuration layer associated with the product.
Proceed Configure the root password in Vault.
Use this procedure if switching from custom keys to the default CSM SSH keys only; otherwise it should be skipped.
The keys can be deleted from Kubernetes directly. The csm-ssh-keys Kubernetes deployment
provided by CSM periodically checks the ConfigMap and secret containing the key information.
If these entries do not exist, it will recreate them from the default CSM keys. Therefore, in
order to manually restore the keys, delete the associated ConfigMap and secret. The default CSM-provided
keys will be republished.
Delete the csm-private-key Kubernetes secret.
ncn-mw# kubectl delete secret -n services csm-private-key
Delete the csm-public-key Kubernetes ConfigMap.
ncn-mw# kubectl delete configmap -n services csm-public-key
root password in VaultThe root password is applied to NCNs by using the csm.password Ansible role
located in the CSM configuration management repository. root passwords are set
and managed in Vault.
Configure Root Password in Vault
procedure in Update NCN Passwords.Prior to running management NCN personalization, gather the following information:
| Field | Value | Description |
|---|---|---|
cloneUrl |
https://api-gw-service-nmn.local/vcs/cray/csm-config-management.git |
CSM configuration repository |
commit |
Example: 5081c1ecea56002df41218ee39f6030c3eebdf27 |
CSM configuration commit hash |
name |
Example: csm-1.9.21 |
CSM configuration layer name |
playbook |
site.yml |
Default site-wide Ansible playbook for CSM |
Retrieve the commit in the repository to use for configuration. If changes have been made to the default branch that was imported during a CSM installation or upgrade, use the commit containing the changes.
If no changes have been made, then the latest commit on the default branch for this version of CSM should be used.
Find the commit in the cray-product-catalog ConfigMap for the current version of CSM. For example:
ncn-mw# kubectl -n services get cm cray-product-catalog -o jsonpath='{.data.csm}'
Look for something similar to the following in the output:
1.0.1:
configuration:
clone_url: https://vcs.SYSTEM_DOMAIN_NAME/vcs/cray/csm-config-management.git
commit: 43ecfa8236bed625b54325ebb70916f55884b3a4
import_branch: cray/csm/1.6.12
import_date: 2021-07-28 03:26:01.869501
ssh_url: git@vcs.SYSTEM_DOMAIN_NAME:cray/csm-config-management.git
The commit will be different for each system and version of CSM. For
this example, it is 43ecfa8236bed625b54325ebb70916f55884b3a4.
Craft a new configuration layer entry for the new CSM.
{
"name": "csm-<version>",
"cloneUrl": "https://api-gw-service-nmn.local/vcs/cray/csm-config-management.git",
"playbook": "site.yml",
"commit": "<retrieved git commit ID>"
}
Create or update the NCN personalization configuration in CFS.
NOTEThe CSM configuration layer MUST be the first layer in the NCN personalization CFS configuration.