Change NCN Image Root Password and SSH Keys on PIT Node

The default SSH keys in the NCN image must be removed. The default password for the root user must be changed. Customize the NCN images by changing the root password and adding different SSH keys for the root account. This procedure shows this process being done on the PIT node during a first time installation of the CSM software.

There is some common preparation before making the Kubernetes image for master nodes and worker nodes, making the Ceph image for utility storage nodes, and then some common cleanup afterwards.

Note: This procedure can only be done before the PIT node is rebuilt to become a normal master node.

Common Preparation

  1. Prepare new SSH keys on the PIT node for the root account in advance. The same key information will be added to both k8s-image and ceph-image.

    Either replace the root public and private SSH keys with your own previously generated keys or generate a new pair with ssh-keygen(1). By default ssh-keygen will create an RSA key, but other types could be chosen and different filenames would need to be substituted in later steps.

    Note: CSM only supports key pairs with empty passphrases (ssh-keygen -N"", or enter an empty passphrase when prompted).

    pit# mkdir /root/.ssh
    pit# ssh-keygen -f /root/.ssh/id_rsa -t rsa
    pit# ls -l /root/.ssh/id_rsa*
    pit# chmod 600 /root/.ssh/id_rsa
    

Kubernetes Image

The Kubernetes image is used by the master and worker nodes.

  1. Change to the working directory for the Kubernetes image.

    pit# cd /var/www/ephemeral/data/k8s
    
  2. Open the image.

    The Kubernetes image will be of the form kubernetes-0.0.57.squashfs in /var/www/ephemeral/data/k8s, but the version number may be different.

    pit# unsquashfs kubernetes-0.0.57.squashfs
    
  3. Remove default SSH keys

    pit# rm -rf squashfs-root/root/.ssh
    pit# rm -f /etc/ssh/*key*
    
  4. Copy the generated public and private SSH keys for the root account into the image.

    This example assumes that an RSA key was generated.

    pit# mkdir -m 0700 squashfs-root/root/.ssh
    pit# cp -p /root/.ssh/id_rsa /root/.ssh/id_rsa.pub squashfs-root/root/.ssh
    
  5. Add the public SSH key for the root account to authorized_keys.

    This example assumes that an RSA key was generated so it adds the id_rsa.pub file to authorized_keys. Note that authorized_keys is being overwritten, not appended.

    pit# cat /root/.ssh/id_rsa.pub > squashfs-root/root/.ssh/authorized_keys
    pit# chmod 640 squashfs-root/root/.ssh/authorized_keys
    
  6. Change into the image root.

    pit# chroot ./squashfs-root
    
  7. Change the password.

    chroot-pit# passwd
    
  8. (Optional) If there are any other things to be changed in the image, they could also be done at this point.

    1. (Optional) Set default timezone on management nodes.

      1. Check whether TZ variable is already set in /etc/environment. The setting for NEWTZ must be a valid timezone from the set under /usr/share/zoneinfo.

        chroot-pit# NEWTZ=US/Pacific
        chroot-pit# grep TZ /etc/environment
        

        Add only if TZ is not present.

        chroot-pit# echo TZ=${NEWTZ} >> /etc/environment
        
      2. Check for utc setting.

        chroot-pit# grep -i utc /srv/cray/scripts/metal/ntp-upgrade-config.sh
        

        Change only if the grep command shows these lines set to UTC.

        chroot-pit# sed -i "s#^timedatectl set-timezone UTC#timedatectl set-timezone $NEWTZ#" /srv/cray/scripts/metal/ntp-upgrade-config.sh
        chroot-pit# sed -i 's/--utc/--localtime/' /srv/cray/scripts/metal/ntp-upgrade-config.sh
        
  9. Create the new SquashFS artifact.

    chroot-pit# /srv/cray/scripts/common/create-kis-artifacts.sh
    
  10. Exit the chroot environment.

    chroot-pit# exit
    
  11. Clean up the SquashFS creation.

    The Kubernetes image directory is /var/www/ephemeral/data/k8s.

    pit# umount -v /var/www/ephemeral/data/k8s/squashfs-root/mnt/squashfs
    
  12. Move new SquashFS image, kernel, and initrd into place.

    pit# mv -v squashfs-root/squashfs/* .
    
  13. Update file permissions on initrd.

    pit# chmod -v 644 initrd.img.xz
    
  14. Rename the new SquashFS, kernel, and initrd to include a new version string.

    If the old name of the SquashFS was kubernetes-0.0.57.squashfs, then its version was ‘0.0.57’, so the newly created version should be renamed to include a version of ‘0.0.57-1’ with an additional dash and a build iteration number of 1. This will help to track what base version was used.

    pit# ls -l old/*squashfs
    -rw-r--r--  1 root root 5135859712 Aug 19 19:10 kubernetes-0.0.57.squashfs
    

    Set the VERSION variable based on the version string displayed by the above command with an incremented suffix added to show a build iteration.

    pit# export VERSION=0.0.57-1
    pit# mv filesystem.squashfs kubernetes-${VERSION}.squashfs
    pit# mv initrd.img.xz initrd.img-${VERSION}.xz
    

    The kernel file will have a name with the kernel version but not this new $VERSION.

    pit# ls -l *kernel
    -rw-r--r--  1 root root    8552768 Aug 19 19:09 5.3.18-24.75-default.kernel
    

    Rename it to include the version string.

    pit# mv 5.3.18-24.75-default.kernel 5.3.18-24.75-default-${VERSION}.kernel
    
  15. Set the boot links. Skip this step if proceeding to the Ceph Image section below.

    pit# cd
    pit# set-sqfs-links.sh
    

The Kubernetes image will have the image changes for the next boot.

Ceph Image

The Ceph image is used by the utility storage nodes.

  1. Change to the working directory for the Ceph image.

    pit# cd /var/www/ephemeral/data/ceph
    
  2. Open the image.

    The Ceph image will be of the form storage-ceph-0.0.47.squashfs in /var/www/ephemeral/data/ceph, but the version number may be different.

    pit# unsquashfs storage-ceph-0.0.47.squashfs
    
  3. Save the old SquashFS image, kernel, and initrd.

    pit# mkdir -v old
    pit# mv -v *squashfs *kernel initrd* old
    
  4. Copy the generated public and private SSH keys for the root account into the image.

    This example assumes that an RSA key was generated.

    pit# cp -p /root/.ssh/id_rsa /root/.ssh/id_rsa.pub squashfs-root/root/.ssh
    
  5. Add the public SSH key for the root account to authorized_keys.

    This example assumes that an RSA key was generated so it adds the id_rsa.pub file to authorized_keys.

    Note that authorized_keys is being overwritten, not appended.

    pit# cat /root/.ssh/id_rsa.pub > squashfs-root/root/.ssh/authorized_keys
    pit# chmod 640 squashfs-root/root/.ssh/authorized_keys
    
  6. Change into the image root.

    pit# chroot ./squashfs-root
    
  7. Change the password.

    chroot-pit# passwd
    
  8. (Optional) If there are any other things to be changed in the image, they could also be done at this point.

    1. (Optional) Set default timezone on management nodes.

      1. Check whether TZ variable is already set in /etc/environment. The setting for NEWTZ must be a valid timezone from the set under /usr/share/zoneinfo.

        chroot-pit# NEWTZ=US/Pacific
        chroot-pit# grep TZ /etc/environment
        

        Add only if TZ is not present.

        chroot-pit# echo TZ=${NEWTZ} >> /etc/environment
        
      2. Check for utc setting.

        chroot-pit# grep -i utc /srv/cray/scripts/metal/ntp-upgrade-config.sh
        

        Change only if the grep command shows these lines set to UTC.

        chroot-pit# sed -i "s#^timedatectl set-timezone UTC#timedatectl set-timezone $NEWTZ#" /srv/cray/scripts/metal/ntp-upgrade-config.sh
        chroot-pit# sed -i 's/--utc/--localtime/' /srv/cray/scripts/metal/ntp-upgrade-config.sh
        
  9. Create the new SquashFS artifact.

    chroot-pit# /srv/cray/scripts/common/create-kis-artifacts.sh
    
  10. Exit the chroot environment.

    chroot-pit# exit
    
  11. Clean up the SquashFS creation.

    The Ceph image directory is /var/www/ephemeral/data/ceph.

    pit# umount -v /var/www/ephemeral/data/ceph/squashfs-root/mnt/squashfs
    
  12. Save old SquashFS image.

    pit# mkdir -v old
    pit# mv -v *squashfs old
    
  13. Move new SquashFS image, kernel, and initrd into place.

    pit# mv -v squashfs-root/squashfs/* .
    
  14. Update file permissions on initrd.

    pit# chmod -v 644 initrd.img.xz
    
  15. Rename the new SquashFS, kernel, and initrd to include a new version string.

    If the old name of the SquashFS was storage-ceph-0.0.47.squashfs, then its version was ‘0.0.47’, so the newly created version should be renamed to include a version of ‘0.0.47-1’ with an additional dash and a build iteration number of 1. This will help to track what base version was used.

    pit# ls -l old/*squashfs
    -rw-r--r--  1 root root 5135859712 Aug 19 19:10 storage-ceph-0.0.47.squashfs
    

    Set the VERSION variable based on the version string displayed by the above command with an incremented suffix added to show a build iteration.

    pit# VERSION=0.0.47-1
    pit# mv filesystem.squashfs storage-ceph-${VERSION}.squashfs
    pit# mv initrd.img.xz initrd.img-${VERSION}.xz
    

    The kernel file will have a name with the kernel version but not this new $VERSION.

    pit# ls -l *kernel
    -rw-r--r--  1 root root    8552768 Aug 19 19:09 5.3.18-24.75-default.kernel
    

    Rename it to include the version string.

    pit# mv 5.3.18-24.75-default.kernel 5.3.18-24.75-default-${VERSION}.kernel
    
  16. Set the boot links.

    pit# cd
    pit# set-sqfs-links.sh
    

The Ceph image will have the image changes for the next boot.

Common Cleanup

  1. Clean up temporary storage used to prepare images.

    These could be removed now or after verification that the nodes are able to boot successfully with the new images.

    pit# cd /var/www/ephemeral/data
    pit# rm -rf ceph/old k8s/old