NCN Images

Overview of NCN images

The management non-compute nodes (NCNs) boot from images which are created from layers on top of a common base image. The common image is customized with a kubernetes layer for the master nodes and worker nodes. The common image is customized with a storage-ceph layer for the utility storage nodes.

When booting NCNs, an administrator will need to choose between stable (Release) and unstable (pre-release/development) images.

In short, each image (for instance, kubernetes and storage-ceph) inherit from the non-compute-common layer. Operationally, these are all that matter; the common layer, Kubernetes layer, Ceph layer, and any other new images.

To boot an NCN, there are three required artifacts for each node-type (kubernetes-master/worker, storage-ceph):

  1. The Kubernetes SquashFS (stable or unstable)

    • initrd.img-[RELEASE].xz
    • $version-[RELEASE].kernel
    • kubernetes-[RELEASE].squashfs

  2. The CEPH SquashFS (stable or unstable)

    • initrd.img-[RELEASE].xz
    • $version-[RELEASE].kernel
    • storage-ceph-[RELEASE].squashfs

LiveCD server

  1. (pit#) View the current ephemeral data payload:

    ls -l /var/www

    Expected output (on a new PIT without any boot links created):

    total 8
drwxr-xr-x 1 dnsmasq tftp 4096 Dec 17 21:20 boot
drwxr-xr-x 7 root    root 4096 Dec  2 04:45 ephemeral

    ls -l /var/www/ephemeral/data/*

    Expected output:

    /var/www/ephemeral/data/ceph:
total 4
drwxr-xr-x 2 root root 4096 Dec 17 21:42 0.3.33

/var/www/ephemeral/data/k8s:
total 4
drwxr-xr-x 2 root root 4096 Dec 17 21:26 0.3.33

  2. (pit#) Setup the “booting repositories”:

    /root/bin/set-sqfs-links.sh

    Expected output:

    Resolving images to boot ...
Images resolved
Kubernetes Boot Selection:
kernel: /var/www/ephemeral/data/k8s/fe775c6-1659395320990/5.3.18-150300.59.76-default-fe775c6-1659395320990.kernel
initrd: /var/www/ephemeral/data/k8s/fe775c6-1659395320990/initrd.img-fe775c6-1659395320990.xz
squash: /var/www/ephemeral/data/k8s/fe775c6-1659395320990/secure-kubernetes-fe775c6-1659395320990.squashfs
Storage Boot Selection:
kernel: /var/www/ephemeral/data/ceph/fe775c6-1659395320990/5.3.18-150300.59.76-default-fe775c6-1659395320990.kernel
initrd: /var/www/ephemeral/data/ceph/fe775c6-1659395320990/initrd.img-fe775c6-1659395320990.xz
squash: /var/www/ephemeral/data/ceph/fe775c6-1659395320990/secure-storage-ceph-fe775c6-1659395320990.squashfs
Attempting to set all known BMCs (from /etc/conman.conf) to DHCP mode
current BMC count: 0
Waiting on 9 to request DHCP ...
Waiting on 9 to request DHCP ...
Waiting on 9 to request DHCP ...
All [9] expected BMCs have requested DHCP.
/root/bin/set-sqfs-links.sh is creating boot directories for each NCN with a BMC that has a lease in /var/lib/misc/dnsmasq.leases
NOTE: Nodes without boot directories will still boot the non-destructive iPXE binary for bare-metal discovery usage.
WARNING: CSM_RELEASE was not set, images will be stored in their default location on the node(s) at /run/initramfs/live/LiveOS/
/var/www is ready.

  3. (pit#) View the currently set links.

    ls -l /var/www/ncn-*

    Expected output:

    ncn-m001:
total 4
lrwxrwxrwx 1 root root 53 Dec 26 06:11 rootfs -> ../ephemeral/data/k8s/0.0.8/kubernetes-0.0.8.squashfs
lrwxrwxrwx 1 root root 47 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/k8s/0.0.8/initrd.img-0.0.8.xz
lrwxrwxrwx 1 root root 61 Dec 26 06:11 kernel -> ../ephemeral/data/k8s/0.0.8/5.3.18-24.37-default-0.0.8.kernel

ncn-m002:
total 4
lrwxrwxrwx 1 root root 53 Dec 26 06:11 rootfs -> ../ephemeral/data/k8s/0.0.8/kubernetes-0.0.8.squashfs
lrwxrwxrwx 1 root root 47 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/k8s/0.0.8/initrd.img-0.0.8.xz
lrwxrwxrwx 1 root root 61 Dec 26 06:11 kernel -> ../ephemeral/data/k8s/0.0.8/5.3.18-24.37-default-0.0.8.kernel

ncn-m003:
total 4
lrwxrwxrwx 1 root root 53 Dec 26 06:11 rootfs -> ../ephemeral/data/k8s/0.0.8/kubernetes-0.0.8.squashfs
lrwxrwxrwx 1 root root 47 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/k8s/0.0.8/initrd.img-0.0.8.xz
lrwxrwxrwx 1 root root 61 Dec 26 06:11 kernel -> ../ephemeral/data/k8s/0.0.8/5.3.18-24.37-default-0.0.8.kernel

ncn-s001:
total 4
lrwxrwxrwx 1 root root 56 Dec 26 06:11 rootfs -> ../ephemeral/data/ceph/0.0.7/storage-ceph-0.0.7.squashfs
lrwxrwxrwx 1 root root 48 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/ceph/0.0.7/initrd.img-0.0.7.xz
lrwxrwxrwx 1 root root 62 Dec 26 06:11 kernel -> ../ephemeral/data/ceph/0.0.7/5.3.18-24.37-default-0.0.7.kernel

ncn-s002:
total 4
lrwxrwxrwx 1 root root 56 Dec 26 06:11 rootfs -> ../ephemeral/data/ceph/0.0.7/storage-ceph-0.0.7.squashfs
lrwxrwxrwx 1 root root 48 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/ceph/0.0.7/initrd.img-0.0.7.xz
lrwxrwxrwx 1 root root 62 Dec 26 06:11 kernel -> ../ephemeral/data/ceph/0.0.7/5.3.18-24.37-default-0.0.7.kernel

ncn-s003:
total 4
lrwxrwxrwx 1 root root 56 Dec 26 06:11 rootfs -> ../ephemeral/data/ceph/0.0.7/storage-ceph-0.0.7.squashfs
lrwxrwxrwx 1 root root 48 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/ceph/0.0.7/initrd.img-0.0.7.xz
lrwxrwxrwx 1 root root 62 Dec 26 06:11 kernel -> ../ephemeral/data/ceph/0.0.7/5.3.18-24.37-default-0.0.7.kernel

ncn-w002:
total 4
lrwxrwxrwx 1 root root 53 Dec 26 06:11 rootfs -> ../ephemeral/data/k8s/0.0.8/kubernetes-0.0.8.squashfs
lrwxrwxrwx 1 root root 47 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/k8s/0.0.8/initrd.img-0.0.8.xz
lrwxrwxrwx 1 root root 61 Dec 26 06:11 kernel -> ../ephemeral/data/k8s/0.0.8/5.3.18-24.37-default-0.0.8.kernel

ncn-w003:
total 4
lrwxrwxrwx 1 root root 53 Dec 26 06:11 rootfs -> ../ephemeral/data/k8s/0.0.8/kubernetes-0.0.8.squashfs
lrwxrwxrwx 1 root root 47 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/k8s/0.0.8/initrd.img-0.0.8.xz
lrwxrwxrwx 1 root root 61 Dec 26 06:11 kernel -> ../ephemeral/data/k8s/0.0.8/5.3.18-24.37-default-0.0.8.kernel