The management non-compute nodes (NCNs) boot from images which are created from layers on top of a common base image.
The common image is customized with a kubernetes layer for the master nodes and worker nodes.
The common image is customized with a storage-ceph layer for the utility storage nodes.
When booting NCNs, an administrator will need to choose between stable (Release) and unstable (pre-release/development) images.
In short, each image (for instance, kubernetes and storage-ceph) inherit from the non-compute-common layer. Operationally, these are all
that matter; the common layer, Kubernetes layer, Ceph layer, and any other new images.
To boot an NCN, there are three required artifacts for each node-type (kubernetes-master/worker, storage-ceph):
The Kubernetes SquashFS (stable or unstable)
initrd.img-[RELEASE]-[ARCH].xz$version-[RELEASE]-[ARCH].kernelkubernetes-[RELEASE]-[ARCH].squashfsThe CEPH SquashFS (stable or unstable)
initrd.img-[RELEASE].xz$version-[RELEASE].kernelstorage-ceph-[RELEASE]-[ARCH].squashfs(pit#) View the current ephemeral data payload:
ls -l /var/www
Expected output (on a new PIT without any boot links created):
total 8
drwxr-xr-x 1 dnsmasq tftp 4096 Dec 17 21:20 boot
drwxr-xr-x 7 root root 4096 Dec 2 04:45 ephemeral
ls -l /var/www/ephemeral/data/*
Expected output:
/var/www/ephemeral/data/ceph:
total 4
drwxr-xr-x 2 root root 4096 Dec 17 21:42 0.3.33
/var/www/ephemeral/data/k8s:
total 4
drwxr-xr-x 2 root root 4096 Dec 17 21:26 0.3.33
(pit#) Setup the “booting repositories”:
/root/bin/set-sqfs-links.sh
Expected output:
Resolving images to boot ...
Images resolved
Kubernetes Boot Selection:
kernel: /var/www/ephemeral/data/k8s/fe775c6-1659395320990/5.3.18-150300.59.76-default-fe775c6-1659395320990-x86_64.kernel
initrd: /var/www/ephemeral/data/k8s/fe775c6-1659395320990/initrd.img-fe775c6-1659395320990-x86_64.xz
squash: /var/www/ephemeral/data/k8s/fe775c6-1659395320990/secure-kubernetes-fe775c6-1659395320990-x86_64.squashfs
Storage Boot Selection:
kernel: /var/www/ephemeral/data/ceph/fe775c6-1659395320990/5.3.18-150300.59.76-default-fe775c6-1659395320990-x86_64.kernel
initrd: /var/www/ephemeral/data/ceph/fe775c6-1659395320990/initrd.img-fe775c6-1659395320990-x86_64.xz
squash: /var/www/ephemeral/data/ceph/fe775c6-1659395320990/secure-storage-ceph-fe775c6-1659395320990-x86_64.squashfs
Attempting to set all known BMCs (from /etc/conman.conf) to DHCP mode
current BMC count: 0
Waiting on 9 to request DHCP ...
Waiting on 9 to request DHCP ...
Waiting on 9 to request DHCP ...
All [9] expected BMCs have requested DHCP.
/root/bin/set-sqfs-links.sh is creating boot directories for each NCN with a BMC that has a lease in /var/lib/misc/dnsmasq.leases
NOTE: Nodes without boot directories will still boot the non-destructive iPXE binary for bare-metal discovery usage.
WARNING: CSM_RELEASE was not set, images will be stored in their default location on the node(s) at /run/initramfs/live/LiveOS/
/var/www is ready.
(pit#) View the currently set links.
ls -l /var/www/ncn-*
Expected output:
ncn-m001:
total 4
lrwxrwxrwx 1 root root 53 Dec 26 06:11 rootfs -> ../ephemeral/data/k8s/0.0.8/kubernetes-0.0.8-x86_64.squashfs
lrwxrwxrwx 1 root root 47 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/k8s/0.0.8/initrd.img-0.0.8-x86_64.xz
lrwxrwxrwx 1 root root 61 Dec 26 06:11 kernel -> ../ephemeral/data/k8s/0.0.8/5.3.18-24.37-default-0.0.8-x86_64.kernel
ncn-m002:
total 4
lrwxrwxrwx 1 root root 53 Dec 26 06:11 rootfs -> ../ephemeral/data/k8s/0.0.8/kubernetes-0.0.8-x86_64.squashfs
lrwxrwxrwx 1 root root 47 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/k8s/0.0.8/initrd.img-0.0.8-x86_64.xz
lrwxrwxrwx 1 root root 61 Dec 26 06:11 kernel -> ../ephemeral/data/k8s/0.0.8/5.3.18-24.37-default-0.0.8-x86_64.kernel
ncn-m003:
total 4
lrwxrwxrwx 1 root root 53 Dec 26 06:11 rootfs -> ../ephemeral/data/k8s/0.0.8/kubernetes-0.0.8-x86_64.squashfs
lrwxrwxrwx 1 root root 47 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/k8s/0.0.8/initrd.img-0.0.8-x86_64.xz
lrwxrwxrwx 1 root root 61 Dec 26 06:11 kernel -> ../ephemeral/data/k8s/0.0.8/5.3.18-24.37-default-0.0.8-x86_64.kernel
ncn-s001:
total 4
lrwxrwxrwx 1 root root 56 Dec 26 06:11 rootfs -> ../ephemeral/data/ceph/0.0.7/storage-ceph-0.0.7-x86_64.squashfs
lrwxrwxrwx 1 root root 48 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/ceph/0.0.7/initrd.img-0.0.7.xz
lrwxrwxrwx 1 root root 62 Dec 26 06:11 kernel -> ../ephemeral/data/ceph/0.0.7/5.3.18-24.37-default-0.0.7.kernel
ncn-s002:
total 4
lrwxrwxrwx 1 root root 56 Dec 26 06:11 rootfs -> ../ephemeral/data/ceph/0.0.7/storage-ceph-0.0.7-x86_64.squashfs
lrwxrwxrwx 1 root root 48 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/ceph/0.0.7/initrd.img-0.0.7.xz
lrwxrwxrwx 1 root root 62 Dec 26 06:11 kernel -> ../ephemeral/data/ceph/0.0.7/5.3.18-24.37-default-0.0.7.kernel
ncn-s003:
total 4
lrwxrwxrwx 1 root root 56 Dec 26 06:11 rootfs -> ../ephemeral/data/ceph/0.0.7/storage-ceph-0.0.7-x86_64.squashfs
lrwxrwxrwx 1 root root 48 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/ceph/0.0.7/initrd.img-0.0.7.xz
lrwxrwxrwx 1 root root 62 Dec 26 06:11 kernel -> ../ephemeral/data/ceph/0.0.7/5.3.18-24.37-default-0.0.7.kernel
ncn-w002:
total 4
lrwxrwxrwx 1 root root 53 Dec 26 06:11 rootfs -> ../ephemeral/data/k8s/0.0.8/kubernetes-0.0.8-x86_64.squashfs
lrwxrwxrwx 1 root root 47 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/k8s/0.0.8/initrd.img-0.0.8-x86_64.xz
lrwxrwxrwx 1 root root 61 Dec 26 06:11 kernel -> ../ephemeral/data/k8s/0.0.8/5.3.18-24.37-default-0.0.8-x86_64.kernel
ncn-w003:
total 4
lrwxrwxrwx 1 root root 53 Dec 26 06:11 rootfs -> ../ephemeral/data/k8s/0.0.8/kubernetes-0.0.8-x86_64.squashfs
lrwxrwxrwx 1 root root 47 Dec 26 06:11 initrd.img.xz -> ../ephemeral/data/k8s/0.0.8/initrd.img-0.0.8-x86_64.xz
lrwxrwxrwx 1 root root 61 Dec 26 06:11 kernel -> ../ephemeral/data/k8s/0.0.8/5.3.18-24.37-default-0.0.8-x86_64.kernel