Some images require the building and installation of kernel drivers using the Dynamic Kernel Module Support (DKMS) tool. This allows kernel modules to be built for the specific kernel used in the image. The DKMS tool requires access to the running kernel that is not usually allowed by the Image Management Service (IMS). In order to safely allow the expanded access, the IMS configuration must be modified to enable the feature.
Many DKMS build and install scripts require access to the system /proc
, /dev
, and /sys
directories which
allows access to running processes and system services. The IMS jobs run as an administrator user since preparing
images requires root access to work properly. Allowing root access to the running system would allow an
unacceptable security vulnerability to the Kubernetes worker node the job is running on.
To address the security concerns, but also allow the DKMS tool to install kernel modules in image customization, a Kata Virtual Machine (VM) is used. When DKMS is enabled in IMS, the jobs are modified to run inside a Kata VM. The DKMS tool then has enhanced access to the running Kata VM kernel, but is unable to interact directly with the Kubernetes worker the job is running on.
It is required that Kubernetes be configured with Kata. That should be part of the standard NCN worker configuration, so documentation on how to do that is outside the scope of the IMS documentation.
**NOTE: Since the IMS job is running inside a VM, there will be a performance impact on the runtime of the job but this is required to provide a secure environment.
The following steps will enable DKMS operation for all IMS jobs including those controlled by the Configuration Management Service (CFS). It will remain in this configuration until manually reverted back to disabling the DKMS operation.
(ncn-mw#
) Check which Kata runtime class is installed.
kubectl get runtimeclass
Expected output is something like:
NAME HANDLER AGE
kata-clh kata-clh 64d
kata-qemu kata-qemu 64d
Make note of the kata configuration to use for the IMS jobs.
**NOTE: if there are no kata runtime classes returned by the above step, then Kata must be configured on the system. Instructions for that are beyond the scope of the IMS documentation.
(ncn-mw#
) Edit the ims-config
Kubernetes configuration map to enable DKMS.
kubectl -n services edit cm ims-config
Look for the lines:
JOB_ENABLE_DKMS: "False"
JOB_KATA_RUNTIME: kata-qemu
Change the value for JOB_ENABLE_DKMS
to True
. If the Kata runtime class on the system is not
kata-qemu
then change the JOB_KATA_RUNTIME
to the desired configuration:
JOB_ENABLE_DKMS: "True"
JOB_KATA_RUNTIME: kata-qemu
Exit editing the configmap, saving the new values.
(ncn-mw#
) Restart the IMS pod to pick up the new ConfigMap values.
Find the current cray-ims
pod:
kubectl -n services get pods | grep ims
Expected output will look something like:
cray-ims-bc875d949-fffk6 2/2 Running 0 4h29m
ims-post-upgrade-gkf4t 0/2 Completed 0 2d3h
Delete the running pod:
kubectl -n services delete pod cray-ims-bc875d949-fffk6
Then wait until the new pod is in the 2/2 Running
status. New IMS jobs will be created in
Kata VMs with enhanced kernel access.
To revert the settings so the IMS jobs no longer run inside a Kata VM with the enhanced kernel
access change the ims-config
setting back to False
and restart the cray-ims
pod again.
(ncn-mw#
) Edit the ims-config
Kubernetes configuration map to disable DKMS.
kubectl -n services edit cm ims-config
Look for the lines:
JOB_ENABLE_DKMS: "True"
JOB_KATA_RUNTIME: kata-qemu
Change the value for JOB_ENABLE_DKMS
to False
. The variableJOB_KATA_RUNTIME
is not used when
under this scenario so its value does not matter.
JOB_ENABLE_DKMS: "False"
JOB_KATA_RUNTIME: kata-qemu
Exit editing the configmap, saving the new values.
(ncn-mw#
) Restart the IMS pod to pick up the new ConfigMap values.
Find the current cray-ims
pod:
kubectl -n services get pods | grep ims
Expected output will look something like:
cray-ims-bc875d949-64fc1 2/2 Running 0 4h29m
ims-post-upgrade-gkf4t 0/2 Completed 0 2d3h
Delete the running pod:
kubectl -n services delete pod cray-ims-bc875d949-64fc1
Then wait until the new pod is in the 2/2 Running
status. Now new IMS jobs will be started running
directly on the Kubernetes node and without the enhanced kernel access.