Create a Session Template to Boot Compute Nodes with CPS

When compute nodes are booted, the Content Projection Service (CPS) and Data Virtualization Service (DVS) project the root file system (rootfs) over the network to the compute nodes by default.

Another option when compute nodes are booted is to download their rootfs into RAM.

This page covers the appropriate contents for a BOS session template in order to use CPS and DVS.

Boot set S3 parameters

The session template boot set contains several Simple Storage Service (S3) parameters. These are listed below, along with the appropriate values to use.

  • type: Set to s3
  • path: Set to s3://<BUCKET_NAME>/<KEY_NAME>
  • etag: Set to <etag\>

Boot set rootfs_provider parameters

The Content Projection Service (CPS) is an optional provider for rootfs on compute nodes.

The rootfs_provider_passthrough boot set parameter is customized according to the following format:

rootfs_provider_passthrough=<transport>:<api_gateway>:<timeout>:<interface>[,<interface>[,<interface>]...]:<ramroot>

The following values need to be set in the boot set of the session template in order to make CPS the rootfs provider:

  • "rootfs_provider": Set to "cpss3"
  • "rootfs_provider_passthrough": Set to "dvs:api-gw-service-nmn.local:300:eth0"

The variables used in this parameter represent the following:

<transport>

File system network transport. For example, nfs or dvs.

Can be left as an empty string to use the default value dvs.

<api_gateway>

Name or address of the Kubernetes API gateway.

Can be left as an empty string to use the default value api-gw-service-nmn.local.

<timeout>

The timeout, in seconds, for attempting to mount the netroot via CPS.

Can be left as an empty string to use the default value of 300 seconds.

<etag>

Lists the syntax in use. BOS fills in the S3 path and etag values, so the user does not need to fill in any data.

<interface>[,<interface>][,<interface>]...

A comma-separated list of interfaces to support. A minimum of one interface must be specified.

The first interface specified must exist on the node or the module will exit with an error. Any other specified interface that is not found on the node will be ignored. The module will wait until all specified and existing interfaces are up before proceeding with boot. The first interface specified will be passed to the CPS mount command to identify the interface to be used for mounting.

<ramroot>

Indicates that the specified S3 path should be copied to RAM (tmpfs) and mounted locally instead of persisting as a remote file system mount.

Can be left empty. Any string except "0" is interpreted as true.

Example rootfs_provider_passthrough

rootfs_provider_passthrough=dvs:api-gw-service-nmn.local:300:eth0

root= kernel parameter

BOS will construct the root= kernel parameter, which will be used by the node when it boots, based on the rootfs_provider and rootfs_provider_passthrough values.

For CPS, BOS supplies a protocol craycps-s3, the S3 path to the rootfs, and the etag value (if it exists). The rest of the parameters are supplied from the rootfs_provider_passthrough values as specified above.

BOS will construct it in the following format:

root=craycps-s3:s3-path:<etag>:<transport>:<api_gateway>:<timeout>:interface[,<interface>[,<interface>]...]:<ramroot>

Example session template input file

The following is an example of an input file to use with the Cray CLI:

{
  "enable_cfs": true,
  "description": "Template for booting compute nodes, generated by the installation",
  "boot_sets": {
    "computes": {
      "rootfs_provider": "cpss3",
      "kernel_parameters": "console=ttyS0,115200 bad_page=panic crashkernel=360M hugepagelist=2m-2g intel_iommu=off intel_pstate=disable iommu=pt ip=dhcp numa_interleave_omit=headless numa_zonelist_order=node oops=panic pageblock_order=14 pcie_ports=native printk.synchronous=y rd.neednet=1 rd.retry=10 rd.shell k8s_gw=api-gw-service-nmn.local quiet turbo_boost_limit=999",
      "node_roles_groups": [
        "Compute"
      ],
      "etag": "b0ace28163302e18b68cf04dd64f2e01",
      "path": "s3://boot-images/ef97d3c4-6f10-4d58-b4aa-7b70fcaf41ba/manifest.json",
      "rootfs_provider_passthrough": "dvs:api-gw-service-nmn.local:300:eth0",
      "type": "s3"
    }
  },
  "cfs": {
    "configuration": "compute-configuration"
  }
}

Refer to Manage a Session Template for more information about creating a session template.

Creating a BOS session using the new template

(ncn-mw#) The new CPS-based session template can be used when creating a BOS session. The following is an example of creating a reboot session using the CLI:

cray bos v2 sessions create --template-name cps_rootfs_template --operation Reboot