Import an External Image to IMS

The Image Management Service (IMS) is typically used to build images from IMS recipes and customize Images that are already known to IMS. However, it is sometimes the case that an image is built using a mechanism other than by IMS and needs to be added to IMS. In these cases, the following procedure can be used to add this external image to IMS and upload the image’s artifacts to the Simple Storage Service (S3).

An automated tool is available to help the specific case of starting with image artifacts for a Non-Compute Node (NCN) that are not yet in S3 or IMS. For more information, see Import an NCN Image to IMS.

Prerequisites

  • CSM is fully installed, configured, and healthy.
    • The Image Management Service (IMS) is healthy.
    • The Simple Storage Service (S3) is healthy.
    • The NCN Certificate Authority (CA) public key has been properly installed into the CA cache for this system.
    • These may be validated by performing the following health checks:
  • The Cray CLI is configured.
  • Image artifact files are available.
    • An image root file is required.
    • Optionally, additional image artifacts may be specified including a kernel, initrd, and kernel parameters file.

Limitations

  • The commands in this procedure must be run as the root user.
  • Images in the .txz compressed format need to be converted to SquashFS in order to use IMS image customization.

Procedure

This procedure may be run on any master or worker NCN.

The procedure on this page uses example commands that assume the image has an associated kernel and initrd artifact. This is the case, for example, for NCN boot images. If the actual set of image artifacts differs from this, then be sure to modify the commands accordingly.

1. Ensure supported format

  1. Ensure that the image root is in a supported format.

    IMS requires that an image’s root filesystem is in SquashFS format. Select one of the following options based on the current state of the image root being used:

    • If the image being added meets the above requirements, then skip the rest of this section and proceed to the next step.
    • If the image being added is in tgz format, then refer to Convert TGZ Archives to SquashFS Images.
    • If the image root is in a format other than tgz or SquashFS, then convert the image root to tgz/SquashFS before continuing.

2. Set helper variables

Set variables for all of the image artifact files, if needed. For example, IMS_ROOTFS_FILENAME, IMS_INITRD_FILENAME, and IMS_KERNEL_FILENAME.

  1. (ncn-mw#) Set the IMS_ROOTFS_FILENAME variable to the file name of the SquashFS image root file to be uploaded.

    For example:

    IMS_ROOTFS_FILENAME=sles_15_image.squashfs

  2. (ncn-mw#) Set the IMS_INITRD_FILENAME variable to the file name of the initrd file to be uploaded.

    Skip this if no initrd file is associated with this image.

    For example:

    IMS_INITRD_FILENAME=initrd

  3. (ncn-mw#) Set the IMS_KERNEL_FILENAME variable to the file name of the kernel file to be uploaded.

    Skip this if no kernel file is associated with this image.

    For example:

    IMS_KERNEL_FILENAME=vmlinuz

3. Record artifact checksums

  1. Navigate to the directory containing the artifact files.

  2. (ncn-mw#) Verify that all image artifacts exist in the current working directory.

    If necessary, modify the following command to reflect the actual set of artifacts included in the image.

    ls -al "${IMS_ROOTFS_FILENAME}" "${IMS_INITRD_FILENAME}" "${IMS_KERNEL_FILENAME}"

  3. (ncn-mw#) Record the checksums of all of the artifacts.

    1. Record the SquashFS image root checksum in the IMS_ROOTFS_MD5SUM variable.

      IMS_ROOTFS_MD5SUM=$(md5sum "${IMS_ROOTFS_FILENAME}" | awk '{ print $1 }')
echo "${IMS_ROOTFS_MD5SUM}"

    2. Record the initrd checksum in the IMS_INITRD_MD5SUM variable.

      Skip this if no initrd file is associated with this image.

      IMS_INITRD_MD5SUM=$(md5sum "${IMS_INITRD_FILENAME}" | awk '{ print $1 }')
echo "${IMS_INITRD_MD5SUM}"

    3. Record the kernel checksum in the IMS_KERNEL_MD5SUM variable.

      Skip this if no kernel file is associated with this image.

      IMS_KERNEL_MD5SUM=$(md5sum "${IMS_KERNEL_FILENAME}" | awk '{ print $1 }')
echo "${IMS_KERNEL_MD5SUM}"

4. Create image record in IMS

  1. (ncn-mw#) Create a new IMS image record for the image.

    cray ims images create --name "${IMS_ROOTFS_FILENAME}" --arch <architecture> --format toml

    Example output:

    created = "2018-12-04T17:25:52.482514+00:00"
id = "4e78488d-4d92-4675-9d83-97adfc17cb19"
name = "sles_15_image.squashfs"
arch = "x86_64"

  2. (ncn-mw#) Create an environment variable for the ID of the new IMS image record.

    Set the IMS_IMAGE_ID variable, using the id field value from the returned data in the previous step.

    IMS_IMAGE_ID=4e78488d-4d92-4675-9d83-97adfc17cb19

5. Upload artifacts to S3

  1. Navigate to the directory containing the artifact files.

  2. (ncn-mw#) Verify that all image artifacts exist in the current working directory.

    If necessary, modify the following command to reflect the actual set of artifacts included in the image.

    ls -al "${IMS_ROOTFS_FILENAME}" "${IMS_INITRD_FILENAME}" "${IMS_KERNEL_FILENAME}"

  3. (ncn-mw#) Upload the artifacts to S3.

    1. Upload the SquashFS image root to S3.

      cray artifacts create boot-images "${IMS_IMAGE_ID}/${IMS_ROOTFS_FILENAME}" "${IMS_ROOTFS_FILENAME}"

    2. Upload the kernel to S3.

      cray artifacts create boot-images "${IMS_IMAGE_ID}/${IMS_KERNEL_FILENAME}" "${IMS_KERNEL_FILENAME}"

    3. Upload the initrd to S3.

      cray artifacts create boot-images "${IMS_IMAGE_ID}/${IMS_INITRD_FILENAME}" "${IMS_INITRD_FILENAME}"

6. Create, upload, and register image manifest

HPE Cray uses a manifest file that associates multiple related boot artifacts (kernel, initrd, image root, etc.) into an image description that is used by IMS and other services to boot nodes. Artifacts listed within the manifest are identified by a type value:

  • application/vnd.cray.image.rootfs.squashfs
  • application/vnd.cray.image.initrd
  • application/vnd.cray.image.kernel
  • application/vnd.cray.image.parameters.boot

  1. (ncn-mw#) Collect etag values for uploaded files.

    ROOTFS_ETAG=$( cray artifacts describe boot-images ${IMS_IMAGE_ID}/${IMS_ROOTFS_FILENAME} --format json | jq -r .artifact.ETag  | tr -d '"' )
KERNEL_ETAG=$( cray artifacts describe boot-images ${IMS_IMAGE_ID}/${IMS_KERNEL_FILENAME} --format json | jq -r .artifact.ETag  | tr -d '"' )
INITRD_ETAG=$( cray artifacts describe boot-images ${IMS_IMAGE_ID}/${IMS_INITRD_FILENAME} --format json | jq -r .artifact.ETag  | tr -d '"' )

  2. (ncn-mw#) Generate an image manifest file.

    If necessary, modify the following example to reflect the actual set of artifacts included in the image.

    Note that the following command makes use of several variables that have been set during this procedure. The command must be run from the Bash shell in order for them to be properly evaluated.

    cat <<EOF> manifest.json
{
  "created": "`date '+%Y-%m-%d %H:%M:%S'`",
  "version": "1.0",
  "artifacts": [
    {
      "link": {
          "etag": "${ROOTFS_ETAG}",
          "path": "s3://boot-images/${IMS_IMAGE_ID}/${IMS_ROOTFS_FILENAME}",
          "type": "s3"
      },
      "md5": "${IMS_ROOTFS_MD5SUM}",
      "type": "application/vnd.cray.image.rootfs.squashfs"
    },
    {
      "link": {
          "etag": "${KERNEL_ETAG}",
          "path": "s3://boot-images/${IMS_IMAGE_ID}/${IMS_KERNEL_FILENAME}",
          "type": "s3"
      },
      "md5": "${IMS_KERNEL_MD5SUM}",
      "type": "application/vnd.cray.image.kernel"
    },
    {
      "link": {
          "etag": "${INITRD_ETAG}",
          "path": "s3://boot-images/${IMS_IMAGE_ID}/${IMS_INITRD_FILENAME}",
          "type": "s3"
      },
      "md5": "${IMS_INITRD_MD5SUM}",
      "type": "application/vnd.cray.image.initrd"
    }
  ]
}
EOF

  3. (ncn-mw#) Upload the manifest to S3.

    cray artifacts create boot-images "${IMS_IMAGE_ID}/manifest.json" manifest.json

  4. (ncn-mw#) Collect the etag for the manifest file.

    MANIFEST_ETAG=$( cray artifacts describe boot-images ${IMS_IMAGE_ID}/manifest.json --format json | jq -r .artifact.ETag  | tr -d '"' )

  5. (ncn-mw#) Update the IMS image record with the image manifest information.

    cray ims images update "${IMS_IMAGE_ID}" \
    --link-type s3 \
    --link-path "s3://boot-images/${IMS_IMAGE_ID}/manifest.json" \
    --link-etag "${MANIFEST_ETAG}" \
    --format toml

    Example output:

    created = "2018-12-04T17:25:52.482514+00:00"
id = "4e78488d-4d92-4675-9d83-97adfc17cb19"
name = "sles_15_image.squashfs"
arch = "x86_64"

[link]
type = "s3"
path = "s3://boot-images/4e78488d-4d92-4675-9d83-97adfc17cb19/manifest.json"
etag = "627264cd4ab4d231a0b2bf42aabb4156"