Restore SLS Postgres without an Existing Backup

This procedure is intended to repopulate SLS in the event when no Postgres backup exists.

Prerequisite

  • (ncn-mw#) Healthy SLS service.

    Verify all three SLS replicas are up and running:

    kubectl -n services get pods -l cluster-name=cray-sls-postgres

    Expected output should look similar to the following:

    NAME                  READY   STATUS    RESTARTS   AGE
cray-sls-postgres-0   3/3     Running   0          18d
cray-sls-postgres-1   3/3     Running   0          18d
cray-sls-postgres-2   3/3     Running   0          18d

Procedure

  1. (ncn-mw#) Retrieve the initial sls_input_file.json that was used to initially install the system with from sls S3 bucket.

    cray artifacts get sls sls_input_file.json sls_input_file.json

  2. (ncn-mw#) Get an API token:

    TOKEN=$(curl -s -S -d grant_type=client_credentials \
            -d client_id=admin-client \
            -d client_secret=`kubectl get secrets admin-client-auth -o jsonpath='{.data.client-secret}' | base64 -d` \
            https://api-gw-service-nmn.local/keycloak/realms/shasta/protocol/openid-connect/token | jq -r '.access_token')

  3. (ncn-mw#) Perform an SLS load state operation to replace the contents of SLS with the data from the sls_input_file.json file.

    Perform the load state operation:

    curl -s -k -H "Authorization: Bearer ${TOKEN}" -X POST -F sls_dump=@sls_input_file.json \
    https://api-gw-service-nmn.local/apis/sls/v1/loadstate

  4. Any previously made customizations made to SLS will need to be applied again.

    This includes any SLS API operations that modified the state of SLS.