Reinstall the Spire Helm chart in the event that spire-postgres databases cannot be restored from a backup.
(ncn-mw#) Uninstall the Spire Helm chart.
helm uninstall -n spire spire
(ncn-mw#) Wait for the pods in the spire namespace to terminate. Once that is done, remove
the spire-data-server PVCs.
kubectl get pvc -n spire | grep spire-data-spire-server | awk '{print $1}' | xargs kubectl delete -n spire pvc
(ncn-mw#) Disable spire-agent on all of the Kubernetes NCNs (all worker nodes and
master nodes) and delete the join data.
for ncn in $(kubectl get nodes -o name | cut -d'/' -f2); do ssh "${ncn}" systemctl stop spire-agent; ssh "${ncn}" rm /var/lib/spire/data/svid.key /var/lib/spire/agent_svid.der /var/lib/spire/bundle.der; done
(ncn-mw#) Follow the Redeploying a Chart procedure with the following specifications:
Name of chart to be redeployed: spire
Base name of manifest: sysmgmt
When reaching the step to update customizations, no edits need to be made to the customizations file.
When reaching the step to validate that the redeploy was successful, perform the following step:
Only follow this step as part of the previously linked chart redeploy procedure.
Verify that all Spire pods have started.
This step may take a few minutes due to a number of pods requiring other pods to be up.
kubectl get pods -n spire
Restart all compute nodes and User Access Nodes (UANs).
Compute nodes and UANs get their join token on boot from the Boot Script Service (BSS). Their old SVID data is no longer valid and a reboot is required in order for them to re-join Spire.