Due to the way the resolver code works in certain versions of Alpine Linux, it may be necessary to reconfigure the spire-postgres-pooler to use the fully qualified domain name of the database in order to prevent DNS lookup errors.
The spire-server pods are logging query_wait_timeout errors.
time="2022-11-15T09:39:38Z" level=error msg="Fatal run error" error="datastore-sql: pq: query_wait_timeout"
time="2022-11-15T09:39:38Z" level=error msg="Server crashed" error="datastore-sql: pq: query_wait_timeout"
The spire-postgres-pooler pods are logging DNS lookup failure errors.
2022-11-15 09:38:40.290 UTC [1] WARNING DNS lookup failed: spire-postgres: result=0
2022-11-15 09:38:56.211 UTC [1] WARNING DNS lookup failed: spire-postgres: result=0
2022-11-15 09:39:11.881 UTC [1] WARNING DNS lookup failed: spire-postgres: result=0
2022-11-15 09:39:27.879 UTC [1] WARNING DNS lookup failed: spire-postgres: result=0
2022-11-15 09:39:38.541 UTC [1] WARNING C-0x55729bbc56c0: spire/(nouser)@127.0.0.6:56151 pooler error: query_wait_timeout
(ncn-mw#) Edit the spire-postgres-pooler deployment.
Command:
kubectl -n spire edit deployment spire-postgres-pooler
Update the PGHOST environment variable to use the fully qualified domain name.
An example of the deployment before being edited:
containers:
- env:
- name: PGHOST
value: spire-postgres
Change PGHOST to:
containers:
- env:
- name: PGHOST
value: spire-postgres.spire.svc.cluster.local
The spire-postgres-pooler pods will automatically restart to pick up the new value.
IMPORTANT: This change will need to be reapplied if the spire Helm chart is re-installed.
This will be resolved in a future CSM release when the PostgreSQL operator is upgraded to a newer version.