Troubleshoot CFS Session Failing to Complete

Troubleshoot issues where Configuration Framework Service (CFS) sessions/pods fail and Ansible hangs. These issues can be resolved by modifying Ansible to produce less output.

Prerequisites

A CFS session or pod is failing to complete, and the Ansible logs are not showing progress or completion.

The following is an example of the error causing Ansible to hang:

PLAY [Compute] *****************************************************************
META: ran handlers
META: ran handlers
META: ran handlers

PLAY [Compute] *****************************************************************
Using module file /usr/lib/python3.6/site-packages/ansible/modules/system/setup.py
Pipelining is enabled.
<x5000c3s3b0n1> ESTABLISH SSH CONNECTION FOR USER: root
<x5000c3s3b0n1> SSH: EXEC ssh -vvv -o ServerAliveInterval=30 -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/secret/key"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/f6f378183d x5000c3s3b0n1 '/bin/sh -c '"'"'/usr/bin/python3 && sleep 0'"'"''

Reduce Ansible output

The steps in this procedure are independent from each other and are used to troubleshoot different underlying problems that both present a similar symptom.

Any of the following steps can be taken to help reduce the output generated by Ansible:

  • Reduce the verbosity when running Ansible commands.

    If the session was created with a higher value of --ansible-verbosity (three or higher), Ansible can generate a lot of output that can cause the pod to hang. Reducing the verbosity by one or more may resolve this issue.

  • Update the Ansible configuration to produce less output.

    See Enable Ansible Profiling for an example of modifying the configuration.

  • Adjust the use of flags used when running Ansible commands.

    The display_ok_hosts and display_skipped_hosts are examples of settings that can be disabled to reduce output. See the Ansible documentation for more information on what flags can be used.