Cray System Management Documentation > Cray System Management (CSM) Administration Guide > network > Management Network User Guide > Prometheus SNMP Exporter

Prometheus SNMP Exporter

The Prometheus SNMP Exporter is deployed by the cray-sysmgmt-health chart to the sysmgmt-health namespace as part of the Cray System Management (CSM) release.

Adding SNMP credentials to the system

Both the Prometheus SNMP Exporter and River Endpoint Discovery Service (REDS) hardware discovery use SNMP credentials stored in Vault. These credentials are read from Vault and pushed into customizations.yaml as a sealed secret. For the SNMP Exporter and REDS hardware discovery to work, three things need to have been done by an administrator:

The SNMP credentials need to have been pushed into the correct path in Vault.
- Updating the SNMP credentials for existing switches
- Updating the SNMP credentials for switches added in the future
The SNMP credentials need to have been encrypted as sealed secrets and written to customizations.yaml (this is covered in the documentation linked in the previous bullet).
The SNMP credentials need to be added to the running configuration on every management network switch.
- Information about managing Dell switches including SNMPv2 community and SNMPv3 user settings
- Information about managing Mellanox switches including SNMPv2 community and SNMPv3 settings
- Information about managing Aruba switches including SNMPv2 community and SNMPv3 settings

Note: While the CSM Automatic Network Utility (CANU) will typically not overwrite SNMP settings that are manually applied to the management switches, there are certain cases where SNMP configuration can be over-written or lost (such as when resetting and reconfiguring a switch from factory defaults). To persist the SNMP settings, see CANU Custom Configuration. CANU custom configuration files are used to persist site management network configurations that are intended to take precedence over configurations generated by CANU.

If the Prometheus SNMP Exporter or REDS hardware discovery emit errors related to SNMP authentication, then an administrator should:

Validate that the correct SNMP credential key/value pairs exist in Vault (see information above).
Validate that the SNMP credential sealed secrets in customizations.yaml match Vault and the credentials stored on the management switches.
Validate that the SNMP credentials configured on the management switches match the credentials stored in Vault (see information above).

Configuration

In order to provide data to the Grafana SNMP dashboards, the SNMP Exporter must be configured with a list of management network switches to scrape metrics from.

NOTE All variables used within this page depend on the /etc/environment setup done in Pre-installation.

(pit#) Obtain the list of switches to use as targets using CANU.

canu init --sls-file ${PITDATA}/prep/${SYSTEM_NAME}/sls_input_file.json --out -

Expected output looks similar to the following:

10.252.0.2
10.252.0.3
10.252.0.4
10.252.0.5
4 IP addresses saved to <stdout>

(pit#) Update customizations.yaml with the list of switches.

yq write -s - -i ${PITDATA}/prep/site-init/customizations.yaml <<EOF
- command: update
  path: spec.kubernetes.services.cray-sysmgmt-health.prometheus-snmp-exporter
  value:
          serviceMonitor:
            enabled: true
            params:
            - name: snmp1
              target: 10.252.0.2
            - name: snmp2
              target: 10.252.0.3
            - name: snmp3
              target: 10.252.0.4
EOF

(pit#) Review the SNMP Exporter configuration.

yq r ${PITDATA}/prep/site-init/customizations.yaml spec.kubernetes.services.cray-sysmgmt-health.prometheus-snmp-exporter

The expected output looks similar to:

serviceMonitor:
  enabled: true
  params:
  - name: snmp1
    target: 10.252.0.2
  - name: snmp2
    target: 10.252.0.3
  - name: snmp3
    target: 10.252.0.4

The most common configuration parameters are specified in the following table. They must be set in the customizations.yaml file under the spec.kubernetes.services.cray-sysmgmt-health.prometheus-snmp-exporter service definition.

Customization	Default	Description
`serviceMonitor.enabled`	`true`	Enables `serviceMonitor` for SNMP Exporter (default chart value is `true`)
`params.enabled`	`true`	Sets the SNMP Exporter `params` change to `true` (default chart value is `false`)
`params.conf.module`	`if_mib`	SNMP Exporter to select which module (default chart value is `if_mib`)
`params.conf.target`	`10.252.0.2`	Add list of switch targets to SNMP Exporter to monitor

For a complete set of available parameters, consult the values.yaml file for the cray-sysmgmt-health chart.