monotux.tech

Using custom MIBs with snmp-exporter

• monitoring, SNMP

I have two PoE capable Brocade ICX switches at home, and I was curious how much wattage I was really using – so it was time to use SNMP again!

Table of Contents

Overview #

I will be using Prometheus and snmp_exporter for this, as I already have the first part setup & working as outlined in a previous entry here.

  • Then I need to locate relevant MIBs,
  • merge them into the snmp_exporter generator configuration (or create a new one) and,
  • finally use this configuration to scrape my target nodes.

One thing I like with this setup is expressed in the snmp_exporter README:

While SNMP uses a hierarchical data structure and Prometheus uses an n-dimensional matrix, the two systems map perfectly, and without the need to walk through data by hand. snmp_exporter maps the data for you.

MIBs #

Management information base is essentially a database containing information on different network devices. Different vendors/organizations has been allocated different prefixes in it so one typically needs to locate the vendor MIBs to translate the number representation returned from the (network) device into human readable strings & descriptions of the metrics.

This is the tricky part. :-)

I found the site mibs.observium.org which has a pretty comprehensive collection. After googling PoE & Brocade I eventually figured out that I was interested in .1.3.6.1.4.1.1991.1.1.2.14.2 which contains the following information:

  • snAgentPoePort (.1.3.6.1.4.1.1991.1.1.2.14.2)
    • snAgentPoePortTable (.1.3.6.1.4.1.1991.1.1.2.14.2.2)
    • snAgentPoePortEntry (.1.3.6.1.4.1.1991.1.1.2.14.2.2.1)
    • snAgentPoePortNumber (.1.3.6.1.4.1.1991.1.1.2.14.2.2.1.1)
    • snAgentPoePortControl (.1.3.6.1.4.1.1991.1.1.2.14.2.2.1.2)
    • snAgentPoePortWattage (.1.3.6.1.4.1.1991.1.1.2.14.2.2.1.3)
    • snAgentPoePortClass (.1.3.6.1.4.1.1991.1.1.2.14.2.2.1.4)
    • snAgentPoePortPriority (.1.3.6.1.4.1.1991.1.1.2.14.2.2.1.5)
    • snAgentPoePortConsumed (.1.3.6.1.4.1.1991.1.1.2.14.2.2.1.6)
    • snAgentPoePortType (.1.3.6.1.4.1.1991.1.1.2.14.2.2.1.7)

That, in turn, is located in FOUNDRY-POE-MIB which depends on FOUNDRY-SN-ROOT-MIB, at least according to Observium. I eventually found both of these MIBs online1, and downloaded them.

snmp_exporter generator #

With MIBs in hand, we need to generate a new snmp_exporter generator.yaml configuration. This will take in our MIBs and translate these into snmp.yaml, which snmp_exporter will use to expose our SNMP data into Prometheus compatible metrics, ready for scraping.

While this sounds complicated it’s actually less terrible than it seems. With net-snmp one needs to collect mibs & extend net-snmp configuration to include the new mibs location for each system you want to use SNMP with. With snmp-exporter we get Prometheus style metrics which we scrape using our normal tooling, and we only need to work with mibs when creating snmp.yaml.

Setting up snmp_exporter generator #

I use NixOSbtw so I just spin up a new shell with prometheus-snmp-exporter:

nix-shell -p prometheus-snmp-exporter go unzip

If you use Debian, prometheus-snmp-exporter is packaged for all modern releases so it’s just a short apt away:

apt install prometheus-snmp-exporter

We will, however, checkout the git repository for snmp_exporter anyways so you can compile it from source as well.

git clone https://github.com/prometheus/snmp_exporter.git

I will checkout the same version of the repository as my snmp_exporter is built to:

snmp_exporter --version
git checkout v0.26.0

Download a lot of MIBs automatically:

cd snmp_exporter/generator/
make mibs

Download my MIBs and store them in snmp_exporter/generator/mibs:

curl https://raw.githubusercontent.com/librenms/librenms/refs/heads/master/mibs/brocade/FOUNDRY-SN-ROOT-MIB > mibs/FOUNDRY-SN-ROOT-MIB
curl https://mibbrowser.online/mibs/FOUNDRY-POE-MIB.mib > mibs/FOUNDRY-POE-MIB

Adding our MIBs to the configuration #

Once you open generator.yml you will notice it consists of two parts – authentication details and modules, and that the latter is 40k+ lines!

If you are ambitious enough, you can probably remove all unnecessary modules, add your new ones, and call it a day. Let’s try that!

---
auths:
  public_v1:
	version: 1
  public_v2:
	version: 2

modules:
  # SNMPv2-MIB for things like sysDescr, sysUpTime, etc.
  system:
	walk:
	  - "SNMPv2-MIB::system"
	lookups:
	  - source_indexes: [sysORIndex]
		lookup: "SNMPv2-MIB::sysORDescr"
  # Default IF-MIB interfaces table with ifIndex.
  if_mib:
	walk:
	  - "IF-MIB::interfaces"
	  - "IF-MIB::ifXTable"
	lookups:
	  - source_indexes: [ifIndex]
		lookup: "IF-MIB::ifAlias"
	  - source_indexes: [ifIndex]
		# Disambiguate from PaloAlto PAN-COMMON-MIB::ifDescr.
		lookup: "IF-MIB::ifDescr"
	  - source_indexes: [ifIndex]
		# Disambiguate from Netscaler NS-ROOT-MIB::ifName.
		lookup: "IF-MIB::ifName"
	overrides:
	  ifAlias:
		ignore: true # Lookup metric
	  ifDescr:
		ignore: true # Lookup metric
	  ifName:
		ignore: true # Lookup metric
	  ifType:
		type: EnumAsInfo
  # Our new bits
  snagentpoe:
	walk:
	  - "snAgentPoePort"

(the walk keyword means that we will scrape the entire tree structure below our starting point)

Run it:

generator generate -m mibs/ -g generator.yml -o /tmp/snmp.yaml

Inspect it’s output in /tmp/snmp.yaml, it should look something like this:

# WARNING: This file was auto-generated using snmp_exporter generator, manual changes will be lost.
auths:
  public_v1:
	community: public
	security_level: noAuthNoPriv
	auth_protocol: MD5
	priv_protocol: DES
	version: 1
  public_v2:
	community: public
	security_level: noAuthNoPriv
	auth_protocol: MD5
	priv_protocol: DES
	version: 2
modules:
  if_mib:
	walk:
	- 1.3.6.1.2.1.2
	- 1.3.6.1.2.1.31.1.1
	metrics:
	- name: ifNumber
	  oid: 1.3.6.1.2.1.2.1
	  type: gauge
	  help: The number of network interfaces (regardless of their current state) present
		on this system. - 1.3.6.1.2.1.2.1
	- name: ifIndex
	  oid: 1.3.6.1.2.1.2.2.1.1
	  type: gauge
	  help: A unique value, greater than zero, for each interface - 1.3.6.1.2.1.2.2.1.1
	  indexes:
	  - labelname: ifIndex
		type: gauge

#
# A lot of lines
#

  snagentpoe:
	walk:
	- 1.3.6.1.4.1.1991.1.1.2.14.2
	metrics:
	- name: snAgentPoePortNumber
	  oid: 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.1
	  type: gauge
	  help: The port number in ifIndex value. - 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.1
	  indexes:
	  - labelname: snAgentPoePortNumber
		type: gauge
	- name: snAgentPoePortControl
	  oid: 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.2
	  type: gauge
	  help: Control inline power on/off to a port - 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.2
	  indexes:
	  - labelname: snAgentPoePortNumber
		type: gauge
	  enum_values:
		1: other
		2: disable
		3: enable
		4: enableLegacyDevice
	- name: snAgentPoePortWattage
	  oid: 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.3
	  type: gauge
	  help: Adjust the inline power wattage - 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.3
	  indexes:
	  - labelname: snAgentPoePortNumber
		type: gauge
	- name: snAgentPoePortClass
	  oid: 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.4
	  type: gauge
	  help: Adjust the inline power class - 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.4
	  indexes:
	  - labelname: snAgentPoePortNumber
		type: gauge
	- name: snAgentPoePortPriority
	  oid: 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.5
	  type: gauge
	  help: Inline power allocation priority for the power device 0- Not a POE port,
		1- Critical, 2- High, 3- Low, 4- Medium, 5- other. - 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.5
	  indexes:
	  - labelname: snAgentPoePortNumber
		type: gauge
	  enum_values:
		0: invalid
		1: critical
		2: high
		3: low
		4: medium
		5: other
	- name: snAgentPoePortConsumed
	  oid: 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.6
	  type: gauge
	  help: Inline power consumed by the port - 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.6
	  indexes:
	  - labelname: snAgentPoePortNumber
		type: gauge
	- name: snAgentPoePortType
	  oid: 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.7
	  type: DisplayString
	  help: Inline Power device type 802.3af, 802.3at or Legacy device. - 1.3.6.1.4.1.1991.1.1.2.14.2.2.1.7
	  indexes:
	  - labelname: snAgentPoePortNumber
		type: gauge

#
# And some more lines...
#

  system:
	walk:
	- 1.3.6.1.2.1.1

Nice! We can copy this file somewhere and use it in as our configuration.

Configuration #

First of all, setting up snmp_exporter is pretty easy – you basically just point it at the configuration file we just generated! Then we will setup our prometheus to scrape snmp_exporter, which in turn will trigger a SNMP walk of our devices.

snmp_exporter #

On my NixOSbtw machine I just include this snippet somewhere:

{
  services.prometheus.exporters.snmp = {
	enable = true;
	listenAddress = "192.0.8.1";
	configurationPath = /etc/nixos/services/snmp.yml;
  };
}

For Debian I’ve used prometheus-community/ansible with great success before.

The result should be a systemd unit that looks something like this, but with the correct paths for ExecStart:

[Unit]
Description=Prometheus SNMP Exporter Service
After=network.target

[Service]
Type=simple
User=prometheus
ExecStart=/usr/bin/snmp_exporter --config.file="/etc/prometheus/snmp.yml"

[Install]
WantedBy=multi-user.target

Prometheus #

We need add our snmp_exporter as a normal scraping target in Prometheus, which might look something like this:

  - job_name: snmp
	metrics_path: /snmp
	params:
	  module: [if_mib]
	static_configs:
	  - targets:
		- 127.0.0.1
	relabel_configs:
	  - source_labels: [__address__]
		target_label: __param_target
	  - source_labels: [__param_target]
		target_label: instance
	  - target_label: __address__
		replacement: 127.0.0.1:9116

Under params.module you can see a list containing if_mib. These are the top level keys under modules in the snmp.yaml we generated. For this example we just have three top level keys:

# yq '.modules|keys[]' /tmp/snmp.yml
"if_mib"
"snagentpoe"
"system"

It is also possible to have multiple SNMP scraping configurations for different devices, like if you have one Juniper and one Brocade switch that you want to scrape, but they have different PoE metrics:

- job_name: snmp
  metrics_path: /snmp
  params:
	module:
	- if_mib
  static_configs:
	- targets:
	  - 192.0.8.2
	  - 192.0.8.3
  relabel_configs:
  - source_labels: [__address__]
	target_label: __param_target
  - source_labels: [__param_target]
	target_label: instance
  - target_label: __address__
	replacement: 192.0.8.1:9116

- job_name: snmp_power_brocade
  metrics_path: /snmp
  params:
	module:
	- snagentpoe
  relabel_configs:
  - source_labels:
	- __address__
	target_label: __param_target
  - source_labels:
	- __param_target
	target_label: instance
  - replacement: 192.0.8.1:9116
	target_label: __address__
  static_configs:
  - labels:
	  job: brocade
	targets:
	- 192.0.8.2

- job_name: snmp_power_juniper
  metrics_path: /snmp
  params:
	module:
	- powerMIB
  relabel_configs:
  - source_labels:
	- __address__
	target_label: __param_target
  - source_labels:
	- __param_target
	target_label: instance
  - replacement: 192.0.8.1:9116
	target_label: __address__
  static_configs:
  - labels:
	  job: juniper
	targets:
	- 192.0.8.3

Conclusion #

snmp_exporter is pretty nice to use and works great with Prometheus in general.

  1. FOUNDRY-SN-ROOT here and FOUNDRY-POE-MIB here ↩︎