A simple python based Spectrum Scale/GPFS filesystem statistics and quota exporter.

Data collected

The exporter collects:

• GPFS IO statistics
• GPFS pool sizes and limits
• GPFS fileset, user and group quotas for all local GPFS filesystems.

It can be ran on any member of the cluster.

GPFS IO metrics

All of the following metrics will have fs label containing the name of the filesystem, e.g. fs="bigdata.storage".

Metric name	Type	Description
gpfs_bytes_read	counter	GPFS bytes read
gpfs_bytes_write	counter	GPFS bytes written
gpfs_requests_open	counter	GPFS open call requests including create
gpfs_requests_close	counter	GPFS close call requests
gpfs_requests_read	counter	GPFS number of read requests
gpfs_requests_write	counter	GPFS number of write requests
gpfs_requests_readdir	counter	GPFS number of readdit requests
gpfs_inode_updates	counter	GPFS number of inode updates to disk

GPFS Pool metrics

In addition to the fs label these metrics also provide these labels:

• pool_name, e.g. "system"
• pool_id, e.g. "0",
• pool_type, e.g. "data,meta"
• block_size, e.g. 4 MB"

Metric name	Type	Description
gpfs_pool_data_size_bytes	gauge	GPFS Pool Data Total Size
gpfs_pool_data_free_bytes	gauge	GPFS Pool Data Free Size
gpfs_pool_meta_size_bytes	gauge	GPFS Pool Meta Total Size
gpfs_pool_meta_free_bytes	gauge	GPFS Pool Meta Free Size

GPFS Fileset/User/Group Quota Metrics

In addition to the fs these metrics can have labels:

• quota_type, e.g. USR or GRP or FILESET
• uid, e.g. "0"
• gid, e.g. "0"
• fileset_id, e.g. "0" (only for FILESET quota types)
• fid="", e.g. "" or a number for the fileset
• filesetname, e.g. "root"
• quota, e.g. "on"
• def_quota, e.g. "off"
• remarks, e.g. "i"

Metric name	Type	Description
gpfs_quota_block_usage_bytes	gauge	GPFS Block Quota Usage
gpfs_quota_block_limit_soft_bytes	gauge	GPFS Block Quota Soft Limit
gpfs_quota_block_limit_hard_bytes	gauge	GPFS Block Quota Hard Limit
gpfs_quota_block_usage_in_doubt_bytes	gauge	GPFS Block Quota Usage In Doubt
gpfs_quota_block_grace_in_seconds	gauge	GPFS Block Quota Grace In Seconds with 0=expired,1=ok or seconds+1
gpfs_quota_files_usage	gauge	GPFS Number Of Files Quota Usage
gpfs_quota_files_limit_soft	gauge	GPFS Number Of Files Quota Soft Limit
gpfs_quota_files_limit_hard	gauge	GPFS Number Of Files Quota Hard Limit
gpfs_quota_files_usage_in_doubt	gauge	GPFS Number Of Files Quota Usage In Doubt
gpfs_quota_files_grace_in_seconds	gauge	GPFS Number Of Files Grace In Seconds with 0=expired, 1=ok or seconds+1

Running

It is designed to run under systemd via gpfs-exporter socket and unit service, found in systemd/ directory in this repository.

For debugging purposes run it directly. It will output on the stdout and exit immediately.

Basic Prometheus Configuration

For example, with cluster called speedy:

---
global:
  scrape_interval: 15s
  evaluation_interval: 15s
scrape_configs:
- job_name: prometheus
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: "/prometheus/metrics"
  static_configs:
  - targets:
    - localhost:9090
    labels:
      alias: Prometheus
- job_name: Speedy GPFS
  static_configs:
  - targets:
    - speedy-member:9001
    labels:
      cluster: speedy
      service: gpfs

Grafana dashboard

You can find the dashboard used at Princeton University in the grafana subdirectory. It has two static variables that are used for easy filtering. One with the list of compute clusters (we add cluster label at collection time) and the other with gpfs filesystem names.