Skip to content

Commit

Permalink
Merge pull request #23 from genegr/main
Browse files Browse the repository at this point in the history
New release to add the enhancement requested in issue #19
  • Loading branch information
genegr authored May 1, 2023
2 parents c2c71f2 + bda1c10 commit e4611f5
Show file tree
Hide file tree
Showing 76 changed files with 46,408 additions and 72 deletions.
3 changes: 2 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@ GOCMD=go
GOTEST=$(GOCMD) test
GOVET=$(GOCMD) vet
BINARY_NAME=pure-fb-om-exporter
VERSION?=1.0.2
MODULE_NAME=purestorage/fb-openmetrics-exporter
VERSION?=1.0.3
SERVICE_PORT?=9491
DOCKER_REGISTRY?= quay.io/purestorage/
EXPORT_RESULT?=false # for CI please set EXPORT_RESULT to true
Expand Down
55 changes: 49 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,47 @@ The provided dockerfile can be used to generate a docker image of the exporter.

docker build -t pure-fb-ome:$VERSION .
```
### Authentication

Authentication is used by the exporter as the mechanism to cross authenticate to the scraped appliance, therefore for each array it is required to provide the REST API token for an account that has a 'readonly' role. The api-token can be provided in two ways

- using the HTTP Authorization header of type 'Bearer', or
- via a configuration map in a specific configuration file.

The first option requires specifying the api-token value as the authorization parameter of the specific job in the Prometheus configuration file.
The second option provides the FlashBlade/api-token key-pair map for a list of arrays in a simple YAML configuration file that is passed as parameter to the exporter. This makes possible to write more concise Prometheus configuration files and also to configure other scrapers that cannot use the HTTP authentication header.

### Usage

```shell

usage: pure-fb-om-exporter [-h|--help] [-a|--address "<value>"] [-p|--port <integer>] [-d|--debug] [-t|--tokens <file>]

Pure Storage FB OpenMetrics exporter

Arguments:

-h --help Print help information
-a --address IP address for this exporter to bind to. Default: 0.0.0.0
-p --port Port for this exporter to listen. Default: 9490
-d --debug Enable debug. Default: false
-t --tokens API token(s) map file
```

The array token configuration file must have to following syntax:

```shell
<array_id1>:
address: <ip-address>|<hosname1>
api_token: <api-token1>
<array_id2>:
address: <ip-address2>|<hostname2>
api_token: <api-token2>
...
<array_idN>:
address: <ip-addressN>|<hostnameN>
api_token: <api-tokenN>
```

### Scraping endpoints

Expand All @@ -76,12 +117,13 @@ Authentication is used by the exporter as the mechanism to cross authenticate to
The exporter understands the following requests:


| URL | GET parameters | description |
| ------------------------------------------------- | -------------- | -------------------- |
| http://\<exporter-host\>:\<port\>/metrics | endpoint | Full array metrics |
| http://\<exporter-host\>:\<port\>/metrics/array | endpoint | Array metrics |
| http://\<exporter-host\>:\<port\>/metrics/clients | endpoint | Clients metrics |
| http://\<exporter-host\>:\<port\>/metrics/usage | endpoint | Quotas usage metrics |
| URL | GET parameters | description |
| ---------------------------------------------------| -------------- | --------------------------|
| http://\<exporter-host\>:\<port\>/metrics | endpoint | Full array metrics |
| http://\<exporter-host\>:\<port\>/metrics/array | endpoint | Array metrics |
| http://\<exporter-host\>:\<port\>/metrics/clients | endpoint | Clients metrics |
| http://\<exporter-host\>:\<port\>/metrics/usage | endpoint | Quotas usage metrics |
| http://\<exporter-host\>:\<port\>/metrics/policies | endpoint | NFS policies info metrics |


Depending on the target array, scraping for the whole set of metrics could result into timeout issues, in which case it is suggested either to increase the scraping timeout or to scrape each single endpoint instead.
Expand Down Expand Up @@ -155,6 +197,7 @@ A simple but complete example to deploy a full monitoring stack on kubernetes ca
| purefb_shardware_connectors_performance_errors | FlashBlade hardware connectors performance errors per sec |
| purefb_file_system_usage_users_bytes | FlashBlade file system users usage |
| purefb_file_system_usage_groups_bytes | FlashBlade file system groups usage |
| purefb_nfs_export_rule | FlashBlade NFS export policies information |

## Monitoring On-Premise with Prometheus and Grafana
Take a holistic overview of your Pure Storage FlashBlade estate on-premise with Prometheus and Grafana to summarize statistics such as:
Expand Down
78 changes: 67 additions & 11 deletions cmd/fb-om-exporter/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,67 @@ package main

import (
"context"
"flag"
"fmt"
"io"
"log"
"net/http"
"os"
config "purestorage/fb-openmetrics-exporter/internal/config"
collectors "purestorage/fb-openmetrics-exporter/internal/openmetrics-exporter"
client "purestorage/fb-openmetrics-exporter/internal/rest-client"
"strings"

"github.com/akamensky/argparse"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
"gopkg.in/yaml.v3"
)

var version string = "development"
var debug bool = false
var arraytokens config.FlashBladeList

func FileExists(args []string) error {
_, err := os.Stat(args[0])
return err
}

func main() {

host := flag.String("host", "0.0.0.0", "Address of the exporter")
port := flag.Int("port", 9491, "Port of the exporter")
d := flag.Bool("debug", false, "Debug")
flag.Parse()
addr := fmt.Sprintf("%s:%d", *host, *port)
debug = *d
log.Printf("Start Pure Storage FlashBlade exporter %s on %s", version, addr)
parser := argparse.NewParser("pure-fb-om-exporter", "Pure Storage FB OpenMetrics exporter")
host := parser.String("a", "address", &argparse.Options{Required: false, Help: "IP address for this exporter to bind to", Default: "0.0.0.0"})
port := parser.Int("p", "port", &argparse.Options{Required: false, Help: "Port for this exporter to listen", Default: 9491})
d := parser.Flag("d", "debug", &argparse.Options{Required: false, Help: "Enable debug", Default: false})
at := parser.File("t", "tokens", os.O_RDONLY, 0600, &argparse.Options{Required: false, Validate: FileExists, Help: "API token(s) map file"})
err := parser.Parse(os.Args)
if err != nil {
log.Fatalf("Error in token file: %v", err)
}
if !isNilFile(*at) {
defer at.Close()
buf := make([]byte, 1024)
arrlist := ""
for {
n, err := at.Read(buf)
if err == io.EOF {
break
}
if err != nil {
log.Fatalf("Reading token file: %v", err)
}
if n > 0 {
arrlist = arrlist + string(buf[:n])
}
}
buf = []byte(arrlist)
err := yaml.Unmarshal(buf, &arraytokens)
if err != nil {
log.Fatalf("Unmarshalling token file: %v", err)
}
}
debug = *d
addr := fmt.Sprintf("%s:%d", *host, *port)
log.Printf("Start Pure FlashBlade exporter %s on %s", version, addr)

http.HandleFunc("/", index)
http.HandleFunc("/metrics/array", func(w http.ResponseWriter, r *http.Request) {
Expand All @@ -37,6 +74,9 @@ func main() {
http.HandleFunc("/metrics/usage", func(w http.ResponseWriter, r *http.Request) {
metricsHandler(w, r)
})
http.HandleFunc("/metrics/policies", func(w http.ResponseWriter, r *http.Request) {
metricsHandler(w, r)
})
http.HandleFunc("/metrics", func(w http.ResponseWriter, r *http.Request) {
metricsHandler(w, r)
})
Expand All @@ -55,6 +95,7 @@ func metricsHandler(w http.ResponseWriter, r *http.Request) {
case "clients":
case "array":
case "usage":
case "policies":
default:
metrics = "all"
}
Expand All @@ -70,14 +111,18 @@ func metricsHandler(w http.ResponseWriter, r *http.Request) {
}
authHeader := r.Header.Get("Authorization")
authFields := strings.Fields(authHeader)
if len(authFields) != 2 || strings.ToLower(authFields[0]) != "bearer" {
address, apitoken := arraytokens.GetArrayParams(endpoint)
if len(authFields) == 2 && strings.ToLower(authFields[0]) == "bearer" {
apitoken = authFields[1]
address = endpoint
}
if apitoken == "" {
http.Error(w, "Target authorization token is missing", http.StatusBadRequest)
return
}
apitoken := authFields[1]

registry := prometheus.NewRegistry()
fbclient := client.NewRestClient(endpoint, apitoken, apiver, debug)
fbclient := client.NewRestClient(address, apitoken, apiver, debug)
if fbclient.Error != nil {
http.Error(w, "Error connecting to FlashBlade. Check your management endpoint and/or api token are correct.", http.StatusBadRequest)
return
Expand Down Expand Up @@ -127,10 +172,21 @@ func index(w http.ResponseWriter, r *http.Request) {
<td>endpoint</td>
<td>Provides only quota related metrics.</td>
</tr>
<tr>
<td>NFS export policies metrics</td>
<td><a href="/metrics/policies?endpoint=host">/metrics/policies</a></td>
<td>endpoint</td>
<td>Provides only NFS policies related metrics.</td>
</tr>
</tbody>
</table>
</body>
</html>`

fmt.Fprintf(w, "%s", msg)
}

func isNilFile(f os.File) bool {
var tf os.File
return f == tf
}
7 changes: 6 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,23 @@ module purestorage/fb-openmetrics-exporter
go 1.20

require (
github.com/akamensky/argparse v1.4.0
github.com/go-resty/resty/v2 v2.7.0
github.com/prometheus/client_golang v1.14.0
github.com/google/go-cmp v0.5.9
github.com/prometheus/client_golang v1.15.0
gopkg.in/yaml.v3 v3.0.1
)

require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/prometheus/client_model v0.3.0 // indirect
github.com/prometheus/common v0.42.0 // indirect
github.com/prometheus/procfs v0.9.0 // indirect
github.com/rogpeppe/go-internal v1.10.0 // indirect
golang.org/x/net v0.8.0 // indirect
golang.org/x/sys v0.6.0 // indirect
google.golang.org/protobuf v1.30.0 // indirect
Expand Down
19 changes: 17 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
github.com/akamensky/argparse v1.4.0 h1:YGzvsTqCvbEZhL8zZu2AiA5nq805NZh75JNj4ajn1xc=
github.com/akamensky/argparse v1.4.0/go.mod h1:S5kwC7IuDcEr5VeXtGPRVZ5o/FdhcMlQz4IZQuw64xA=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/go-resty/resty/v2 v2.7.0 h1:me+K9p3uhSmXtrBZ4k9jcEAfJmuC8IivWHwaLZwPrFY=
github.com/go-resty/resty/v2 v2.7.0/go.mod h1:9PWDzw47qPphMRFfhsyk0NnSgvluHcljSMVIq3w7q0I=
Expand All @@ -12,17 +15,25 @@ github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg
github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo=
github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
github.com/prometheus/client_golang v1.14.0 h1:nJdhIvne2eSX/XRAFV9PcvFFRbrjbcTUj0VP62TMhnw=
github.com/prometheus/client_golang v1.14.0/go.mod h1:8vpkKitgIVNcqrRBWh1C4TIUQgYNtG/XQE4E/Zae36Y=
github.com/prometheus/client_golang v1.15.0 h1:5fCgGYogn0hFdhyhLbw7hEsWxufKtY9klyvdNfFlFhM=
github.com/prometheus/client_golang v1.15.0/go.mod h1:e9yaBhRPU2pPNsZwE+JdQl0KEt1N9XgF6zxWmaC0xOk=
github.com/prometheus/client_model v0.3.0 h1:UBgGFHqYdG/TPFD1B1ogZywDqEkwp3fBMvqdiQ7Xew4=
github.com/prometheus/client_model v0.3.0/go.mod h1:LDGWKZIo7rky3hgvBe+caln+Dr3dPggB5dvjtD7w9+w=
github.com/prometheus/common v0.42.0 h1:EKsfXEYo4JpWMHH5cg+KOUWeuJSov1Id8zGR8eeI1YM=
github.com/prometheus/common v0.42.0/go.mod h1:xBwqVerjNdUDjgODMpudtOMwlOwf2SaTr1yjz4b7Zbc=
github.com/prometheus/procfs v0.9.0 h1:wzCHvIvM5SxWqYvwgVL7yJY8Lz3PKn49KQtpgMYJfhI=
github.com/prometheus/procfs v0.9.0/go.mod h1:+pB4zwohETzFnmlpe6yd2lSc+0/46IYZRB/chUwxUZY=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
golang.org/x/net v0.0.0-20211029224645-99673261e6eb/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.8.0 h1:Zrh2ngAOFYneWTAIAPethzeaQLuHwhuBkuV6ZiRnUaQ=
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
Expand All @@ -38,3 +49,7 @@ google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp0
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.30.0 h1:kPPoIgf3TsEvrm0PFe15JQ+570QVxYzEvvHqChK+cng=
google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
17 changes: 17 additions & 0 deletions internal/config/auth_tokens.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package config

type FlashBlade struct {
Address string `yaml:"address"`
ApiToken string `yaml:"api_token"`
}

type FlashBladeList map[string]FlashBlade

func (f *FlashBladeList) GetArrayParams(fb string) (string, string) {
for a_name, a := range *f {
if a_name == fb {
return a.Address, a.ApiToken
}
}
return "", ""
}
60 changes: 60 additions & 0 deletions internal/openmetrics-exporter/alerts_collector_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
package collectors


import (
"fmt"
"testing"
"regexp"
"strings"
"net/http"
"net/http/httptest"
"encoding/json"
"os"

"purestorage/fb-openmetrics-exporter/internal/rest-client"
)

func TestAlertsCollector(t *testing.T) {

ropen, _ := os.ReadFile("../../test/data/alerts_open.json")
rall, _ := os.ReadFile("../../test/data/alerts.json")
vers, _ := os.ReadFile("../../test/data/versions.json")
var aopen client.AlertsList
var aall client.AlertsList
json.Unmarshal(ropen, &aopen)
json.Unmarshal(rall, &aall)
server := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
urlall := regexp.MustCompile(`^/api/([0-9]+.[0-9]+)?/alerts$`)
urlopen := regexp.MustCompile(`^/api/([0-9]+.[0-9]+)?/alerts\?filter=state%3D%27open%27$`)
if r.URL.Path == "/api/api_version" {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
w.Write([]byte(vers))
} else if urlopen.MatchString(r.URL.Path + "?" + r.URL.RawQuery) {
w.Header().Set("x-auth-token", "faketoken")
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
w.Write([]byte(ropen))
} else if urlall.MatchString(r.URL.Path) {
w.Header().Set("x-auth-token", "faketoken")
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
w.Write([]byte(rall))
}
}))
endp := strings.Split(server.URL, "/")
e := endp[len(endp)-1]
al := make(map[string]float64)
for _, a := range aopen.Items {
al[fmt.Sprintf("%s,%s,%s", a.ComponentName, a.ComponentType, a.Severity)] += 1
}
want := make(map[string]bool)
for a, n := range al {
alert := strings.Split(a, ",")
want[fmt.Sprintf("label:<name:\"component_name\" value:\"%s\" > label:<name:\"component_type\" value:\"%s\" > label:<name:\"severity\" value:\"%s\" > gauge:<value:%g > ", alert[0], alert[1], alert[2], n)] = true
}
c := client.NewRestClient(e, "fake-api-token", "latest", false)
ac := NewAlertsCollector(c)
metricsCheck(t, ac, want)
server.Close()
}
Loading

0 comments on commit e4611f5

Please sign in to comment.