diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index d6017e75..f6c6dba7 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -27,9 +27,9 @@ jobs: run: | cd /tmp - wget https://dl.influxdata.com/influxdb/releases/influxdb_2.0.0-beta.16_linux_amd64.tar.gz - tar xvfz influxdb_2.0.0-beta.16_linux_amd64.tar.gz - sudo cp influxdb_2.0.0-beta.16_linux_amd64/{influx,influxd} /usr/local/bin/ + wget https://dl.influxdata.com/platform/nightlies/influx_nightly_linux_amd64.tar.gz + tar xvfz influx_nightly_linux_amd64.tar.gz + sudo cp influx_nightly_linux_amd64/{influx,influxd} /usr/local/bin/ influxd --http-bind-address :8086 --reporting-disabled > /dev/null 2>&1 & until curl -s http://localhost:8086/health; do sleep 1; done influx setup --host http://localhost:8086 -f -b dummy -o influxdata -u ci_user -p password diff --git a/Example_readme.md b/Example_readme.md index 9996519f..2e98c41d 100644 --- a/Example_readme.md +++ b/Example_readme.md @@ -28,12 +28,12 @@ List what resources your template provides in this section. That will allow user **Example:** - - 1 Bucket: `Telegraf`, 7d retention - - 3 Labels: `InfluxDB1.x`,`Solution`,`Telegraf` - - 1 Telegraf Configuration - - 3 Checks: `Disk Usage Check`, `Host Deadman`, and `Memory Usage Check` - - 2 Dashboards: `InfluxDB 1.x` and `Telegraf` - - 3 Variables: `bucket`, `influxdb_host`, and `telegraf_host` + - 1 Bucket: `Telegraf`, 7d retention + - 3 Labels: `InfluxDB1.x`,`Solution`,`Telegraf` + - 1 Telegraf Configuration + - 3 Checks: `Disk Usage Check`, `Host Deadman`, and `Memory Usage Check` + - 2 Dashboards: `InfluxDB 1.x` and `Telegraf` + - 3 Variables: `bucket`, `influxdb_host`, and `telegraf_host` ## Setup Instructions @@ -43,13 +43,13 @@ Describe any steps needed to finish setting up and running your template, includ **Example:** - Telegraf Configuration requires the following environment variables - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization - You **MUST** set these environment variables before running Telegraf using something similar to the following commands - - This can be found on the `Load Data` > `Tokens` page in your browser: `export INFLUX_TOKEN=TOKEN` - - Your Organization name can be found on the Settings page in your browser: `export INFLUX_ORG=my_org` +You **MUST** set these environment variables before running Telegraf using something similar to the following commands + - This can be found on the `Load Data` > `Tokens` page in your browser: `export INFLUX_TOKEN=TOKEN` + - Your Organization name can be found on the Settings page in your browser: `export INFLUX_ORG=my_org` ## Customizations @@ -57,7 +57,7 @@ Show off the flexibility of your template by letting users know different ways t **Example:** - You can easily update the Telegraf configurations to point to a specific InfluxDB 1.x location by setting the options in the [InfluxDB Input](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/influxdb) or the [InfluxDB 2.0 Output](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/influxdb_v2). +You can easily update the Telegraf configurations to point to a specific InfluxDB 1.x location by setting the options in the [InfluxDB Input](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/influxdb) or the [InfluxDB 2.0 Output](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/influxdb_v2). ## Contact diff --git a/README.md b/README.md index 64161e77..cc16e82d 100644 --- a/README.md +++ b/README.md @@ -16,21 +16,28 @@ Start by reading [how to use a template](docs/use_a_template.md), then check eac | Template | Description | Author | |----------------------|-------------|:------:| | [Air Quality](airquality/) | Retrieve air quality statistics from the US EPA website | [Kristina Robinson](https://github.com/kristinarobinson) | +| [Algorithmia](algorithmia/) | Monitor machine learning model performance metrics | [@koverholt](https://github.com/koverholt) | | [Apex Legends](apex_legends/) | Collect player metrics from the game Apex Legends | [@b3vis](https://github.com/b3vis) | | [AWS Cloudwatch Monitoring](aws_cloudwatch/) | Monitor AWS EC2 and ELB | [bonitoo.io](.) | +| [Ceph Cluster](ceph/) | Monitor your Ceph Cluster with Prometheus metrics | [@bonitoo.io](https://github.com/bonitoo-io) | | [Counter Strike: Global Offensive](csgo/) | Get stats about your game. Kills, Deaths and stats by weapon. | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | +| [Covid-19 in South America](covid19-southamerica/) | Current data and graphs covering Covid-19 cases and deaths in South America | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | +| [Cribl LogStream](criblio_logstream/) | Monitor and visualize your metric data from Cribl LogStream. | [Clint Sharp](https://github.com/criblio) | | [Currency Exchange Rates](currency_exchange_rates/) | Visualize and analyze currency exchange rates using Quandl. | Wojciech Kocjan | -| [Covid-19 in South America](InfluxDBv2_Covid19_SouthAmerica/) | Current data and graphs covering Covid-19 cases and deaths in South America | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | +| [DigitalOcean Billing](do_billing/) | Get your balance, month consumption and month to date balance from your DigitalOcean account | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | | [Docker](docker/) | Monitor your running docker containers. | [@russorat](https://github.com/russorat) | | [Downsampling](downsampling/) | Downsample common Telegraf metrics. | [@russorat](https://github.com/russorat) | | [Endpoint Security State](endpoint-security-state/) | Watch endpoint authentication availability and certificate status | [Darin Fisher](https://github.com/darinfisher) | | [Enviro+](enviro_plus/) | View the air quality readings from a Pimoroni Enviro+ particulate matter sensor. | Russ Savage | +| [Fail2Ban](fail2ban/) | Monitor your Fail2Ban instance on multiple hosts and services. | [bonitoo.io](.) | +| [Fireboard](fireboard/) | Monitor data collected by a Fireboard thermometer. | [Scott Anderson](https://github.com/sanderson) | | [Fortnite](fortnite/) | Track and analyze Fortnite player performance. | [@abalone23](https://github.com/abalone23) | | [GCP Monitoring](gcp_monitoring/) | Monitor Google Cloud Platform. | [bonitoo.io](.) | | [Github](github/) | View information about your Github Repos. Forks, Issues, Stars and more. | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | | [HAProxy](haproxy/) | Get metrics of your HAProxy instance. | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | | [InfluxDB 1.x Monitor](monitoring_influxdb_1.x/) | Monitor your already running InfluxDB 1.x instance. | [@russorat](https://github.com/russorat) | | [InfluxDB 2 OSS Metrics](influxdb2_oss_metrics/) | Monitor your InfluxDB 2 OSS instance using scrapers. | [@russorat](https://github.com/russorat) | +| [InfluxDB 2 Operational Monitoring](influxdb2_operational_monitoring/) | Monitor you tasks and cardinality on your already running InfluxDB 2 instance. | [@Anaisdg](https://github.com/Anaisdg) | | [Island Pulse (Modbus)](modbus/) | Monitor Island Pulse energy gauge over Modbus | Ray Farias | | [Jboss Wildfly](jboss_wildfly/) | View information your Jboss Wildfly Instance using Jolokia. Uptime, Heap Memory, Non Heap Memory, Memory Pool, etc | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | | [JMeter](apache_jmeter/) | This template provides Apache JMeter dashboard | [bonitoo.io](.) | @@ -39,12 +46,15 @@ Start by reading [how to use a template](docs/use_a_template.md), then check eac | [Linux System Monitor](linux_system/) | Monitor system resources on one or more Linux hosts. | [@russorat](https://github.com/russorat) | | [Jenkins](jenkins/) | Monitor your Jenkins node and jobs. | Ray Farias | | [Microsoft SQL Server](mssql/) | View information of SQL Server Instance. Uptime, Databases Activities, Read and Write times and more. | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | -| [MinIO](minio/) | Monitor MinIO instance. Uptime, CPU process time, Memory allocated, s3 total and current request, Storage used and available and more. | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | +| [MinIO](minio/) | Monitor MinIO instance. Uptime, CPU process time, Memory allocated, s3 total and current request, Storage used and available and more. | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | | [MongoDB](mongodb/) | View information of MongoDB Server. Uptime, Connectios, Queries, Documents Operations and more. | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | | [MySQL / MariaDB](mysql_mariadb/) | View information of MySQL Instance. Uptime, Current Queries, Active Threads, Connections, Traffic and more. | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | | [Network Interface Monitor](network_interface_performance/) | Monitor network interfaces on one or more hosts. | [@russorat](https://github.com/russorat) | +| [Nextcloud](nextcloud/) | Show stats about your Nextcloud Instance. | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | | [Postgres Monitor](postgresql/) | Monitor Postgres Server. CPU, Deadlocks, Data and more | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | +| [Raspberry Pi System Monitor](raspberry-pi/) | System overview monitoring for your Raspberry Pi with Raspbian. | [@bonitoo.io](https://github.com/bonitoo-io) | | [Redis Monitor](redis/) | Monitor your Redis server. | [@russorat](https://github.com/russorat) | +| [Sensu Go](sensu_go/) | Monitor the performance of your Sensu Go observability tool. | [@nikkictl](https://github.com/nikkictl) | | [sFlow Traffic Monitor](sflow/) | Monitor your sFlow traffic. | [@russorat](https://github.com/russorat) | | [SNMP Monitoring](snmp/) | Dashboards showing metrics provided via SNMP protocol | [bonitoo.io](.) | | [Speedtest Dashboard](speedtest/) | This template will show you information about the speed of your Internet connection using speedtest-cli | [Ignacio Van Droogenbroeck](https://github.com/xe-nvdk) | diff --git a/algorithmia/algorithmia.yml b/algorithmia/algorithmia.yml new file mode 100644 index 00000000..75165fc2 --- /dev/null +++ b/algorithmia/algorithmia.yml @@ -0,0 +1,253 @@ +apiVersion: influxdata.com/v2alpha1 +kind: Bucket +metadata: + name: unruffled-almeida-c0c001 +spec: + name: insights +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: mystifying-booth-40c001 +spec: + charts: + - axes: + - base: "10" + name: x + scale: linear + - base: "2" + name: y + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + - hex: '#DC4E58' + name: fire + type: text + value: 0.7 + decimalPlaces: 2 + height: 4 + kind: Single_Stat_Plus_Line + name: Risk Score + position: overlaid + queries: + - query: |- + from(bucket: "insights") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "kafka_consumer") + |> filter(fn: (r) => r["_field"] == "risk_score") + |> group() + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 4 + xCol: _time + yCol: _value + - colors: + - hex: '#32B08C' + name: viridian + type: min + - hex: '#DC4E58' + name: fire + type: max + value: 1 + decimalPlaces: 2 + height: 4 + kind: Gauge + name: Risk Score + queries: + - query: |- + from(bucket: "insights") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "kafka_consumer") + |> filter(fn: (r) => r["_field"] == "risk_score") + |> group() + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 4 + yPos: 4 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + - hex: '#DC4E58' + name: fire + type: text + value: 0.8 + decimalPlaces: 2 + height: 4 + kind: Single_Stat_Plus_Line + name: Approvals + position: overlaid + queries: + - query: |- + from(bucket: "insights") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "kafka_consumer") + |> filter(fn: (r) => r["_field"] == "approved") + |> group() + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 4 + xCol: _time + xPos: 4 + yCol: _value + - axes: + - name: x + - domain: + - 0 + - 1 + name: y + binSize: 20 + colors: + - hex: '#000004' + - hex: '#110a30' + - hex: '#320a5e' + - hex: '#57106e' + - hex: '#781c6d' + - hex: '#9a2865' + - hex: '#bc3754' + - hex: '#d84c3e' + - hex: '#ed6925' + - hex: '#f98e09' + - hex: '#fbb61a' + - hex: '#f4df53' + height: 4 + kind: Heatmap + name: Approved + queries: + - query: |- + from(bucket: "insights") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "kafka_consumer") + |> filter(fn: (r) => r["_field"] == "approved") + |> group() + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 4 + xCol: _time + xPos: 4 + yCol: _value + yPos: 4 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 4 + kind: Single_Stat_Plus_Line + name: Algorithm Duration + position: overlaid + queries: + - query: |- + from(bucket: "insights") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "kafka_consumer") + |> filter(fn: (r) => r["_field"] == "duration_milliseconds") + |> group() + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + suffix: ' ms' + width: 4 + xCol: _time + xPos: 8 + yCol: _value + - colors: + - hex: '#00C9FF' + name: laser + type: min + - hex: '#9394FF' + name: comet + type: max + value: 100 + decimalPlaces: 0 + height: 4 + kind: Gauge + name: Algorithm Duration + queries: + - query: |- + from(bucket: "insights") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "kafka_consumer") + |> filter(fn: (r) => r["_field"] == "duration_milliseconds") + |> group() + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + suffix: ' ms' + width: 4 + xPos: 8 + yPos: 4 + description: Monitor machine learning model performance metrics + name: Algorithmia - ML Model Performance Metrics +--- +apiVersion: influxdata.com/v2alpha1 +kind: Telegraf +metadata: + name: algorithmia-telegraf +spec: + name: Algorithmia + config: | + [agent] + interval = "10s" + round_interval = true + metric_batch_size = 1000 + metric_buffer_limit = 10000 + collection_jitter = "0s" + flush_interval = "10s" + flush_jitter = "0s" + precision = "" + hostname = "" + omit_hostname = false + + ############################################################################### + # OUTPUT PLUGINS # + ############################################################################### + + [[outputs.influxdb_v2]] + urls = ["$INFLUX_HOST"] + token = "$INFLUX_TOKEN" + organization = "$INFLUX_ORG" + bucket = "insights" + + ############################################################################### + # PROCESSOR PLUGINS # + ############################################################################### + + + ############################################################################### + # AGGREGATOR PLUGINS # + ############################################################################### + + + ############################################################################### + # INPUT PLUGINS # + ############################################################################### + + [[inputs.kafka_consumer]] + brokers = ["$KAFKA_BROKER"] + topics = ["$KAFKA_TOPIC"] + data_format = "json" + tag_keys = [ + "algorithm_name", + "algorithm_owner", + "algorithm_version" + ] + json_string_fields = [ + "session_id", + "request_id" + ] + name_override = "algorithmia" diff --git a/algorithmia/img/algorithmia-model-risk-dashboard.png b/algorithmia/img/algorithmia-model-risk-dashboard.png new file mode 100644 index 00000000..7434b17d Binary files /dev/null and b/algorithmia/img/algorithmia-model-risk-dashboard.png differ diff --git a/algorithmia/readme.md b/algorithmia/readme.md new file mode 100644 index 00000000..92fca142 --- /dev/null +++ b/algorithmia/readme.md @@ -0,0 +1,108 @@ +# Algorithmia + +Provided by: [Algorithmia](https://algorithmia.com/) + +Algorithmia is an MLOps platform that includes capabilities for data scientists, +application developers, and IT operators to deploy, manage, govern, and secure +machine learning and other probabilistic models in production. + +Algorithmia Insights is a feature of Algorithmia Enterprise and provides a +metrics pipeline that can be used to instrument, measure, and monitor your +machine learning models. + +![Algorithmia Model Risk Dashboard](img/algorithmia-model-risk-dashboard.png) + +This template allows you to stream operational metrics and user-defined, +inference-related metrics from Algorithmia to InfluxDB using Telegraf and Kafka. + +### Quick Install + +#### InfluxDB UI + +In the InfluxDB UI, go to Settings->Templates and enter this URL: +https://raw.githubusercontent.com/influxdata/community-templates/master/algorithmia/algorithmia.yml + +#### Influx CLI + +If you have your InfluxDB credentials [configured in the +CLI](https://v2.docs.influxdata.com/v2.0/reference/cli/influx/config/), you can +install this template with: + +``` +influx apply -u https://raw.githubusercontent.com/influxdata/community-templates/master/algorithmia/algorithmia.yml +``` + +## Included Resources + +This template includes the following: + + - 1 Bucket: `insights` + - 1 Dashboard: `Algorithmia - ML Model Performance Metrics` + - 1 Telegraf configuration + +## Setup Instructions + +This template allows you to monitor and visualize metrics from Algorithmia when +a model that has Insights enabled is queried. Each JSON payload from Algorithmia +includes operational metrics and inference-related metrics that can be analyzed +in InfluxDB. + +1. From your Algorithmia instance, configure Algorithmia Insights to connect to + a Kafka broker and specify a Kafka topic. +2. Start Telegraf using the configuration included in this template. +3. In Algorithmia, instrument your algorithm and publish a version that has + Insights enabled. +4. When your model is queried in Algorithmia, it will send metrics to InfluxDB + where you can monitor and visualize relevant model performance metrics. + +The Telegraf configuration requires the following environment variables to be +defined: + + - `INFLUX_HOST` - The URL for your InfluxDB host. + + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and + write data to the `telegraf` bucket. You can use your operator token to get + started. + + - `INFLUX_ORG` - The name of your organization in InfluxDB. + + - `KAFKA_BROKER` - The URL for your Kafka broker. + + - `KAFKA_TOPIC` - The Kafka topic that Algorithmia Insights is configured to + use. + +You **MUST** set these environment variables before running Telegraf using +something similar to the following commands: + + - This is the URL for your InfluxDB host and should be in the following format: + - `export INFLUX_HOST=http://1.2.3.4:8086/` + + - This can be found on the `Load Data` > `Tokens` page in InfluxDB: + - `export INFLUX_TOKEN=TOKEN` + + - Your organization name can be found on the `Settings` page in InfluxDB: + - `export INFLUX_ORG=my_org` + + - This is the URL for your Kafka broker and should be in the following format: + - `export KAFKA_BROKER=1.2.3.4:9092` + + - The Kafka topic should be the same topic that you configured in Algorithmia: + - `export KAFKA_TOPIC=insights` + +## Customizations + +Note that this dashboard uses the `duration_milliseconds` metric, which is one +of the operational metrics that is included in the default JSON payload from +Algorithmia. The other metrics used in the dashboard (`risk_score` and +`approved`) were defined by the algorithm developer and will be different +depending on your particular algorithm and use case. + +Additional metrics can include any user-defined, inference-related metrics that +are specified in Algorithmia by the algorithm developer then analyzed in +InfluxDB. + +## Contact + +Author: Kristopher Overholt, https://algorithmia.com/ + +Github: @koverholt diff --git a/apache_jmeter/readme.md b/apache_jmeter/readme.md index 072ef15e..4a9ee512 100644 --- a/apache_jmeter/readme.md +++ b/apache_jmeter/readme.md @@ -39,8 +39,8 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates The template consists of the following: - - 1 Dashboard: `JMeter` - - 1 Variable: `bucket` + - 1 Dashboard: `JMeter` + - 1 Variable: `bucket` The sample **Test Plan** boilerplate project in the `samples` directory. @@ -52,7 +52,7 @@ General instructions on using InfluxDB Templates can be found in the [use a temp **Import the template** ``` - influx apply --file ./apache_jmeter.yml +influx apply --file ./apache_jmeter.yml ``` **Start with a sample test project:** diff --git a/apex_legends/readme.md b/apex_legends/readme.md index 73e05633..531588af 100644 --- a/apex_legends/readme.md +++ b/apex_legends/readme.md @@ -25,9 +25,9 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Bucket: apexlegends - - 2 Dashboards: Apex Legends for InfluxDB 2 and for Grafana - - 1 Label: Game Data + - 1 Bucket: apexlegends + - 2 Dashboards: Apex Legends for InfluxDB 2 and for Grafana + - 1 Label: Game Data ## Setup Instructions @@ -53,13 +53,14 @@ Due to limitations on the API and the way Telegraf processes the JSON data it's This will run curl every 5 minutes to pull the stats from the API endpoint and write them to `/etc/telegraf/apex_legends.json` - This can be anywhere your Telegraf instance can read from. - Telegraf Configuration requires the following environment variables - - `INFLUX_HOST` - The URL for your InfluxDB instance. - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization - You **MUST** set these environment variables before running Telegraf using something similar to the following commands - - This can be found on the `Load Data` > `Tokens` page in your browser: `export INFLUX_TOKEN=TOKEN` - - Your Organization name can be found on the Settings page in your browser: `export INFLUX_ORG=my_org` +Telegraf Configuration requires the following environment variables + - `INFLUX_HOST` - The URL for your InfluxDB instance. + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization + +You **MUST** set these environment variables before running Telegraf using something similar to the following commands + - This can be found on the `Load Data` > `Tokens` page in your browser: `export INFLUX_TOKEN=TOKEN` + - Your Organization name can be found on the Settings page in your browser: `export INFLUX_ORG=my_org` ## Contact diff --git a/ceph/ceph-cluster.yml b/ceph/ceph-cluster.yml new file mode 100755 index 00000000..a3449bf0 --- /dev/null +++ b/ceph/ceph-cluster.yml @@ -0,0 +1,1110 @@ +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: shiny-chatelet-9b6001 +spec: + color: '#F95F53' + name: ceph +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: cranky-allen-db6001 +spec: + language: flux + name: bucket + query: |- + buckets() + |> filter(fn: (r) => r.name !~ /^_/) + |> rename(columns: {name: "_value"}) + |> keep(columns: ["_value"]) + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: optimistic-feynman-db6001 +spec: + charts: + - height: 2 + kind: Markdown + name: Name this Cell + note: |- + # Ceph Cluster + + It is compatible with a Rook Ceph cluster and possibly also most Ceph Cluster with the Ceph MGR Prometheus module enabled. + + The Telegraf Prometheus input plugin needs to scrape the Ceph MGR(s) Prometheus Metrics Endpoints. + width: 4 + - colors: + - hex: '#7CE490' + id: base + name: honeydew + type: background + - hex: '#FFD255' + id: 54d4f688-00b3-421e-b1a5-aee54f09d0b2 + name: thunder + type: background + value: 1 + - hex: '#F95F53' + id: 083348b3-2600-4c36-9122-6f48b8585485 + name: curacao + type: background + value: 2 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Cluster Health Status + queries: + - query: "from(bucket: v.bucket)\n |> range(start: v.timeRangeStart, stop: + v.timeRangeStop)\n |> filter(fn: (r) => r[\"_measurement\"] == \"prometheus\")\n + \ |> filter(fn: (r) => r[\"_field\"] == \"ceph_health_status\") \n + \ |> yield()" + width: 2 + yPos: 2 + - height: 1 + kind: Markdown + name: Name this Cell + note: '## OSDs' + width: 2 + yPos: 3 + - colors: + - hex: '#00C9FF' + id: base + name: laser + type: text + decimalPlaces: 1 + height: 1 + kind: Single_Stat + name: PGs per OSD + queries: + - query: |- + num_pg = from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_numpg") + |> aggregateWindow(every: v.windowPeriod, fn: last) + |> group(columns: ["_time", "_stop", "_start", "host", "_measurement"], mode:"by") + |> sum(column: "_value") + + num_osd = from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_metadata") + |> aggregateWindow(every: v.windowPeriod, fn: last) + |> group(columns: ["_time", "_stop", "_start", "host", "_measurement"], mode:"by") + |> sum(column: "_value") + + join(tables: {pgs:num_pg, osds:num_osd}, on: ["_time", "_stop", "_start", "host"]) + |> map(fn: (r) => ({ + _time: r._time, + _value: (r._value_pgs / r._value_osds) + }) + ) + suffix: ' PGs/OSD' + width: 4 + yPos: 4 + - colors: + - hex: '#00C9FF' + id: base + name: laser + type: background + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: OSDs Total + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_metadata") + |> group(columns: ["_field", "_measurement", "_time"]) + |> count() + |> yield(name: "count") + suffix: ' OSDs' + width: 2 + yPos: 5 + - colors: + - hex: '#F95F53' + id: base + name: curacao + type: background + - hex: '#383846' + id: e415d0a8-8ab1-457e-9e7f-713e42634d77 + name: pepper + type: background + value: 1 + - hex: '#7CE490' + id: 717e0ba2-d16d-4d40-ad9a-1d4535309157 + name: honeydew + type: background + value: 2 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: OSDs In + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_in") + |> group(columns: ["_time", "_measurement", "_field"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> yield(name: "sum") + suffix: ' In' + width: 2 + yPos: 6 + - colors: + - hex: '#FFD255' + id: base + name: thunder + type: background + - hex: '#7CE490' + id: daa462ee-5f11-45b0-9242-32755810cb5a + name: honeydew + type: background + value: 2 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: OSDs Up + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_up") + |> group(columns: ["_time", "_measurement", "_field"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> yield(name: "sum") + suffix: ' Up' + width: 2 + yPos: 7 + - height: 1 + kind: Markdown + name: Name this Cell + note: '## MONs' + width: 2 + yPos: 8 + - colors: + - hex: '#F95F53' + id: base + name: curacao + type: background + - hex: '#FFD255' + id: 317466dc-92b6-40f9-a493-1960d1d6eb3e + name: thunder + type: background + value: 1 + - hex: '#7CE490' + id: daa462ee-5f11-45b0-9242-32755810cb5a + name: honeydew + type: background + value: 2 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: MON Quorum Status + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_mon_quorum_status") + |> group(columns: ["_time", "_measurement", "_field"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> yield(name: "sum") + suffix: ' In Quorum' + width: 2 + yPos: 9 + - colors: + - hex: '#F95F53' + id: base + name: curacao + type: background + - hex: '#FFD255' + id: 317466dc-92b6-40f9-a493-1960d1d6eb3e + name: thunder + type: background + value: 1 + - hex: '#7CE490' + id: daa462ee-5f11-45b0-9242-32755810cb5a + name: honeydew + type: background + value: 2 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: MON Quorum Total + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_mon_quorum_status") + |> group(columns: ["_time", "_measurement", "_field"]) + |> count() + |> yield(name: "count") + suffix: ' In Total' + width: 2 + yPos: 10 + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + - hex: '#F95F53' + id: f50310d1-f341-4153-ab4a-a93699f8689d + name: curacao + type: text + value: 1 + - hex: '#ffffff' + id: ef9e459e-4159-4ea2-b098-578a7c75fae7 + name: white + type: text + value: 75 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Cluster Health Status + queries: + - query: "from(bucket: v.bucket)\n |> range(start: v.timeRangeStart, stop: + v.timeRangeStop)\n |> filter(fn: (r) => r[\"_measurement\"] == \"prometheus\")\n + \ |> filter(fn: (r) => r[\"_field\"] == \"ceph_health_status\") \n + \ |> toString()\n |> map(fn: (r) => ({ r with _value: \n if + r._value == \"0\"\n then \"HEALTH_OK\"\n else if r._value + == \"1\"\n then \"HEALTH_WARN\"\n else \"HEALTH_ERROR\"\n + \ }))\n |> yield()" + width: 2 + xPos: 2 + yPos: 2 + - height: 1 + kind: Markdown + name: Name this Cell + note: '## PGs' + width: 2 + xPos: 2 + yPos: 3 + - colors: + - hex: '#00C9FF' + id: base + name: laser + type: background + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: PGs Total + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_pg_total") + |> group(columns: ["_field", "_measurement", "_time", "_stop", "_start"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> yield(name: "sum") + suffix: ' PGs' + width: 2 + xPos: 2 + yPos: 5 + - colors: + - hex: '#7CE490' + id: base + name: honeydew + type: background + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: PGs Active + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_pg_active") + |> group(columns: ["_field", "_measurement", "_time", "_stop", "_start"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> yield(name: "sum") + suffix: ' Active' + width: 2 + xPos: 2 + yPos: 6 + - colors: + - hex: '#7CE490' + id: base + name: honeydew + type: text + - hex: '#FFD255' + id: 523bf8ce-a739-43e9-ac03-597a79d0478c + name: thunder + type: text + value: 1 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: PGs Down + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_pg_down") + |> group(columns: ["_field", "_measurement", "_time", "_stop", "_start"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> yield(name: "sum") + suffix: ' Down' + width: 2 + xPos: 2 + yPos: 7 + - colors: + - hex: '#7CE490' + id: base + name: honeydew + type: text + - hex: '#FFB94A' + id: c09ffecf-2494-4aef-b374-62da0832be6e + name: pineapple + type: text + value: 1 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: PGs Undersized + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_pg_undersized") + |> group(columns: ["_field", "_measurement", "_time", "_stop", "_start"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> yield(name: "sum") + suffix: ' Undersized' + width: 2 + xPos: 2 + yPos: 8 + - colors: + - hex: '#7CE490' + id: base + name: honeydew + type: text + - hex: '#FFD255' + id: dd7a19b3-88b9-4231-ad94-a5b670996d2c + name: thunder + type: text + value: 1 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: PGs Degraded + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_pg_degraded") + |> group(columns: ["_field", "_measurement", "_time", "_stop", "_start"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> yield(name: "sum") + suffix: ' Degraded' + width: 2 + xPos: 2 + yPos: 9 + - colors: + - hex: '#7CE490' + id: base + name: honeydew + type: text + - hex: '#FFD255' + id: 612b6132-79d4-4abe-b8df-f7b8bd97e0f2 + name: thunder + type: text + value: 1 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: PGs Inconsistent + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_pg_inconsistent") + |> group(columns: ["_field", "_measurement", "_time", "_stop", "_start"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> yield(name: "sum") + suffix: ' Inconsistent' + width: 2 + xPos: 2 + yPos: 10 + - height: 1 + kind: Markdown + name: Name this Cell + note: '## Cluster Latency' + width: 2 + xPos: 4 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + suffix: ' ms' + colors: + - hex: '#8F8AF4' + id: 7cb1c6f3-f015-4872-9aea-be96ed91ac2b + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#A51414' + id: 1b7af10d-2b32-4890-9318-929473a7bc67 + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#F4CF31' + id: 44279c3e-4168-433d-88d6-ab822150f805 + name: Do Androids Dream of Electric Sheep? + type: scale + geom: monotoneX + height: 2 + hoverDimension: auto + kind: Xy + legendOpacity: 1 + legendOrientationThreshold: 10 + name: OSD Apply Latency + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_apply_latency_ms") + |> group(columns: ["ceph_daemon"]) + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + shade: true + width: 2 + xCol: _time + xPos: 4 + yCol: _value + yPos: 1 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + suffix: ' ms' + colors: + - hex: '#8F8AF4' + id: 7cb1c6f3-f015-4872-9aea-be96ed91ac2b + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#A51414' + id: 1b7af10d-2b32-4890-9318-929473a7bc67 + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#F4CF31' + id: 44279c3e-4168-433d-88d6-ab822150f805 + name: Do Androids Dream of Electric Sheep? + type: scale + geom: monotoneX + height: 2 + hoverDimension: auto + kind: Xy + legendOpacity: 1 + legendOrientationThreshold: 10 + name: OSD Commit Latency + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_commit_latency_ms") + |> group(columns: ["ceph_daemon"]) + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + shade: true + width: 2 + xCol: _time + xPos: 4 + yCol: _value + yPos: 3 + - colors: + - hex: '#00C9FF' + id: base + name: laser + type: text + decimalPlaces: 1 + height: 1 + kind: Single_Stat + name: OSD Apply Latency Average 10m + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_apply_latency_ms") + |> group(columns: ["ceph_daemon"]) + |> timedMovingAverage(every: 1m, period: 10m) + suffix: ' ms' + width: 2 + xPos: 4 + yPos: 5 + - colors: + - hex: '#00C9FF' + id: base + name: laser + type: text + decimalPlaces: 1 + height: 1 + kind: Single_Stat + name: OSD Commit Latency Average 10m + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_commit_latency_ms") + |> group(columns: ["ceph_daemon"]) + |> timedMovingAverage(every: 1m, period: 10m) + suffix: ' ms' + width: 2 + xPos: 4 + yPos: 6 + - height: 1 + kind: Markdown + name: Name this Cell + note: '## Cluster Pools' + width: 2 + xPos: 4 + yPos: 7 + - colors: + - hex: '#00C9FF' + id: base + name: laser + type: background + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: PGs Total + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_pool_metadata") + |> group(columns: ["_field", "_measurement", "_time"]) + |> count() + |> yield(name: "count") + suffix: ' Pools' + width: 2 + xPos: 4 + yPos: 8 + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + fieldOptions: + - displayName: _time + fieldName: _time + visible: true + - displayName: _field + fieldName: _field + visible: true + - displayName: _measurement + fieldName: _measurement + visible: true + - displayName: _value + fieldName: _value + visible: true + - displayName: pool_id + fieldName: pool_id + visible: true + - displayName: _start + fieldName: _start + visible: true + - displayName: _stop + fieldName: _stop + visible: true + - displayName: address + fieldName: address + visible: true + - displayName: host + fieldName: host + visible: true + - displayName: name + fieldName: name + visible: true + - displayName: url + fieldName: url + visible: true + height: 2 + kind: Table + name: Pools + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: -1m) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_pool_metadata") + |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false) + |> group(columns: ["_time"]) + |> keep(columns: ["name", "pool_id", "name"]) + tableOptions: + sortBy: pool_id + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 2 + xPos: 4 + yPos: 9 + - height: 1 + kind: Markdown + name: Name this Cell + note: '## Cluster Capacity' + width: 3 + xPos: 6 + - colors: + - hex: '#00C9FF' + id: "0" + name: laser + type: min + - hex: '#DC4E58' + id: "1" + name: fire + type: max + value: 100 + decimalPlaces: 1 + height: 3 + kind: Gauge + name: Capacity Used In % + queries: + - query: |- + bytes_used = from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_stat_bytes_used") + |> aggregateWindow(every: v.windowPeriod, fn: last) + |> group(columns: ["_time", "_stop", "_start", "host", "_measurement"], mode:"by") + |> sum(column: "_value") + + bytes_total = from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_stat_bytes") + |> aggregateWindow(every: v.windowPeriod, fn: last) + |> group(columns: ["_time", "_stop", "_start", "host", "_measurement"], mode:"by") + |> sum(column: "_value") + + join(tables: {used:bytes_used, total:bytes_total}, on: ["_time", "_stop", "_start", "host"]) + |> map(fn: (r) => ({ + _time: r._time, + _value: (r._value_used / r._value_total) * 100.0 + }) + ) + suffix: '%' + tickSuffix: "true" + width: 5 + xPos: 6 + yPos: 1 + - height: 1 + kind: Markdown + name: Name this Cell + note: '## Cluster I/O' + width: 3 + xPos: 6 + yPos: 4 + - axes: + - base: "10" + name: x + scale: linear + - base: "2" + name: y + scale: linear + suffix: Bps + colors: + - hex: '#8F8AF4' + id: 7cb1c6f3-f015-4872-9aea-be96ed91ac2b + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#A51414' + id: 1b7af10d-2b32-4890-9318-929473a7bc67 + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#F4CF31' + id: 44279c3e-4168-433d-88d6-ab822150f805 + name: Do Androids Dream of Electric Sheep? + type: scale + geom: monotoneX + height: 2 + hoverDimension: auto + kind: Xy + legendOpacity: 1 + legendOrientationThreshold: 10 + name: Throughput + position: overlaid + queries: + - query: "from(bucket: v.bucket)\n |> range(start: v.timeRangeStart, stop: + v.timeRangeStop)\n |> filter(fn: (r) => r[\"_measurement\"] == \"prometheus\")\n + \ |> filter(fn: (r) => r[\"_field\"] == \"ceph_osd_op_w_in_bytes\" + or r[\"_field\"] == \"ceph_osd_op_r_out_bytes\" or r[\"_field\"] == + \"ceph_osd_op_rw_in_bytes\") \n |> group(columns: [\"_field\"], mode:\"by\")\n + \ |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: + false)\n |> derivative(unit: 1s, nonNegative: true, columns: [\"_value\"], + timeColumn: \"_time\")" + shade: true + width: 3 + xCol: _time + xPos: 6 + yCol: _value + yPos: 5 + - axes: + - base: "10" + name: x + scale: linear + - name: y + scale: linear + colors: + - hex: '#8F8AF4' + id: 7cb1c6f3-f015-4872-9aea-be96ed91ac2b + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#A51414' + id: 1b7af10d-2b32-4890-9318-929473a7bc67 + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#F4CF31' + id: 44279c3e-4168-433d-88d6-ab822150f805 + name: Do Androids Dream of Electric Sheep? + type: scale + geom: monotoneX + height: 2 + hoverDimension: auto + kind: Xy + legendOpacity: 1 + legendOrientationThreshold: 10 + name: IOPS + position: overlaid + queries: + - query: "from(bucket: v.bucket)\n |> range(start: v.timeRangeStart, stop: + v.timeRangeStop)\n |> filter(fn: (r) => r[\"_measurement\"] == \"prometheus\")\n + \ |> filter(fn: (r) => r[\"_field\"] == \"ceph_osd_op_r\" or r[\"_field\"] + == \"ceph_osd_op_w\" or r[\"_field\"] == \"ceph_osd_op_rw\") \n |> + group(columns: [\"_field\"], mode:\"by\")\n |> aggregateWindow(every: + v.windowPeriod, fn: sum, createEmpty: false)\n |> derivative(unit: + 1s, nonNegative: true, columns: [\"_value\"], timeColumn: \"_time\")" + shade: true + width: 3 + xCol: _time + xPos: 6 + yCol: _value + yPos: 7 + - axes: + - base: "10" + name: x + scale: linear + - base: "2" + name: y + scale: linear + suffix: Op/s + colors: + - hex: '#8F8AF4' + id: 7cb1c6f3-f015-4872-9aea-be96ed91ac2b + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#A51414' + id: 1b7af10d-2b32-4890-9318-929473a7bc67 + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#F4CF31' + id: 44279c3e-4168-433d-88d6-ab822150f805 + name: Do Androids Dream of Electric Sheep? + type: scale + geom: monotoneX + height: 2 + hoverDimension: auto + kind: Xy + legendOpacity: 1 + legendOrientationThreshold: 10 + name: Recovery Rate + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_recovery_ops") + |> group(columns: ["pool_id"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> derivative(unit: 1s, nonNegative: true, columns: ["_value"], timeColumn: "_time") + shade: true + width: 3 + xCol: _time + xPos: 6 + yCol: _value + yPos: 9 + - colors: + - hex: '#00C9FF' + id: base + name: laser + type: text + decimalPlaces: 1 + height: 1 + kind: Single_Stat + name: Capacity Total GB + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_stat_bytes") + |> group(columns: ["_field", "_measurement", "_time", "_stop", "_start"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> map(fn: (r) => ({ r with _value: r._value / 1073741824.0 })) + suffix: ' GB' + width: 1 + xPos: 9 + - height: 1 + kind: Markdown + name: Name this Cell + note: '## Objects' + width: 2 + xPos: 9 + yPos: 4 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#00C9FF' + id: base + name: laser + type: text + - hex: '#8F8AF4' + id: eaba9af5-31b5-44c4-aec4-aac18b8d78ee + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#A51414' + id: 2fcab4d3-ea8e-4d34-87f2-f3f97367cf0d + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#F4CF31' + id: 996ca3aa-0738-489b-be67-bcd913e0fccf + name: Do Androids Dream of Electric Sheep? + type: scale + decimalPlaces: 0 + height: 2 + hoverDimension: auto + kind: Single_Stat_Plus_Line + legendOpacity: 1 + legendOrientationThreshold: 10 + name: Objects In Pools + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_pool_objects") + |> group(columns: ["_field"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> yield(name: "sum") + shade: true + width: 2 + xCol: _time + xPos: 9 + yCol: _value + yPos: 5 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#31C0F6' + id: 58262bed-f65a-4f87-8b0f-598eca74964a + name: Nineteen Eighty Four + type: scale + - hex: '#A500A5' + id: 324f44e9-a15d-4b63-90e2-c6c3e9251f32 + name: Nineteen Eighty Four + type: scale + - hex: '#FF7E27' + id: 9ec0304b-96a3-47f7-bdbc-f4b2f8a093dd + name: Nineteen Eighty Four + type: scale + geom: line + height: 2 + hoverDimension: auto + kind: Xy + legendOpacity: 1 + legendOrientationThreshold: 10 + name: Objects - Status + position: overlaid + queries: + - query: |- + from(bucket: "demo") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_num_objects_degraded" or r["_field"] == "ceph_num_objects_misplaced" or r["_field"] == "ceph_num_objects_unfound") + |> group(columns: ["_field"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> yield(name: "sum") + shade: true + width: 2 + xCol: _time + xPos: 9 + yCol: _value + yPos: 7 + - axes: + - base: "10" + name: x + scale: linear + - base: "2" + name: y + scale: linear + suffix: Bps + colors: + - hex: '#8F8AF4' + id: 7cb1c6f3-f015-4872-9aea-be96ed91ac2b + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#A51414' + id: 1b7af10d-2b32-4890-9318-929473a7bc67 + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#F4CF31' + id: 44279c3e-4168-433d-88d6-ab822150f805 + name: Do Androids Dream of Electric Sheep? + type: scale + geom: monotoneX + height: 2 + hoverDimension: auto + kind: Xy + legendOpacity: 1 + legendOrientationThreshold: 10 + name: Egress / Ingress + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_pool_stored") + |> group(columns: ["pool_id"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> derivative(unit: 1s, nonNegative: true, columns: ["_value"], timeColumn: "_time") + shade: true + width: 2 + xCol: _time + xPos: 9 + yCol: _value + yPos: 9 + - colors: + - hex: '#00C9FF' + id: base + name: laser + type: text + decimalPlaces: 1 + height: 1 + kind: Single_Stat + name: Capacity Used GB + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "prometheus") + |> filter(fn: (r) => r["_field"] == "ceph_osd_stat_bytes_used") + |> group(columns: ["_field", "_measurement", "_time", "_stop", "_start"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> map(fn: (r) => ({ r with _value: r._value / 1073741824.0 })) + suffix: ' GB' + width: 1 + xPos: 10 + description: Ceph cluster running in Kubernetes monitored by Prometheus metrics + name: Ceph Cluster +--- +apiVersion: influxdata.com/v2alpha1 +kind: Telegraf +metadata: + name: relaxed-euclid-db6001 +spec: + config: "# Configuration for telegraf agent\n[agent]\n ## Default data collection + interval for all inputs\n interval = \"1m\"\n ## Rounds collection interval + to 'interval'\n ## ie, if interval=\"10s\" then always collect on :00, :10, + :20, etc.\n round_interval = true\n\n ## Telegraf will send metrics to outputs + in batches of at most\n ## metric_batch_size metrics.\n ## This controls + the size of writes that Telegraf sends to output plugins.\n metric_batch_size + = 10000\n\n ## For failed writes, telegraf will cache metric_buffer_limit + metrics for each\n ## output, and will flush this buffer on a successful + write. Oldest metrics\n ## are dropped first when this buffer fills.\n ## + This buffer only fills when writes fail to output plugin(s).\n metric_buffer_limit + = 100000\n\n ## Collection jitter is used to jitter the collection by a random + amount.\n ## Each plugin will sleep for a random time within jitter before + collecting.\n ## This can be used to avoid many plugins querying things like + sysfs at the\n ## same time, which can have a measurable effect on the system.\n + \ collection_jitter = \"0s\"\n\n ## Default flushing interval for all outputs. + Maximum flush_interval will be\n ## flush_interval + flush_jitter\n flush_interval + = \"10s\"\n ## Jitter the flush interval by a random amount. This is primarily + to avoid\n ## large write spikes for users running a large number of telegraf + instances.\n ## ie, a jitter of 5s and interval 10s means flushes will happen + every 10-15s\n flush_jitter = \"0s\"\n\n ## By default or when set to \"0s\", + precision will be set to the same\n ## timestamp order as the collection + interval, with the maximum being 1s.\n ## ie, when interval = \"10s\", + precision will be \"1s\"\n ## when interval = \"250ms\", precision + will be \"1ms\"\n ## Precision will NOT be used for service inputs. It is + up to each individual\n ## service input to set the timestamp at the appropriate + precision.\n ## Valid time units are \"ns\", \"us\" (or \"µs\"), \"ms\", + \"s\".\n precision = \"\"\n\n ## Logging configuration:\n ## Run telegraf + with debug log messages.\n debug = false\n ## Run telegraf in quiet mode + (error log messages only).\n quiet = false\n ## Specify the log file name. + The empty string means to log to stderr.\n logfile = \"\"\n\n ## Override + default hostname, if empty use os.Hostname()\n hostname = \"\"\n ## If set + to true, do no set the \"host\" tag in the telegraf agent.\n omit_hostname + = false\n[[outputs.influxdb_v2]]\t\n ## The URLs of the InfluxDB cluster + nodes.\n ##\n ## Multiple URLs can be specified for a single cluster, only + ONE of the\n ## urls will be written to each interval.\n ## urls exp: http://127.0.0.1:9999\n + \ urls = [\"$INFLUX_HOST\"]\n\n ## Token for authentication.\n token = \"$INFLUX_TOKEN\"\n\n + \ ## Organization is the name of the organization you wish to write to; must + exist.\n organization = \"$INFLUX_ORG\"\n\n ## Destination bucket to write + into.\n bucket = \"$INFLUX_BUCKET\"\n\n# Read metrics from one or many prometheus + clients\n[[inputs.prometheus]]\n ## An array of urls to scrape metrics from.\n + \ # urls = [\"http://localhost:9100/metrics\"]\n\n ## Metric version controls + the mapping from Prometheus metrics into\n ## Telegraf metrics. When using + the prometheus_client output, use the same\n ## value in both plugins to + ensure metrics are round-tripped without\n ## modification.\n ##\n ## example: + metric_version = 1; deprecated in 1.13\n ## metric_version = 2; + recommended version\n metric_version = 2\n\n ## An array of Kubernetes services + to scrape metrics from.\n ## \n kubernetes_services = [\"$CEPH_MGR_SVC_URLS\"]\n\n + \ ## Kubernetes config file to create client from.\n # kube_config = \"/path/to/kubernetes.config\"\n\n + \ ## Scrape Kubernetes pods for the following prometheus annotations:\n ## + - prometheus.io/scrape: Enable scraping for this pod\n ## - prometheus.io/scheme: + If the metrics endpoint is secured then you will need to\n ## set this + to `https` & most likely set the tls config.\n ## - prometheus.io/path: If + the metrics path is not /metrics, define it with this annotation.\n ## - + prometheus.io/port: If port is not 9102 use this annotation\n monitor_kubernetes_pods + = false\n ## Restricts Kubernetes monitoring to a single namespace\n ## + \ ex: monitor_kubernetes_pods_namespace = \"default\"\n # monitor_kubernetes_pods_namespace + = \"\"\n # label selector to target pods which have the label\n # kubernetes_label_selector + = \"env=dev,app=nginx\"\n # field selector to target pods\n # eg. To scrape + pods on a specific node\n # kubernetes_field_selector = \"spec.nodeName=$HOSTNAME\"\n\n + \ ## Use bearer token for authorization. ('bearer_token' takes priority)\n + \ bearer_token = \"/run/secrets/kubernetes.io/serviceaccount/token\"\n ## + OR\n # bearer_token_string = \"abc_123\"\n\n ## HTTP Basic Authentication + username and password. ('bearer_token' and\n ## 'bearer_token_string' take + priority)\n # username = \"\"\n # password = \"\"\n\n ## Specify timeout + duration for slower prometheus clients (default is 3s)\n # response_timeout + = \"3s\"\n\n ## Optional TLS Config\n # tls_ca = /path/to/cafile\n # tls_cert + = /path/to/certfile\n # tls_key = /path/to/keyfile\n ## Use TLS but skip + chain & host verification\n # insecure_skip_verify = false\n" + description: using ceph-mgr's built-in prometheus exporter + name: ceph-prometheus diff --git a/ceph/img/ceph_dashboard.jpg b/ceph/img/ceph_dashboard.jpg new file mode 100755 index 00000000..96dc6b5b Binary files /dev/null and b/ceph/img/ceph_dashboard.jpg differ diff --git a/ceph/readme.md b/ceph/readme.md new file mode 100644 index 00000000..cc4bb4cc --- /dev/null +++ b/ceph/readme.md @@ -0,0 +1,63 @@ +# Ceph Cluster Template + +Provided by: @bonitoo.io + +Ceph is a free-software storage platform, implements object storage on a single distributed computer cluster, and provides interfaces for object-, block- and file-level storage. Ceph aims primarily for completely distributed operation without a single point of failure, and freely available. + +This InfluxDB template can be used to monitor your Ceph Cluster system. It uses Telegraf Prometheus input plugin to scrape Prometheus metrics from Ceph's MGR service endpoint. +> It is compatible with a Rook Ceph cluster and possibly also most Ceph Cluster with the Ceph MGR Prometheus module enabled. + +![Example Dashboard Screenshot](img/ceph_dashboard.jpg) + +### Quick Install + +#### InfluxDB UI + +In the InfluxDB UI, go to Settings->Templates and enter this URL: https://raw.githubusercontent.com/influxdata/community-templates/master/ceph/ceph-cluster.yml + +#### Influx CLI +If you have your InfluxDB credentials [configured in the CLI](https://v2.docs.influxdata.com/v2.0/reference/cli/influx/config/), you can install this template with: + +``` +influx apply -u https://raw.githubusercontent.com/influxdata/community-templates/master/ceph/ceph-cluster.yml +``` + +## Included Resources + + - 2 Labels: `ceph`,`inputs.prometheus` + - 1 Telegraf Configuration + - 1 Dashboard: `Ceph Cluster` + - 1 Variable: `bucket` + +## Setup Instructions + +General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. + +### Requirements: + +#### Ceph Cluster: + +It is compatible with a [Rook Ceph cluster](https://rook.io/docs/rook/v1.5/) running in Kubernetes and possibly also most Ceph Cluster with the Ceph MGR [Prometheus module enabled](https://rook.io/docs/rook/v1.5/ceph-monitoring.html). + +#### Telegraf: + +The Telegraf Prometheus input plugin needs to scrape the Ceph MGR(s) Prometheus Metrics Endpoint(s). + +Telegraf configuration requires the following environment variables + - `INFLUX_HOST` + - `INFLUX_BUCKET` + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization + - `CEPH_MGR_SVC_URLS` - URLs to Ceph Manager metrics endpoint service(s) e.g. `http://rook-ceph-mgr.rook-ceph.svc.cluster.local:9283/metrics`. + +Any configuration changes reflecting your specific Kubernetes or Ceph installation can be set in the Telegraf configuration manually. + +You **MUST** set these environment variables before running Telegraf using something similar to the following commands + - This can be found on the `Load Data` > `Tokens` page in your browser: `export INFLUX_TOKEN=TOKEN` + - Your Organization name can be found on the Settings page in your browser: `export INFLUX_ORG=my_org` + +## Contact + +Author: Tomas Klapka, Bonitoo s.r.o. + +Github: Bonitoo.io \ No newline at end of file diff --git a/InfluxDBv2_Covid19_SouthAmerica/argentina.sh b/covid19-southamerica/argentina.sh similarity index 100% rename from InfluxDBv2_Covid19_SouthAmerica/argentina.sh rename to covid19-southamerica/argentina.sh diff --git a/InfluxDBv2_Covid19_SouthAmerica/bolivia.sh b/covid19-southamerica/bolivia.sh similarity index 100% rename from InfluxDBv2_Covid19_SouthAmerica/bolivia.sh rename to covid19-southamerica/bolivia.sh diff --git a/InfluxDBv2_Covid19_SouthAmerica/brasil.sh b/covid19-southamerica/brasil.sh similarity index 100% rename from InfluxDBv2_Covid19_SouthAmerica/brasil.sh rename to covid19-southamerica/brasil.sh diff --git a/InfluxDBv2_Covid19_SouthAmerica/chile.sh b/covid19-southamerica/chile.sh similarity index 100% rename from InfluxDBv2_Covid19_SouthAmerica/chile.sh rename to covid19-southamerica/chile.sh diff --git a/InfluxDBv2_Covid19_SouthAmerica/covid.yml b/covid19-southamerica/covid.yml old mode 100755 new mode 100644 similarity index 100% rename from InfluxDBv2_Covid19_SouthAmerica/covid.yml rename to covid19-southamerica/covid.yml diff --git a/InfluxDBv2_Covid19_SouthAmerica/paraguay.sh b/covid19-southamerica/paraguay.sh similarity index 100% rename from InfluxDBv2_Covid19_SouthAmerica/paraguay.sh rename to covid19-southamerica/paraguay.sh diff --git a/InfluxDBv2_Covid19_SouthAmerica/readme.md b/covid19-southamerica/readme.md similarity index 70% rename from InfluxDBv2_Covid19_SouthAmerica/readme.md rename to covid19-southamerica/readme.md index 5d00cfaa..e06febbe 100644 --- a/InfluxDBv2_Covid19_SouthAmerica/readme.md +++ b/covid19-southamerica/readme.md @@ -10,7 +10,7 @@ This Dashboard graph information about COVID-19 focused in Argentina, Bolivia, B #### InfluxDB UI -In the InfluxDB UI, go to Settings->Templates and enter this URL: https://raw.githubusercontent.com/influxdata/community-templates/master/InfluxDBv2_Covid19_SouthAmerica/covid.yml +In the InfluxDB UI, go to Settings->Templates and enter this URL: https://raw.githubusercontent.com/influxdata/community-templates/master/covid19-southamerica/covid.yml #### Influx CLI If you have your InfluxDB credentials [configured in the CLI](https://v2.docs.influxdata.com/v2.0/reference/cli/influx/config/), you can install this template with: @@ -21,20 +21,20 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Bucket: 'covid' - - 5 .sh files that get the information from the API and convert in JSON. - - 1 Dashboards: covid.yml - COVID19 América del Sur y Resto del mundo - - 1 Telegraf: 'Configuration: Exec Inputs' - - 1 Label: 'covid' + - 1 Bucket: 'covid' + - 5 .sh files that get the information from the API and convert in JSON. + - 1 Dashboards: covid.yml - COVID19 América del Sur y Resto del mundo + - 1 Telegraf: 'Configuration: Exec Inputs' + - 1 Label: 'covid' ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_HOST` - The host running InfluxDB - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. +Telegraf Configuration requires the following environment variables + - `INFLUX_HOST` - The host running InfluxDB + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. As the bucket was provided in this template, you need to set the following environment data ```export INFLUX_BUCKET=covid```. Also may requiere adjust the information about the location of the executables files and set the agent interval to at least 1h, this for not hit to the api too much and cause service disruption. diff --git a/InfluxDBv2_Covid19_SouthAmerica/screenshot.png b/covid19-southamerica/screenshot.png similarity index 100% rename from InfluxDBv2_Covid19_SouthAmerica/screenshot.png rename to covid19-southamerica/screenshot.png diff --git a/InfluxDBv2_Covid19_SouthAmerica/uruguay.sh b/covid19-southamerica/uruguay.sh similarity index 100% rename from InfluxDBv2_Covid19_SouthAmerica/uruguay.sh rename to covid19-southamerica/uruguay.sh diff --git a/InfluxDBv2_Covid19_SouthAmerica/worldwide.sh b/covid19-southamerica/worldwide.sh similarity index 100% rename from InfluxDBv2_Covid19_SouthAmerica/worldwide.sh rename to covid19-southamerica/worldwide.sh diff --git a/criblio_logstream/README.md b/criblio_logstream/README.md new file mode 100644 index 00000000..bcc28c44 --- /dev/null +++ b/criblio_logstream/README.md @@ -0,0 +1,55 @@ +# Cribl LogStream +Provided by [Cribl](https://cribl.io) + +This InfluxDB Template can be used to see the metrics created by your Cribl LogStream deployment. + +![Screenshot](img/cribl_logstream_influxdb.png) + +## Quick Install + +### InfluxDB UI +In the InfluxDB UI, go to Settings -> Templates and enter this URL: https://raw.githubusercontent.com/influxdb/community-templates/master/criblio_logstream/cribl_logstream.yml + +### Influx CLI + +If you have your InfluxDB credentials conrigured in the CLI, you can isntall this template with: + +`influx apply -f https://raw.githubusercontent.com/influxdb/community-templates/master/criblio_logstream/cribl_logstream.yml` + +## Included Resources +- 1 Dashboard: Cribl LogStream +- 1 bucket: 'criblBucket' + +## Setup Instructions + +General instructions on using InfluxDB [Templates](https://github.com/influxdata/community-templates/blob/master/docs/use_a_template.md) can be found in the use a template document. + +Log Into your Cribl LogStream master node and configure your InfluxDB Destination: + +![Screenshot](img/influx_db_cribl_destination_01.png) + +- Set environemnt variables on for the InfluxDB Destination: + +![Screenshot InfluxDB](img/influx_db_cribl_destination_02.png) + +- `Write API URL` - Your InfluxDB Cloud Endpoint, here's an example: `https://us-west-2-1.aws.cloud2.influxdata.com/api/v2/write?org=&bucket=criblLogstream&precision=ms` +- `Database Name` - Your InfluxDB Name: `criblBucket` +- `Extra HTTP Headers` - Navigate to Advanced Settings and add the `Authorization` Header with your `Token` as the value. + +![Screenshot InfluxDB](img/influx_db_cribl_destination_03.png) + +## Pre-Work +To start sending Cribl metrics with the template-provided configuration, you need to setup the following: +Click on Sources and select the Cribl Internal Icon: + +![Screenshot](img/cribl_internal_sources_01.png) + + +Once installed, you'll need to setup Cribl LogStream to send the internal metrics to your influx DB deployment. Check out the Cribl LogStream docs on how to send your [Cribl Internal metrics](https://docs.cribl.io/docs/sources-cribl-internal) to the [InfluxDB destination](https://docs.cribl.io/docs/destinations-influxdb). + + +## Contact +- Author: Clint Sharp +- Email: clint@cribl.io +- Github: criblio +- Influx Slack: @Clint Sharp diff --git a/criblio_logstream/cribl_logstream.yml b/criblio_logstream/cribl_logstream.yml new file mode 100755 index 00000000..40904035 --- /dev/null +++ b/criblio_logstream/cribl_logstream.yml @@ -0,0 +1,212 @@ +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: obstinate-moore-59a001 +spec: + color: '#00C9FF' + name: cribl +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: intelligent-hypatia-99a001 +spec: + associations: + - kind: Label + name: obstinate-moore-59a001 + language: flux + name: criblBucket + query: |- + buckets() + |> filter(fn: (r) => r.name !~ /^_/) + |> rename(columns: {name: "_value"}) + |> keep(columns: ["_value"]) + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: earning-williamson-99a001 +spec: + associations: + - kind: Label + name: obstinate-moore-59a001 + charts: + - height: 4 + kind: Markdown + name: Name this Cell + note: '# Cluster Metrics' + width: 2 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#FD7A5D' + name: Delorean + type: scale + - hex: '#5F1CF2' + name: Delorean + type: scale + - hex: '#4CE09A' + name: Delorean + type: scale + geom: line + height: 3 + kind: Xy + name: Sources (events) + position: overlaid + queries: + - query: |- + from(bucket: v.criblBucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "cribl.logstream.total") + |> filter(fn: (r) => r["_field"] == "in_events") + |> filter(fn: (r) => exists r["input"] and r["input"] != "cribl:CriblMetrics") + |> group(columns: ["input"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum) + |> fill(usePrevious: true) + |> yield(name: "sum") + width: 12 + yPos: 4 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#DA6FF1' + name: Ectoplasm + type: scale + - hex: '#00717A' + name: Ectoplasm + type: scale + - hex: '#ACFF76' + name: Ectoplasm + type: scale + geom: line + height: 3 + kind: Xy + name: Destinations (events) + position: overlaid + queries: + - query: |- + from(bucket: v.criblBucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "cribl.logstream.total") + |> filter(fn: (r) => r["_field"] == "out_events") + |> filter(fn: (r) => exists r["output"]) + |> group(columns: ["output"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum) + |> fill(usePrevious: true) + |> yield(name: "sum") + width: 12 + yPos: 7 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#31C0F6' + name: Nineteen Eighty Four + type: scale + - hex: '#A500A5' + name: Nineteen Eighty Four + type: scale + - hex: '#FF7E27' + name: Nineteen Eighty Four + type: scale + geom: line + height: 3 + kind: Xy + name: Routes + position: overlaid + queries: + - query: |- + from(bucket: v.criblBucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "cribl.logstream.pipe") + |> filter(fn: (r) => r["_field"] == "in_events" or r["_field"] == "out_events") + |> group(columns: ["id"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum) + |> fill(usePrevious: true) + |> yield(name: "sum") + width: 12 + yPos: 10 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#31C0F6' + name: Nineteen Eighty Four + type: scale + - hex: '#A500A5' + name: Nineteen Eighty Four + type: scale + - hex: '#FF7E27' + name: Nineteen Eighty Four + type: scale + geom: line + height: 4 + kind: Xy + name: Events In/Out + position: overlaid + queries: + - query: |- + from(bucket: v.criblBucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "cribl.logstream.total") + |> filter(fn: (r) => r["_field"] == "in_events" or r["_field"] == "out_events") + |> group(columns:["_field"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum) + |> fill(usePrevious: true) + |> yield(name: "sum") + width: 5 + xPos: 2 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#74D495' + name: Atlantis + type: scale + - hex: '#3F3FBA' + name: Atlantis + type: scale + - hex: '#FF4D9E' + name: Atlantis + type: scale + geom: line + height: 4 + kind: Xy + name: Bytes In/Out + position: overlaid + queries: + - query: |- + from(bucket: v.criblBucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "cribl.logstream.total") + |> filter(fn: (r) => r["_field"] == "in_bytes" or r["_field"] == "out_bytes") + |> group(columns:["_field"]) + |> aggregateWindow(every: v.windowPeriod, fn: sum) + |> fill(usePrevious: true) + |> yield(name: "sum") + width: 5 + xPos: 7 + name: Cribl LogStream diff --git a/criblio_logstream/img/cribl_internal_sources_01.png b/criblio_logstream/img/cribl_internal_sources_01.png new file mode 100644 index 00000000..8131987f Binary files /dev/null and b/criblio_logstream/img/cribl_internal_sources_01.png differ diff --git a/criblio_logstream/img/cribl_logstream_influxdb.png b/criblio_logstream/img/cribl_logstream_influxdb.png new file mode 100644 index 00000000..d180831b Binary files /dev/null and b/criblio_logstream/img/cribl_logstream_influxdb.png differ diff --git a/criblio_logstream/img/influx_db_cribl_destination_01.png b/criblio_logstream/img/influx_db_cribl_destination_01.png new file mode 100644 index 00000000..2761d950 Binary files /dev/null and b/criblio_logstream/img/influx_db_cribl_destination_01.png differ diff --git a/criblio_logstream/img/influx_db_cribl_destination_02.png b/criblio_logstream/img/influx_db_cribl_destination_02.png new file mode 100644 index 00000000..a9657ab9 Binary files /dev/null and b/criblio_logstream/img/influx_db_cribl_destination_02.png differ diff --git a/criblio_logstream/img/influx_db_cribl_destination_03.png b/criblio_logstream/img/influx_db_cribl_destination_03.png new file mode 100644 index 00000000..fcc3aca0 Binary files /dev/null and b/criblio_logstream/img/influx_db_cribl_destination_03.png differ diff --git a/csgo/readme.md b/csgo/readme.md index e026c25e..721f42b4 100644 --- a/csgo/readme.md +++ b/csgo/readme.md @@ -21,21 +21,21 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf Configuration: 'csgo-telegraf' - - 1 Dashboards - - 1 Label: 'csgo' - - 1 Bucket: 'csgo' - - 1 Executable: 'csgo.sh' + - 1 Telegraf Configuration: 'csgo-telegraf' + - 1 Dashboards + - 1 Label: 'csgo' + - 1 Bucket: 'csgo' + - 1 Executable: 'csgo.sh' ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. - - `INFLUX_HOST` - The address of you InfluxDB - - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```INFLUX_BUCKET=mssql``` +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The address of you InfluxDB + - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```INFLUX_BUCKET=mssql``` In order to use this dashboard you need to get an API key and your Steam User ID: diff --git a/currency_exchange_rates/currency_exchange_rates.yml b/currency_exchange_rates/currency_exchange_rates.yml index a4e78021..90378e1c 100644 --- a/currency_exchange_rates/currency_exchange_rates.yml +++ b/currency_exchange_rates/currency_exchange_rates.yml @@ -82,7 +82,7 @@ spec: ## Multiple URLs can be specified for a single cluster, only ONE of the ## urls will be written to each interval. ## ex: urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"] - urls = ["$INFLUX_URL"] + urls = ["$INFLUX_HOST"] ## Token for authentication. token = "$INFLUX_TOKEN" ## Organization is the name of the organization you wish to write to; must exist. diff --git a/do_billing/balance.sh b/do_billing/balance.sh new file mode 100644 index 00000000..7b586128 --- /dev/null +++ b/do_billing/balance.sh @@ -0,0 +1,5 @@ +#!/bin/bash + +curl -s -X GET -H "Content-Type: application/json" -H "Authorization: Bearer $DO_TOKEN" "https://api.digitalocean.com/v2/customers/my/balance" + + diff --git a/do_billing/dobilling.yml b/do_billing/dobilling.yml new file mode 100644 index 00000000..33f4c638 --- /dev/null +++ b/do_billing/dobilling.yml @@ -0,0 +1,232 @@ +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: agitated-ptolemy-010001 +spec: + color: '#066fc5' + name: digitalocean +--- +apiVersion: influxdata.com/v2alpha1 +kind: Bucket +metadata: + name: mystifying-jang-010005 +spec: + associations: + - kind: Label + name: agitated-ptolemy-010001 + name: do_billing +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: sleepy-black-810000 +spec: + associations: + - kind: Label + name: agitated-ptolemy-010001 + charts: + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Last Update + queries: + - query: |- + from(bucket: "do_billing") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "exec_do_billing") + |> filter(fn: (r) => r["_field"] == "generated_at") + width: 2 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Current Balance + note: "Negative values is when you have credit in your account. \nPositive + Balance is when you have debs in your account." + prefix: '$ ' + queries: + - query: |- + from(bucket: "do_billing") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "exec_do_billing") + |> filter(fn: (r) => r["_field"] == "account_balance") + |> toFloat() + |> last() + width: 2 + yPos: 1 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Month to Date Consumption + prefix: '$ ' + queries: + - query: |- + from(bucket: "do_billing") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "exec_do_billing") + |> filter(fn: (r) => r["_field"] == "month_to_date_usage") + |> toFloat() + |> last() + width: 2 + yPos: 2 + - axes: + - base: "10" + name: x + scale: linear + - base: "2" + label: Cost + name: y + prefix: '$ ' + scale: linear + colors: + - hex: '#FD7A5D' + name: Delorean + type: scale + - hex: '#5F1CF2' + name: Delorean + type: scale + - hex: '#4CE09A' + name: Delorean + type: scale + geom: monotoneX + height: 3 + kind: Xy + name: Month to Date Consumption (Last 30 days) + note: This metrics reset every first of the month. + position: stacked + queries: + - query: |- + from(bucket: "do_billing") + |> range(start: -30d) + |> filter(fn: (r) => r["_measurement"] == "exec_do_billing") + |> filter(fn: (r) => r["_field"] == "month_to_date_usage") + |> toFloat() + shade: true + width: 4 + xCol: _time + xPos: 2 + yCol: _value + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 2 + height: 3 + kind: Single_Stat + name: Month to Date Balance (To Pay) + note: Negative values is when you have credit in your account. Positive Balance + is when you have debs in your account. + prefix: '$ ' + queries: + - query: |- + from(bucket: "do_billing") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "exec_do_billing") + |> filter(fn: (r) => r["_field"] == "month_to_date_balance") + |> toFloat() + width: 2 + xPos: 6 + - axes: + - base: "10" + name: x + scale: linear + - base: "2" + name: y + prefix: '$ ' + scale: linear + colors: + - hex: '#74D495' + name: Atlantis + type: scale + - hex: '#3F3FBA' + name: Atlantis + type: scale + - hex: '#FF4D9E' + name: Atlantis + type: scale + geom: monotoneX + height: 3 + kind: Xy + name: Month to Date Balance (Last 10 days) + note: Negative values is when you have credit in your account. Positive Balance + is when you have debs in your account. + position: overlaid + queries: + - query: |- + from(bucket: "do_billing") + |> range(start: -10d) + |> filter(fn: (r) => r["_measurement"] == "exec_do_billing") + |> filter(fn: (r) => r["_field"] == "month_to_date_balance") + |> toFloat() + shade: true + width: 4 + xCol: _time + xPos: 8 + yCol: _value + name: DigitalOcean Billing + +--- + +apiVersion: influxdata.com/v2alpha1 +kind: Telegraf +metadata: + name: do-billing +spec: + associations: + - kind: Label + name: agitated-ptolemy-010001 + config: | + [[outputs.influxdb_v2]] + ## The URLs of the InfluxDB cluster nodes. + ## + ## Multiple URLs can be specified for a single cluster, only ONE of the + ## urls will be written to each interval. + ## urls exp: http://127.0.0.1:8086 + urls = ["$INFLUX_HOST"] + + ## Token for authentication. + token = "$INFLUX_TOKEN" + + ## Organization is the name of the organization you wish to write to; must exist. + organization = "$INFLUX_ORG" + + ## Destination bucket to write into. + bucket = "do_billing" + + [agent] + interval = "24h" + + [[inputs.exec]] + ## Commands array + commands = [ + "sh $PWD/balance.sh" + ] + + ## String fields is an array of keys that should be added as string fields. + json_string_fields = ["month_to_date_balance","account_balance","month_to_date_usage","generated_at","amount","invoice_period"] + + ## Tag keys is an array of keys that should be added as tags. + tag_keys = [] + + ## Timeout for each command to complete. + timeout = "8s" + + ## measurement name suffix (for separating different commands) + name_suffix = "_do_billing" + + ## Data format to consume. + ## Each data format has its own unique set of configuration options, read + ## more about them here: + ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md + data_format = "json" \ No newline at end of file diff --git a/do_billing/readme.md b/do_billing/readme.md new file mode 100644 index 00000000..25fb5d18 --- /dev/null +++ b/do_billing/readme.md @@ -0,0 +1,51 @@ +# DigitalOcean Billing Dashboard for InfluxDB v2 + +Provided by: Ignacio Van Droogenbroeck + +This Dashboard offers you information your balance, consumption to date and balance to date of your DigitalOcean Account. + +![Dashboard Screenshot](screenshot.png) + +### Quick Install + +#### InfluxDB UI + +In the InfluxDB UI, go to Settings->Templates and enter this URL: https://raw.githubusercontent.com/influxdata/community-templates/master/do_billing/dobilling.yml + +#### Influx CLI +If you have your InfluxDB credentials [configured in the CLI](https://v2.docs.influxdata.com/v2.0/reference/cli/influx/config/), you can install this template with: + +``` +influx apply -u https://raw.githubusercontent.com/influxdata/community-templates/master/do_billing/dobilling.yml +``` + +## Included Resources + + - 1 Telegraf Configuration: 'dobilling' + - 1 Dashboards: 'DigitalOcean Billing' + - 1 Label: 'digitalocean' + - 1 Bucket: 'do_billing' + - 1 sh file: balance.sh + +## Setup Instructions + +General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. + +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The address of you InfluxDB + +To use this dashboard, you need a DigitalOcean Token. Read Only is sufficient and you can get one [from here](https://cloud.digitalocean.com/account/api/tokens) and pass as to environment variable. In the balance.sh file, the variable expected is $DO_TOKEN. + +The Telegraf config has set up a path to the balance.sh file the working directory from where Telegraf is running. + +## Contact + +Author: Ignacio Van Droogenbroeck + +Email: ignacio[at]vandroogenbroeck[dot]net + +Github and Gitlab user: @xe-nvdk + +Influx Slack: Ignacio Van Droogenbroeck diff --git a/do_billing/screenshot.png b/do_billing/screenshot.png new file mode 100644 index 00000000..d045d2db Binary files /dev/null and b/do_billing/screenshot.png differ diff --git a/docker/docker.yml b/docker/docker.yml index aaf90030..510cb84a 100644 --- a/docker/docker.yml +++ b/docker/docker.yml @@ -97,13 +97,6 @@ spec: type: expire --- apiVersion: influxdata.com/v2alpha1 -kind: Bucket -metadata: - name: relaxed-turing-d4d019 -spec: - name: dave ---- -apiVersion: influxdata.com/v2alpha1 kind: CheckThreshold metadata: name: crumbling-meitner-14d002 @@ -237,20 +230,6 @@ spec: previousLevel: OK --- apiVersion: influxdata.com/v2alpha1 -kind: Variable -metadata: - name: tender-mayer-14d00a -spec: - language: flux - name: bucket - query: |- - buckets() - |> filter(fn: (r) => r.name !~ /^_/) - |> rename(columns: {name: "_value"}) - |> keep(columns: ["_value"]) - type: query ---- -apiVersion: influxdata.com/v2alpha1 kind: Dashboard metadata: name: earning-boyd-94d001 @@ -329,7 +308,7 @@ spec: name: Swap queries: - query: |- - from(bucket: v.bucket) + from(bucket: "docker") |> range(start: v.timeRangeStart) |> filter(fn: (r) => r._measurement == "swap") |> filter(fn: (r) => r._field == "total" or r._field == "used") @@ -355,13 +334,16 @@ spec: kind: Xy name: Network TX trafic per container / sec queries: - - query: "from(bucket: v.bucket)\n |> range(start: v.timeRangeStart, stop:v.timeRangeStop)\n - \ |> filter(fn: (r) => r._measurement == \"docker_container_net\")\n - \ |> filter(fn: (r) => r._field == \"tx_bytes\") \n |> derivative(unit:1s, - nonNegative: false)\n |> window(period: v.windowPeriod)\n |> mean()\n - \ |> keep(columns: [\"_measurement\",\"container_name\", \"host\",\"_value\",\"_field\",\"_stop\"])\n - \ |> group(columns: [\"_value\", \"_time\", \"_start\", \"_stop\"],mode: - \"except\")" + - query: |- + from(bucket: "docker") + |> range(start: v.timeRangeStart, stop:v.timeRangeStop) + |> filter(fn: (r) => r._measurement == "docker_container_net") + |> filter(fn: (r) => r._field == "tx_bytes") + |> derivative(unit:1s, nonNegative: false) + |> window(period: v.windowPeriod) + |> mean() + |> keep(columns: ["_measurement","container_name", "host","_value","_field","_stop"]) + |> group(columns: ["_value", "_time", "_start", "_stop"],mode: "except") width: 6 xCol: _stop yCol: _value @@ -389,7 +371,7 @@ spec: name: Disk Usage queries: - query: |- - from(bucket: v.bucket) + from(bucket: "docker") |> range(start: v.timeRangeStart) |> filter(fn: (r) => r._measurement == "disk") |> filter(fn: (r) => r._field == "used_percent") @@ -426,7 +408,7 @@ spec: name: System Load queries: - query: |- - from(bucket: v.bucket) + from(bucket: "docker") |> range(start: v.timeRangeStart) |> filter(fn: (r) => r._measurement == "system") |> filter(fn: (r) => r._field == "load1") @@ -454,7 +436,7 @@ spec: name: System Load queries: - query: |- - from(bucket: v.bucket) + from(bucket: "docker") |> range(start: v.timeRangeStart) |> filter(fn: (r) => r._measurement == "system") |> filter(fn: (r) => r._field == "load1" or r._field == "load5" or r._field == "load15") @@ -478,7 +460,7 @@ spec: name: Memory usage % per container queries: - query: |- - from(bucket: v.bucket) + from(bucket: "docker") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r._measurement == "docker_container_mem") |> filter(fn: (r) => r._field == "usage_percent") @@ -518,7 +500,7 @@ spec: name: Docker Daemon nCPUs queries: - query: |- - from(bucket: v.bucket) + from(bucket: "docker") |> range(start: v.timeRangeStart) |> filter(fn: (r) => r._measurement == "docker") |> filter(fn: (r) => r._field == "n_cpus") @@ -547,7 +529,7 @@ spec: name: CPU Usage queries: - query: |- - from(bucket: v.bucket) + from(bucket: "docker") |> range(start: v.timeRangeStart) |> filter(fn: (r) => r._measurement == "cpu") |> filter(fn: (r) => r._field == "usage_user" or r._field == "usage_system" or r._field == "usage_idle") @@ -586,7 +568,7 @@ spec: name: Network RX trafic per container / sec queries: - query: |- - from(bucket: v.bucket) + from(bucket: "docker") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r._measurement == "docker_container_net") |> filter(fn: (r) => r._field == "rx_bytes" ) @@ -613,7 +595,7 @@ spec: name: Memory usage per container queries: - query: |- - from(bucket: v.bucket) + from(bucket: "docker") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r._measurement == "docker_container_mem") |> filter(fn: (r) => r._field == "usage") @@ -635,7 +617,7 @@ spec: name: Docker Daemon Total Mem queries: - query: |- - from(bucket: v.bucket) + from(bucket: "docker") |> range(start: v.timeRangeStart) |> filter(fn: (r) => r._measurement == "docker") |> filter(fn: (r) => r._field == "memory_total") @@ -678,7 +660,7 @@ spec: name: System Memory Usage queries: - query: |- - from(bucket: v.bucket) + from(bucket: "docker") |> range(start: v.timeRangeStart) |> filter(fn: (r) => r._measurement == "mem") |> filter(fn: (r) => r._field == "used_percent") @@ -705,7 +687,7 @@ spec: name: CPU usage per container queries: - query: |- - from(bucket: v.bucket) + from(bucket: "docker") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r._measurement == "cpu" or r._measurement == "docker_container_cpu") |> filter(fn: (r) => r._field == "usage_percent") @@ -778,7 +760,7 @@ spec: # Configuration for sending metrics to InfluxDB [[outputs.influxdb_v2]] - urls = ["$INFLUX_URL"] + urls = ["$INFLUX_HOST"] token = "$INFLUX_TOKEN" organization = "$INFLUX_ORG" ## Destination bucket to write into. bucket = "docker" diff --git a/docs/best_practices.md b/docs/best_practices.md index 225ac92e..96758bea 100644 --- a/docs/best_practices.md +++ b/docs/best_practices.md @@ -18,9 +18,9 @@ Rather than duplicating graphs to show filtered views of the same data, use [Var When using Buckets in your Template, you can: - - Provide a default Bucket with your Template - - Reuse an existing Bucket by name - - Let the user choose their Bucket + - Provide a default Bucket with your Template + - Reuse an existing Bucket by name + - Let the user choose their Bucket #### Provide a default Bucket with your Template You can include a uniquely-named [Bucket](https://v2.docs.influxdata.com/v2.0/reference/key-concepts/data-elements/#bucket) with your Template and use it as the default in Dashboards and Telegraf configurations. @@ -135,16 +135,16 @@ To add a custom Telegraf configuration file to your template: 1. Export your template using the command in the [submitting a template](submit_a_template.md) doc. 2. Add the following to the end of the file, being sure to give a name for your Telegraf configuration: - ``` - --- - apiVersion: influxdata.com/v2alpha1 - kind: Telegraf - metadata: - name: unique-name-for-your-config - spec: - name: The Name Of Your Configuration - config: | - ``` +``` +--- +apiVersion: influxdata.com/v2alpha1 +kind: Telegraf +metadata: + name: unique-name-for-your-config +spec: + name: The Name Of Your Configuration + config: | +``` 3. Copy and paste the contents of your Telegraf configuration file below what you just added, indenting it all until it is 4 spaces further indented than the `config:` line. @@ -156,8 +156,7 @@ After you export your finished Template, test it to make sure that it applies cl To create a clean, ephemeral InfluxDB testing environment, run InfluxDB inside of a Docker container: ``` - docker run -p 8086:8086 -p 9999:9999 quay.io/influxdb/influxdb:2.0.0-beta - +docker run -p 8086:8086 -p 9999:9999 quay.io/influxdb/influxdb:2.0.0-beta ``` After you create a new `Organization` and `Token` in your Docker instance (available at http://localhost:9999), follow the [instructions for using a Template](./use_a_template.md) to apply your Template. diff --git a/docs/submit_a_template.md b/docs/submit_a_template.md index ff94da1e..924a5035 100644 --- a/docs/submit_a_template.md +++ b/docs/submit_a_template.md @@ -27,7 +27,7 @@ To contribute a new template or enhance an existing template, submit a pull requ * To update an existing template, make the changes to template files in the appropriate directory. - > **Tip:** Replace any hard-coded URLs to InfluxDB in your Telegraf configurations with the `$INFLUX_URL` environment variable so users can easily point it to their own InfluxDB instance location. For example: `urls = ["$INFLUX_URL"]` + > **Tip:** Replace any hard-coded URLs to InfluxDB in your Telegraf configurations with the `$INFLUX_HOST` environment variable so users can easily point it to their own InfluxDB instance location. For example: `urls = ["$INFLUX_HOST"]` 3. If you are submitting a new Template, add it to the table of Templates in the main `README.md` file in the root of the repository. diff --git a/endpoint-security-state/readme.md b/endpoint-security-state/readme.md index 5cc425ee..ba210b52 100644 --- a/endpoint-security-state/readme.md +++ b/endpoint-security-state/readme.md @@ -28,10 +28,10 @@ influx apply --template-url https://raw.githubusercontent.com/influxdata/communi - 4 Labels: `outputs.influxdb_v2`, `Telegraf`, `Solution`, `security` - 1 Telegraf Configuration - 4 Checks: - - Endpoint availablility - - x509 Certificate status - - Authentication is enabled - - Authentication is working + - Endpoint availablility + - x509 Certificate status + - Authentication is enabled + - Authentication is working - 1 Dashboard: `Endpoint Security State` - 1 Variable: `bucket` diff --git a/fail2ban/fail2ban-template.yml b/fail2ban/fail2ban-template.yml new file mode 100755 index 00000000..d6a90968 --- /dev/null +++ b/fail2ban/fail2ban-template.yml @@ -0,0 +1,467 @@ +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: amusing-pasteur-94d001 +spec: + color: '#67d74e' + name: fail2ban +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: gracious-montalcini-94d009 +spec: + associations: + - kind: Label + name: amusing-pasteur-94d001 + language: flux + name: jail + query: |- + import "influxdata/influxdb/v1" + + all_data = buckets() |> limit(n:1) |> set(key: "_value", value: "*") |> keep(columns:["_value"]) + jails = v1.measurementTagValues(bucket: v.bucket, measurement: "fail2ban", tag: "jail") + + union(tables: [all_data, jails]) |> sort() + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: kind-shtern-94d00d +spec: + associations: + - kind: Label + name: amusing-pasteur-94d001 + language: flux + name: host + query: |- + import "influxdata/influxdb/v1" + + all_data = buckets() |> limit(n:1) |> set(key: "_value", value: "*") |> keep(columns:["_value"]) + hosts = v1.measurementTagValues(bucket: v.bucket, measurement: "fail2ban", tag: "host") + + union(tables: [all_data, hosts]) |> sort() + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: rightful-wing-94d005 +spec: + associations: + - kind: Label + name: amusing-pasteur-94d001 + language: flux + name: bucket + query: |- + buckets() + |> filter(fn: (r) => r.name !~ /^_/) + |> rename(columns: {name: "_value"}) + |> keep(columns: ["_value"]) + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: epic-perlman-94d001 +spec: + associations: + - kind: Label + name: amusing-pasteur-94d001 + charts: + - height: 1 + kind: Markdown + name: Name this Cell + note: This dashboard gives you an overview of metrics from `fail2ban` `system`, + `mem`, `diskio`, and `net` measurements. See the [Fail2Ban Template Documentation](https://github.com/influxdata/community-templates/tree/master/fail2ban) + for help configuring these plugins. + width: 12 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#31C0F6' + name: Nineteen Eighty Four + type: scale + - hex: '#A500A5' + name: Nineteen Eighty Four + type: scale + - hex: '#FF7E27' + name: Nineteen Eighty Four + type: scale + geom: line + height: 3 + kind: Xy + name: Failed + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "fail2ban") + |> filter(fn: (r) => r["_field"] == "failed") + |> filter(fn: (r) => (if v.host == "*" then true else r["host"] == v.host)) + |> filter(fn: (r) => (if v.jail == "*" then true else r["jail"] == v.jail)) + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 6 + xCol: _time + yCol: _value + yPos: 1 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + suffix: '%' + - base: "10" + name: y2 + scale: linear + geom: line + height: 4 + kind: Xy + name: Disk Usage + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart) + |> filter(fn: (r) => r._measurement == "disk") + |> filter(fn: (r) => (if v.host == "*" then true else r["host"] == v.host)) + |> filter(fn: (r) => r._field == "used_percent") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 4 + yPos: 4 + - axes: + - base: "10" + name: x + scale: linear + - base: "2" + label: Bytes + name: y + scale: linear + - base: "10" + name: y2 + scale: linear + geom: line + height: 3 + kind: Xy + name: Network + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart) + |> filter(fn: (r) => r._measurement == "net") + |> filter(fn: (r) => (if v.host == "*" then true else r["host"] == v.host)) + |> filter(fn: (r) => r._field == "bytes_recv" or r._field == "bytes_sent") + |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false) + |> derivative(unit: 1s, nonNegative: false) + |> yield(name: "derivative") + width: 6 + xCol: _time + yCol: _value + yPos: 8 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: nCPUs + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart) + |> filter(fn: (r) => (if v.host == "*" then true else r["host"] == v.host)) + |> filter(fn: (r) => r._measurement == "system") + |> filter(fn: (r) => r._field == "n_cpus") + |> last() + |> yield(name: "last") + suffix: ' cpus' + width: 2 + xPos: 4 + yPos: 4 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Total Memory + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart) + |> filter(fn: (r) => r._measurement == "mem") + |> filter(fn: (r) => (if v.host == "*" then true else r["host"] == v.host)) + |> filter(fn: (r) => r._field == "total") + |> last() + |> map(fn: (r) => ({r with _value: float(v: r._value) / 1024.0 / 1024.0 / 1024.0})) + |> yield(name: "last") + suffix: ' GB' + width: 2 + xPos: 4 + yPos: 5 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 2 + height: 2 + kind: Single_Stat + name: System Load + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart) + |> filter(fn: (r) => r._measurement == "system") + |> filter(fn: (r) => (if v.host == "*" then true else r["host"] == v.host)) + |> filter(fn: (r) => r._field == "load1") + |> last() + |> yield(name: "mean") + width: 2 + xPos: 4 + yPos: 6 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#31C0F6' + name: Nineteen Eighty Four + type: scale + - hex: '#A500A5' + name: Nineteen Eighty Four + type: scale + - hex: '#FF7E27' + name: Nineteen Eighty Four + type: scale + geom: line + height: 3 + kind: Xy + name: Banned IPs + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "fail2ban") + |> filter(fn: (r) => r["_field"] == "banned") + |> filter(fn: (r) => (if v.host == "*" then true else r["host"] == v.host)) + |> filter(fn: (r) => (if v.jail == "*" then true else r["jail"] == v.jail)) + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 6 + xCol: _time + xPos: 6 + yCol: _value + yPos: 1 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + suffix: '%' + - base: "10" + name: y2 + scale: linear + geom: line + height: 3 + kind: Xy + name: CPU Usage + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart) + |> filter(fn: (r) => r._measurement == "cpu") + |> filter(fn: (r) => (if v.host == "*" then true else r["host"] == v.host)) + |> filter(fn: (r) => r._field == "usage_user" or r._field == "usage_system" or r._field == "usage_idle") + |> filter(fn: (r) => r.cpu == "cpu-total") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 3 + xCol: _time + xPos: 6 + yCol: _value + yPos: 4 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + - base: "10" + name: y2 + scale: linear + geom: line + height: 4 + kind: Xy + name: Processes + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart) + |> filter(fn: (r) => r._measurement == "processes") + |> filter(fn: (r) => (if v.host == "*" then true else r["host"] == v.host)) + |> filter(fn: (r) => r._field == "running" or r._field == "blocked" or r._field == "idle" or r._field == "unknown") + |> aggregateWindow(every: v.windowPeriod, fn: max) + |> yield(name: "max") + width: 3 + xCol: _time + xPos: 6 + yCol: _value + yPos: 7 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + label: Load + name: y + scale: linear + - base: "10" + name: y2 + scale: linear + geom: line + height: 3 + kind: Xy + name: System Load + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart) + |> filter(fn: (r) => r._measurement == "system") + |> filter(fn: (r) => (if v.host == "*" then true else r["host"] == v.host)) + |> filter(fn: (r) => r._field == "load1" or r._field == "load5" or r._field == "load15") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 3 + xCol: _time + xPos: 9 + yCol: _value + yPos: 4 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + suffix: '%' + - base: "10" + name: y2 + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + - hex: '#8F8AF4' + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#A51414' + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#F4CF31' + name: Do Androids Dream of Electric Sheep? + type: scale + decimalPlaces: 1 + height: 4 + kind: Single_Stat_Plus_Line + name: Memory Usage + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart) + |> filter(fn: (r) => r._measurement == "mem") + |> filter(fn: (r) => (if v.host == "*" then true else r["host"] == v.host)) + |> filter(fn: (r) => r._field == "used_percent") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + suffix: '%' + width: 3 + xCol: _time + xPos: 9 + yCol: _value + yPos: 7 + description: A collection of useful visualizations for monitoring your system + stats + name: Fail2Ban +--- +apiVersion: influxdata.com/v2alpha1 +kind: Telegraf +metadata: + name: intelligent-kalam-94d005 +spec: + associations: + - kind: Label + name: amusing-pasteur-94d001 + config: | + [agent] + + interval = "10s" + round_interval = true + metric_batch_size = 1000 + metric_buffer_limit = 1000 + collection_jitter = "0s" + flush_interval = "10s" + flush_jitter = "0s" + + # Outputs + [[outputs.influxdb_v2]] + + urls = ["$INFLUX_HOST"] + token = "$INFLUX_TOKEN" + organization = "$INFLUX_ORG" + bucket = "$INFLUX_BUCKET" + + ############################################################################### + # INPUT PLUGINS # + ############################################################################### + + # for more info on input plugins: https://github.com/influxdata/telegraf/tree/master/plugins/inputs + + # Read system metrics + [[inputs.cpu]] + [[inputs.system]] + [[inputs.kernel]] + [[inputs.mem]] + [[inputs.net]] + [[inputs.processes]] + [[inputs.disk]] + ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"] + [[inputs.diskio]] + + + # Read metrics from fail2ban. + [[inputs.fail2ban]] + ## Use sudo to run fail2ban-client + use_sudo = false + + + ## Timeout for metric collections from all servers. Minimum timeout is "1s". + # timeout = "5s" + + ## Optional TLS Config + # enable_tls = true + # tls_ca = "/etc/telegraf/ca.pem" + # tls_cert = "/etc/telegraf/cert.pem" + # tls_key = "/etc/telegraf/key.pem" + ## If false, skip chain & host verification + # insecure_skip_verify = true + description: This is telegraf config for Fail2Ban template. + name: Fail2Ban Telegraf config diff --git a/fail2ban/img/fail2ban-dashboard.png b/fail2ban/img/fail2ban-dashboard.png new file mode 100644 index 00000000..854a5027 Binary files /dev/null and b/fail2ban/img/fail2ban-dashboard.png differ diff --git a/fail2ban/img/fail2ban-telegraf1.png b/fail2ban/img/fail2ban-telegraf1.png new file mode 100644 index 00000000..bf259f7a Binary files /dev/null and b/fail2ban/img/fail2ban-telegraf1.png differ diff --git a/fail2ban/readme.md b/fail2ban/readme.md new file mode 100644 index 00000000..b39e83ca --- /dev/null +++ b/fail2ban/readme.md @@ -0,0 +1,120 @@ +## Fail2Ban Monitoring Template with Telegraf + +### Dashboards + +#### Fail2Ban + + + +#### InfluxDB UI + +In the InfluxDB UI, go to Settings->Templates and enter this URL: https://raw.githubusercontent.com/influxdata/community-templates/master/fail2ban/img/fail2ban-template.yml + +## Setup Instructions + +If you have your InfluxDB credentials configured in the CLI, you can install this template with: + +`influx apply -f https://raw.githubusercontent.com/influxdata/community-templates/master/fail2ban/fail2ban-template.yml` + +Once installed, you'll need to start up Telegraf to pull metrics from your Fail2Ban client. Checkout [Telegraf docs](https://docs.influxdata.com/telegraf/) for installing and starting Telegraf. + +Telegraf Configuration requires the following environment variables: + +- `INFLUX_ORG` +- `INFLUX_TOKEN` +- `INFLUX_HOST` +- `INFLUX_BUCKET` + +To start Telegraf with the template-provided configuration follow setup instructions: + + + +## Setup Fail2Ban + +You can use Fail2Ban that is packed in your favorite Linux Distribution. +On Debian based distributions you can use `apt install fail2ban`. + +Telegraf uses `fail2ban-client` commandline tool to read metrics. Telegraf needs to have a permission +to access `/var/run/fail2ban/fail2ban.sock` to execute `fail2ban-client`. + +To check this try run `fail2ban-client status` under telegraf user account. +If you see: +``` +fail2ban-client status +Permission denied to socket: /var/run/fail2ban/fail2ban.sock, (you must be root) +``` +You can fix socket permission issue like this: + +#### Add new group fail2ban + +`root@debian:~$ addgroup --group fail2ban` + +#### Add user telegraf to fail2ban group + +`root@debian:~$ usermod -a -G fail2ban telegraf` + +#### Set permissions + +``` +root@server:~$ chown root:fail2ban /var/run/fail2ban/fail2ban.sock +root@server:~$ chmod g+rwx /var/run/fail2ban/fail2ban.sock +``` + +Now you can run `fail2ban-client status` again, and you should see: + +``` +rob@server:~$ fail2ban-client status +Status +|- Number of jail: 1 +`- Jail list: sshd +``` + +Note that this permission fix can be reset by systemd. + +### Systemd fail2ban.service settings +To make permissions permanent you can append them into systemd `/lib/systemd/system/fail2ban.service` configuration: + +``` +[Service] +... +ExecStartPost=/bin/sh -c "while ! [ -S /run/fail2ban/fail2ban.sock ]; do sleep 1; done" +ExecStartPost=/bin/chgrp fail2ban /run/fail2ban/fail2ban.sock +ExecStartPost=/bin/chmod g+w /run/fail2ban/fail2ban.sock +``` + +Then restart the service: +``` +root@server:~$ systemctl restart fail2ban.service +``` + +## Included resources + +- 1 label: `fail2ban` +- 1 Dashboard: `Fail2Ban Metrics` +- 3 Variables: `bucket`, `host`, `jail` +- 1 Telegraf Configuration: + - `[[inputs.fail2ban]]` - [https://github.com/influxdata/telegraf/tree/master/plugins/inputs/fail2ban](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/fail2ban) + - basic system metrics from `[[inputs.cpu]], [[inputs.system]], [[inputs.kernel]], [[inputs.mem]], [[inputs.net]], [[inputs.processes]],[[inputs.disk]], [[inputs.diskio]]` plugins + +## Measurements +
+ +fail2ban + +* Tags + * host + * jail +* Fields + * failed + * banned +
+ +## Contact + +Author: Robert Hájek + +Email: robert.hajek@bonitoo.io + +Github: rhajek + +Influx Slack: @robert.hajek diff --git a/fireboard/fireboard.yml b/fireboard/fireboard.yml new file mode 100755 index 00000000..b5ddf818 --- /dev/null +++ b/fireboard/fireboard.yml @@ -0,0 +1,975 @@ +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: hopeful-golick-f36001 +spec: + color: '#E85B1C' + name: Fireboard +--- +apiVersion: influxdata.com/v2alpha1 +kind: Bucket +metadata: + name: youthful-rubin-f36005 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: fireboard +--- +apiVersion: influxdata.com/v2alpha1 +kind: NotificationEndpointHTTP +metadata: + name: wonderful-goodall-336001 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + description: Example endpoint. Add your own notification endpoint to send Fireboard + notifications to and delete this one. + method: POST + name: Example Endpoint + status: active + type: none + url: https://www.example.com/endpoint +--- +apiVersion: influxdata.com/v2alpha1 +kind: NotificationRule +metadata: + name: loving-jones-336005 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + description: Send a notification whenever the temperature on a probe is stalling. + endpointName: wonderful-goodall-336001 + every: 5m0s + name: Fireboard Stall Notification + statusRules: + - currentLevel: WARN + previousLevel: OK +--- +apiVersion: influxdata.com/v2alpha1 +kind: Task +metadata: + name: adventuring-roentgen-336009 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + every: 5m + name: Fireboard Stall Check + query: |- + import "influxdata/influxdb/monitor" + + + meatProbes = [2,3,4,5,6] + + checkData = { + _check_name: "Fireboard stall check", + _check_id: "fireboard_stalling", + _type: "custom", + tags: {} + } + + check_data = from(bucket:"fireboard") + |> range(start: -15m) + |> filter(fn: (r) => r._measurement == "fireboard" and contains(set: meatProbes, value: int(v: r.channel))) + |> difference(nonNegative: false) + |> aggregateWindow(every: 5m, fn: mean) + |> fill(usePrevious: true) + |> map(fn: (r) => ({ r with + status: + if r._value <= 0.0 then "cooling" + else if r._value <= 0.02 then "stalling" + else "cooking" + })) + + check_data + |> monitor.check( + crit: (r) => r.status == "cooling", + warn: (r) => r.status == "stalling", + ok: (r) => r.status <= "cooking", + messageFn: (r) => "Probe${r.channel} appears to be ${r.status}", + data: checkData + ) +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: blissful-franklin-336035 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: probe3_target + selected: + - "200" + type: constant + values: + - "35" + - "40" + - "45" + - "50" + - "55" + - "60" + - "65" + - "70" + - "75" + - "80" + - "85" + - "90" + - "95" + - "100" + - "105" + - "110" + - "115" + - "120" + - "125" + - "130" + - "135" + - "140" + - "145" + - "150" + - "155" + - "160" + - "165" + - "170" + - "175" + - "180" + - "185" + - "190" + - "195" + - "200" + - "205" + - "210" + - "215" + - "220" + - "225" + - "230" + - "235" + - "240" + - "245" + - "250" + - "255" + - "260" + - "265" + - "270" + - "275" + - "280" + - "285" + - "290" + - "295" + - "300" +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: bold-jang-336031 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: probe3_type + selected: + - Food + type: constant + values: + - Ambient + - Food +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: busy-wing-336025 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: probe1_type + selected: + - Ambient + type: constant + values: + - Ambient + - Food +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: cool-ramanujan-336021 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: probe1_target + selected: + - "200" + type: constant + values: + - "35" + - "40" + - "45" + - "50" + - "55" + - "60" + - "65" + - "70" + - "75" + - "80" + - "85" + - "90" + - "95" + - "100" + - "105" + - "110" + - "115" + - "120" + - "125" + - "130" + - "135" + - "140" + - "145" + - "150" + - "155" + - "160" + - "165" + - "170" + - "175" + - "180" + - "185" + - "190" + - "195" + - "200" + - "205" + - "210" + - "215" + - "220" + - "225" + - "230" + - "235" + - "240" + - "245" + - "250" + - "255" + - "260" + - "265" + - "270" + - "275" + - "280" + - "285" + - "290" + - "295" + - "300" +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: dangerous-kirch-33600d +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: probe5_type + selected: + - Food + type: constant + values: + - Ambient + - Food +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: distracted-davinci-336039 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: probe4_target + selected: + - "200" + type: constant + values: + - "35" + - "40" + - "45" + - "50" + - "55" + - "60" + - "65" + - "70" + - "75" + - "80" + - "85" + - "90" + - "95" + - "100" + - "105" + - "110" + - "115" + - "120" + - "125" + - "130" + - "135" + - "140" + - "145" + - "150" + - "155" + - "160" + - "165" + - "170" + - "175" + - "180" + - "185" + - "190" + - "195" + - "200" + - "205" + - "210" + - "215" + - "220" + - "225" + - "230" + - "235" + - "240" + - "245" + - "250" + - "255" + - "260" + - "265" + - "270" + - "275" + - "280" + - "285" + - "290" + - "295" + - "300" +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: eager-khorana-33602d +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: probe6_type + selected: + - Food + type: constant + values: + - Ambient + - Food +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: endangered-hermann-33601d +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: probe2_target + selected: + - "200" + type: constant + values: + - "35" + - "40" + - "45" + - "50" + - "55" + - "60" + - "65" + - "70" + - "75" + - "80" + - "85" + - "90" + - "95" + - "100" + - "105" + - "110" + - "115" + - "120" + - "125" + - "130" + - "135" + - "140" + - "145" + - "150" + - "155" + - "160" + - "165" + - "170" + - "175" + - "180" + - "185" + - "190" + - "195" + - "200" + - "205" + - "210" + - "215" + - "220" + - "225" + - "230" + - "235" + - "240" + - "245" + - "250" + - "255" + - "260" + - "265" + - "270" + - "275" + - "280" + - "285" + - "290" + - "295" + - "300" +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: focused-goodall-336029 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: probe4_type + selected: + - Food + type: constant + values: + - Ambient + - Food +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: gallant-mclean-336011 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: probe2_type + selected: + - Food + type: constant + values: + - Ambient + - Food +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: hungry-golick-336015 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: probe5_target + selected: + - "200" + type: constant + values: + - "35" + - "40" + - "45" + - "50" + - "55" + - "60" + - "65" + - "70" + - "75" + - "80" + - "85" + - "90" + - "95" + - "100" + - "105" + - "110" + - "115" + - "120" + - "125" + - "130" + - "135" + - "140" + - "145" + - "150" + - "155" + - "160" + - "165" + - "170" + - "175" + - "180" + - "185" + - "190" + - "195" + - "200" + - "205" + - "210" + - "215" + - "220" + - "225" + - "230" + - "235" + - "240" + - "245" + - "250" + - "255" + - "260" + - "265" + - "270" + - "275" + - "280" + - "285" + - "290" + - "295" + - "300" +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: shiny-sanderson-336019 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + name: probe6_target + selected: + - "200" + type: constant + values: + - "35" + - "40" + - "45" + - "50" + - "55" + - "60" + - "65" + - "70" + - "75" + - "80" + - "85" + - "90" + - "95" + - "100" + - "105" + - "110" + - "115" + - "120" + - "125" + - "130" + - "135" + - "140" + - "145" + - "150" + - "155" + - "160" + - "165" + - "170" + - "175" + - "180" + - "185" + - "190" + - "195" + - "200" + - "205" + - "210" + - "215" + - "220" + - "225" + - "230" + - "235" + - "240" + - "245" + - "250" + - "255" + - "260" + - "265" + - "270" + - "275" + - "280" + - "285" + - "290" + - "295" + - "300" +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: angry-tereshkova-f36001 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + charts: + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + suffix: °F + colors: + - hex: '#8F8AF4' + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#A51414' + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#F4CF31' + name: Do Androids Dream of Electric Sheep? + type: scale + geom: line + height: 6 + kind: Xy + name: Overview + position: overlaid + queries: + - query: |- + from(bucket: "fireboard") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "fireboard") + |> filter(fn: (r) => r["_field"] == "temperature") + shade: true + width: 8 + xCol: _time + yCol: _value + - colors: + - hex: '#22ADF6' + name: pool + type: background + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Probe 1 + queries: + - query: |- + from(bucket: "fireboard") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "fireboard") + |> filter(fn: (r) => r["_field"] == "temperature") + |> filter(fn: (r) => r["channel"] == "1") + suffix: ° + width: 2 + xPos: 8 + - colors: + - hex: '#DC4E58' + id: base + name: fire + type: background + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Probe 2 + queries: + - query: |- + from(bucket: "fireboard") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "fireboard") + |> filter(fn: (r) => r["_field"] == "temperature") + |> filter(fn: (r) => r["channel"] == "2") + suffix: ° + width: 2 + xPos: 8 + yPos: 1 + - colors: + - hex: '#F48D38' + id: base + name: tiger + type: background + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Probe 3 + queries: + - query: |- + from(bucket: "fireboard") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "fireboard") + |> filter(fn: (r) => r["_field"] == "temperature") + |> filter(fn: (r) => r["channel"] == "3") + suffix: ° + width: 2 + xPos: 8 + yPos: 2 + - colors: + - hex: '#FFB94A' + id: base + name: pineapple + type: background + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Probe 4 + queries: + - query: |- + from(bucket: "fireboard") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "fireboard") + |> filter(fn: (r) => r["_field"] == "temperature") + |> filter(fn: (r) => r["channel"] == "4") + suffix: ° + width: 2 + xPos: 8 + yPos: 3 + - colors: + - hex: '#FFD255' + id: base + name: thunder + type: background + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Probe 5 + queries: + - query: |- + from(bucket: "fireboard") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "fireboard") + |> filter(fn: (r) => r["_field"] == "temperature") + |> filter(fn: (r) => r["channel"] == "5") + suffix: ° + width: 2 + xPos: 8 + yPos: 4 + - colors: + - hex: '#7CE490' + id: base + name: honeydew + type: background + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Probe 6 + queries: + - query: |- + from(bucket: "fireboard") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "fireboard") + |> filter(fn: (r) => r["_field"] == "temperature") + |> filter(fn: (r) => r["channel"] == "6") + suffix: ° + width: 2 + xPos: 8 + yPos: 5 + - colors: + - hex: '#22ADF6' + name: pool + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Probe 1 Status + queries: + - query: "import \"experimental\"\n\ntype = v.probe1_type\ntarget = float(v: + v.probe1_target)\n\naddStatus = (tables=<-) =>\n tables\n |> map(fn: + (r) => ({ r with\n _value: if r.scale == \"Celsius\" then (r._value + * (9.0/5.0)) + 32.0 else r._value,\n status: if r._value >= target + then \"Done\" else \"\"\n }))\n |> difference()\n |> experimental.group(columns: + [\"status\"], mode: \"extend\")\n |> aggregateWindow(every: 5m, + fn: mean)\n |> group(columns: [\"_time\", \"_value\", \"status\"], + mode: \"except\")\n |> fill(usePrevious: true)\n |> map(fn: + (r) => ({ r with\n _value: \n if type == \"Ambient\" then + \"--\"\n else if r.status == \"Done\" then \"Done\"\n else + if r._value < 0.0 then \"Cooling\"\n else if r._value <= 0.02 + then \"Stalling\"\n else \"Cooking\"\n }))\n\nfrom(bucket: + \"fireboard\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n + \ |> filter(fn: (r) => r[\"_measurement\"] == \"fireboard\")\n |> + filter(fn: (r) => r[\"_field\"] == \"temperature\")\n |> filter(fn: + (r) => r[\"channel\"] == \"1\")\n |> addStatus()\n |> last()" + width: 2 + xPos: 10 + - colors: + - hex: '#F95F53' + id: base + name: curacao + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Probe 2 Status + queries: + - query: "import \"experimental\"\n\ntype = v.probe2_type\ntarget = float(v: + v.probe2_target)\n\naddStatus = (tables=<-) =>\n tables\n |> map(fn: + (r) => ({ r with\n _value: if r.scale == \"Celsius\" then (r._value + * (9.0/5.0)) + 32.0 else r._value,\n status: if r._value >= target + then \"Done\" else \"\"\n }))\n |> difference()\n |> experimental.group(columns: + [\"status\"], mode: \"extend\")\n |> aggregateWindow(every: 5m, + fn: mean)\n |> group(columns: [\"_time\", \"_value\", \"status\"], + mode: \"except\")\n |> fill(usePrevious: true)\n |> map(fn: + (r) => ({ r with\n _value: \n if type == \"Ambient\" then + \"--\"\n else if r.status == \"Done\" then \"Done\"\n else + if r._value < 0.0 then \"Cooling\"\n else if r._value <= 0.02 + then \"Stalling\"\n else \"Cooking\"\n }))\n\nfrom(bucket: + \"fireboard\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n + \ |> filter(fn: (r) => r[\"_measurement\"] == \"fireboard\")\n |> + filter(fn: (r) => r[\"_field\"] == \"temperature\")\n |> filter(fn: + (r) => r[\"channel\"] == \"2\")\n |> addStatus()\n |> last()" + width: 2 + xPos: 10 + yPos: 1 + - colors: + - hex: '#F48D38' + id: base + name: tiger + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Probe 3 Status + queries: + - query: "import \"experimental\"\n\ntype = v.probe3_type\ntarget = float(v: + v.probe3_target)\n\naddStatus = (tables=<-) =>\n tables\n |> map(fn: + (r) => ({ r with\n _value: if r.scale == \"Celsius\" then (r._value + * (9.0/5.0)) + 32.0 else r._value,\n status: if r._value >= target + then \"Done\" else \"\"\n }))\n |> difference()\n |> experimental.group(columns: + [\"status\"], mode: \"extend\")\n |> aggregateWindow(every: 5m, + fn: mean)\n |> group(columns: [\"_time\", \"_value\", \"status\"], + mode: \"except\")\n |> fill(usePrevious: true)\n |> map(fn: + (r) => ({ r with\n _value: \n if type == \"Ambient\" then + \"--\"\n else if r.status == \"Done\" then \"Done\"\n else + if r._value < 0.0 then \"Cooling\"\n else if r._value <= 0.02 + then \"Stalling\"\n else \"Cooking\"\n }))\n\nfrom(bucket: + \"fireboard\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n + \ |> filter(fn: (r) => r[\"_measurement\"] == \"fireboard\")\n |> + filter(fn: (r) => r[\"_field\"] == \"temperature\")\n |> filter(fn: + (r) => r[\"channel\"] == \"3\")\n |> addStatus()\n |> last()" + width: 2 + xPos: 10 + yPos: 2 + - colors: + - hex: '#FFB94A' + id: base + name: pineapple + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Probe 4 Status + queries: + - query: "import \"experimental\"\n\ntype = v.probe4_type\ntarget = float(v: + v.probe4_target)\n\naddStatus = (tables=<-) =>\n tables\n |> map(fn: + (r) => ({ r with\n _value: if r.scale == \"Celsius\" then (r._value + * (9.0/5.0)) + 32.0 else r._value,\n status: if r._value >= target + then \"Done\" else \"\"\n }))\n |> difference()\n |> experimental.group(columns: + [\"status\"], mode: \"extend\")\n |> aggregateWindow(every: 5m, + fn: mean)\n |> group(columns: [\"_time\", \"_value\", \"status\"], + mode: \"except\")\n |> fill(usePrevious: true)\n |> map(fn: + (r) => ({ r with\n _value: \n if type == \"Ambient\" then + \"--\"\n else if r.status == \"Done\" then \"Done\"\n else + if r._value < 0.0 then \"Cooling\"\n else if r._value <= 0.002 + then \"Stalling\"\n else \"Cooking\"\n }))\n\nfrom(bucket: + \"fireboard\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n + \ |> filter(fn: (r) => r[\"_measurement\"] == \"fireboard\")\n |> + filter(fn: (r) => r[\"_field\"] == \"temperature\")\n |> filter(fn: + (r) => r[\"channel\"] == \"4\")\n |> addStatus()\n |> last()" + width: 2 + xPos: 10 + yPos: 3 + - colors: + - hex: '#FFD255' + id: base + name: thunder + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Probe 5 Status + queries: + - query: "import \"experimental\"\n\ntype = v.probe5_type\ntarget = float(v: + v.probe5_target)\n\naddStatus = (tables=<-) =>\n tables\n |> map(fn: + (r) => ({ r with\n _value: if r.scale == \"Celsius\" then (r._value + * (9.0/5.0)) + 32.0 else r._value,\n status: if r._value >= target + then \"Done\" else \"\"\n }))\n |> difference()\n |> experimental.group(columns: + [\"status\"], mode: \"extend\")\n |> aggregateWindow(every: 5m, + fn: mean)\n |> group(columns: [\"_time\", \"_value\", \"status\"], + mode: \"except\")\n |> fill(usePrevious: true)\n |> map(fn: + (r) => ({ r with\n _value: \n if type == \"Ambient\" then + \"--\"\n else if r.status == \"Done\" then \"Done\"\n else + if r._value < 0.0 then \"Cooling\"\n else if r._value <= 0.02 + then \"Stalling\"\n else \"Cooking\"\n }))\n\nfrom(bucket: + \"fireboard\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n + \ |> filter(fn: (r) => r[\"_measurement\"] == \"fireboard\")\n |> + filter(fn: (r) => r[\"_field\"] == \"temperature\")\n |> filter(fn: + (r) => r[\"channel\"] == \"5\")\n |> addStatus()\n |> last()" + width: 2 + xPos: 10 + yPos: 4 + - colors: + - hex: '#7CE490' + id: base + name: honeydew + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Probe 6 Status + queries: + - query: "import \"experimental\"\n\ntype = v.probe6_type\ntarget = float(v: + v.probe6_target)\n\naddStatus = (tables=<-) =>\n tables\n |> map(fn: + (r) => ({ r with\n _value: if r.scale == \"Celsius\" then (r._value + * (9.0/5.0)) + 32.0 else r._value,\n status: if r._value >= target + then \"Done\" else \"\"\n }))\n |> difference()\n |> experimental.group(columns: + [\"status\"], mode: \"extend\")\n |> aggregateWindow(every: 5m, + fn: mean)\n |> group(columns: [\"_time\", \"_value\", \"status\"], + mode: \"except\")\n |> fill(usePrevious: true)\n |> map(fn: + (r) => ({ r with\n _value: \n if type == \"Ambient\" then + \"--\"\n else if r.status == \"Done\" then \"Done\"\n else + if r._value < 0.0 then \"Cooling\"\n else if r._value <= 0.02 + then \"Stalling\"\n else \"Cooking\"\n }))\n\nfrom(bucket: + \"fireboard\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n + \ |> filter(fn: (r) => r[\"_measurement\"] == \"fireboard\")\n |> + filter(fn: (r) => r[\"_field\"] == \"temperature\")\n |> filter(fn: + (r) => r[\"channel\"] == \"6\")\n |> addStatus()\n |> last()" + width: 2 + xPos: 10 + yPos: 5 + description: Display real-time temperature data collected from the Fireboard API + name: Fireboard +--- +apiVersion: influxdata.com/v2alpha1 +kind: Telegraf +metadata: + name: compassionate-rosalind-f36005 +spec: + associations: + - kind: Label + name: hopeful-golick-f36001 + config: |- + # Global tags can be specified here in key="value" format. + [global_tags] + + # Configuration for telegraf agent + [agent] + interval = "30s" + round_interval = true + metric_batch_size = 1000 + metric_buffer_limit = 10000 + collection_jitter = "0s" + flush_interval = "10s" + flush_jitter = "30s" + precision = "" + hostname = "" + omit_hostname = true + + + ############################################################################### + # OUTPUT PLUGINS # + ############################################################################### + + [[outputs.influxdb_v2]] + urls = ["$INFLUX_HOST"] + token = "$INFLUX_TOKEN" + organization = "$INFLUX_ORG" + bucket = "fireboard" + + [[inputs.fireboard]] + ## Specify auth token for your account + auth_token = "$FIREBOARD_TOKEN" + description: Pull sensor data from the Fireboard API + name: Fireboard diff --git a/fireboard/img/fireboard_dashboard.png b/fireboard/img/fireboard_dashboard.png new file mode 100644 index 00000000..b558bdb8 Binary files /dev/null and b/fireboard/img/fireboard_dashboard.png differ diff --git a/fireboard/img/fireboard_setup_telegraf.png b/fireboard/img/fireboard_setup_telegraf.png new file mode 100644 index 00000000..e3aa3cb7 Binary files /dev/null and b/fireboard/img/fireboard_setup_telegraf.png differ diff --git a/fireboard/readme.md b/fireboard/readme.md new file mode 100644 index 00000000..815406d5 --- /dev/null +++ b/fireboard/readme.md @@ -0,0 +1,144 @@ +## Fireboard Monitoring Template + +Provided by: [Scott Anderson](https://github.com/sanderson/) + +Use this InfluxDB template to monitor data collected by the +[Fireboard cloud-connected thermometer](https://www.fireboard.com/). + +![Fireboard Dashboard Screenshot](img/fireboard_dashboard.png) + +## Quick Install + +### InfluxDB UI + +In the InfluxDB UI, go to **Settings > Templates** and enter the following URL: + +```sh +https://raw.githubusercontent.com/influxdata/community-templates/master/fireboard/fireboard.yml +``` + +### Influx CLI + +If you have your InfluxDB credentials [configured in the CLI](https://docs.influxdata.com/influxdb/v2.0/reference/cli/influx/config/), install this template with: + +```sh +influx apply -u https://raw.githubusercontent.com/influxdata/community-templates/master/fireboard/fireboard.yml +``` + +## Included Resources + +- 1 Bucket: `fireboard` +- 1 Telegraf Configuration: `Fireboard` +- 1 Dashboard: `Fireboard` +- 12 Dashboard Variables: + - `probe1_target` + - `probe1_type` + - `probe2_target` + - `probe2_type` + - `probe3_target` + - `probe3_type` + - `probe4_target` + - `probe4_type` + - `probe5_target` + - `probe5_type` + - `probe6_target` + - `probe6_type` +- 1 Task: `Fireboard Stall Check` +- 1 Notification Endpoint: `Example Endpoint` +- 1 Notification Rule: `Fireboard Stall Notification` +- 1 Label: `Fireboard` + +## Setup Instructions + +General instructions on using InfluxDB Templates can be found in [Use a template](../docs/use_a_template.md). + +### Get Your Fireboard Authentication Token + +1. [Create a Fireboard account](https://docs.fireboard.io/start/account.html#create-your-account). +2. `POST` your login credentials to the Fireboard API to retrieve your Fireboard authentication token. + + ```sh + curl https://fireboard.io/api/rest-auth/login/ + -X POST + -H 'Content-Type: application/json' + -d '{"username":"your-username-here","password":"your-password-here"}' + ``` + + ##### Example response + ```json + { "key": "9944bb9966cc22cc9418ad846dd0e4bbdfc6ee4b" } + ``` + +### Configure Environment Variables + +The data for the Fireboard dashboard is populated by the Telegraf configuration +included in the template. +The Telegraf configuration requires the following environment variables: + +- `INFLUX_TOKEN` - InfluxDB authentication token with the permissions to read + Telegraf configurations and write data to the `fireboard` bucket. + Use your operator token to get started or generate a new token in the + Telegraf configuration setup instructions. + Find tokens in the InfluxDB UI under **Load Data > Tokens**. +- `INFLUX_ORG` - Name of your InfluxDB organization. + If using InfluxDB Cloud, this is the email address you used to create your account. + To find your organization name, click your user avatar in the InfluxDB UI. + Your organization name appears under your username. +- `INFLUX_HOST` - The URL of your InfluxDB instance. + See [InfluxDB Cloud regions](https://docs.influxdata.com/influxdb/cloud/reference/regions/) + or [InfluxDB OSS 2.x URLs](https://docs.influxdata.com/influxdb/latest/reference/urls/). +- `FIREBOARD_TOKEN` – Your [Fireboard authentication token](#get-your-fireboard-authentication-token). + +You **MUST** set these environment variables **before** running Telegraf. +Using commands similar to the following: + +```sh +export INFLUX_TOKEN=YourInfluxDBAuthToken +export INFLUX_ORG=YourInfluxDBOrg +export INFLUX_HOST=http://localhost:8086 +export FIREBOARD_TOKEN=YourFireboardAuthToken +``` + +## Run Telegraf + +1. [Download and install Telegraf](https://portal.influxdata.com/downloads/). +2. Start Telegraf using the instructions provided in **Load Data > Telegraf > Setup Instructions** + in the InfluxDB UI. + + ![Telegraf Setup Instructions](img/fireboard_setup_telegraf.png) + +## Customizations + +### Dashboard Cells + +The **Fireboard dashboard** provided by the template includes cells for individual +Fireboard probes that display the current temperature and cooking status. +Use the dashboard variables specific to each cell to select the following: + +- **Probe target:** target temperature for the probe (`30 - 300`) +- **Probe type:** `Ambient` or `Food` + +### Fireboard Stall Check Task + +The **Fireboard Stall Check** task provided by the template includes a `meatProbes` +variable that lists which Fireboard probe channels should be tracked when +identifying stalls. By default, the task assumes **Probe 1** is an ambient probe +and all other probes should be tracked. To change what channels are tracked, +edit the task and customize the variable: + +```js +meatProbes = [2,3,4,5,6] +``` + +_**NOTE:** The **Fireboard Stall Check** task is disabled by default._ + +### Notification Endpoint +The template provides tasks that trigger notifications. +To send notifications, [add a notification endpoint](https://docs.influxdata.com/influxdb/v2.0/monitor-alert/notification-endpoints/create/). + +## Contact + +- Author: Scott Anderson +- Email: scott@influxdata.com +- Github: [@sanderson](https://github.com/sanderson) +- Influx Slack: [@Scott](https://influxdata.com/slack) diff --git a/fortnite/readme.md b/fortnite/readme.md index f2d6f742..787c0e56 100644 --- a/fortnite/readme.md +++ b/fortnite/readme.md @@ -41,48 +41,48 @@ Note: If `fortnite` is not used for the `fn_bucket` name, the bucket name will n ## Included Resources - 1 Bucket: `fortnite` - - 30d retention - - `fn_bucket` - user-definable resource name + - 30d retention + - `fn_bucket` - user-definable resource name - 1 Label: `fortnite` - 1 Telegraf Configuration - - `exec` input plugin - - `influxdb_v2` output plugin + - `exec` input plugin + - `influxdb_v2` output plugin - 1 Task: `wins` - - uses Slack webhook + - uses Slack webhook - 3 Dashboards: - - `Fortnite - All Players` - - `Fortnite - Individual Stats` - - `Fortnite - Player Comparison` + - `Fortnite - All Players` + - `Fortnite - Individual Stats` + - `Fortnite - Player Comparison` - 4 Query Variables: - - `bucket` - - `player` - - `player2` - - `season` + - `bucket` + - `player` + - `player2` + - `season` - 1 Python script: `get_fn_stats.py` - - used by Telegraf exec plugin to access Fortnite API endpoint + - used by Telegraf exec plugin to access Fortnite API endpoint - 1 CSV file: `players.csv` - - seeded with professional Fortnite player ids - - used by Python script + - seeded with professional Fortnite player ids + - used by Python script ## Setup Instructions 1. Register for an account at [fortniteapi.io](https://fortniteapi.io/) 2. Retrieve your Fortnite API token at [dashboard.fortniteapi.io](https://dashboard.fortniteapi.io) 3. Look up Fortnite account ids with the following curl request: - ``` - curl --request GET 'https://fortniteapi.io/lookup?username=' \ - --header 'Authorization: ' - ``` +``` +curl --request GET 'https://fortniteapi.io/lookup?username=' \ + --header 'Authorization: ' +``` 4. Check that the account returns metrics: - ``` - curl --request GET 'https://fortniteapi.io/stats?account=&season=' \ - --header 'Authorization: ' - ``` - - Note: If the response contains `"global_stats":null` it's likely that that the **Show on Career Leaderboard** setting is set to **OFF** in the **Account and Privacy** section of the player's profile. +``` +curl --request GET 'https://fortniteapi.io/stats?account=&season=' \ + --header 'Authorization: ' +``` + - Note: If the response contains `"global_stats":null` it's likely that that the **Show on Career Leaderboard** setting is set to **OFF** in the **Account and Privacy** section of the player's profile. 5. Add accounts to track in `players.csv`. For example: - - `4735ce9132924caf8a5b17789b40f79c,yes,Ninja` - - The second column indicates professional status: `yes`|`no` + - `4735ce9132924caf8a5b17789b40f79c,yes,Ninja` + - The second column indicates professional status: `yes`|`no` ### Telegraf The Telegraf configuration requires the following environmental variables: @@ -96,15 +96,15 @@ Information on using enviromental variables can be found in the [Telegraf Config The Telegraf `exec` input plugin requires the following files: - `get_fn_stats.py` - Python script used to ingest metrics via the unofficial [Fortnite API](https://fortniteapi.io/) - - `requests` module required - - The current season is hardcoded: `season = 14` - - Update this when a new season starts. + - `requests` module required + - The current season is hardcoded: `season = 14` + - Update this when a new season starts. - `players.csv` - CSV file containing a list of players to track using the following fields: - - `acct_id` - Fortnite Account ID - - `pro` - **yes** for professional, **no** for friends and family - - `player_name` - not used by Telegraf - - Note: This file is populated with several professional players. Add friends, family and others that will be tracked separately from the pro players. + - `acct_id` - Fortnite Account ID + - `pro` - **yes** for professional, **no** for friends and family + - `player_name` - not used by Telegraf + - Note: This file is populated with several professional players. Add friends, family and others that will be tracked separately from the pro players. ### Dashboards The `Fortnite - All Players` dashboard displays a url drilldown to the `Fortnite - Individual Stats` dashboard. This url must be changed in the `Friends & Family` and `Pros` cells: @@ -117,7 +117,7 @@ To get the full link, open the `Fortnite - Individual Stats` dashboard and copy The Influx task requires the following [influx secret](https://v2.docs.influxdata.com/v2.0/reference/cli/influx/secret/): - `SLACK_WEBHOOK` - The Slack webhook is used to send a message when a player wins a match. - - https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX + - https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX ``` influx secret update -k SLACK_WEBHOOK ``` diff --git a/github/readme.md b/github/readme.md index a86def97..9723c991 100644 --- a/github/readme.md +++ b/github/readme.md @@ -21,20 +21,20 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf: `Configuration: Github Inputs` - - 1 Dashboards: `Github - InfluxData [InfluxDB & Telegraf repos]` - - 1 Bucket: `github` - - 1 Label: `github` + - 1 Telegraf: `Configuration: Github Inputs` + - 1 Dashboards: `Github - InfluxData [InfluxDB & Telegraf repos]` + - 1 Bucket: `github` + - 1 Label: `github` ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. - - `INFLUX_HOST` - The URL of your InfluxDB host (this can be your localhost, a remote instance, or InfluxDB Cloud). - - `GITHUB_ACCESS_TOKEN` - Your personal Github token for making API calls +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The URL of your InfluxDB host (this can be your localhost, a remote instance, or InfluxDB Cloud). + - `GITHUB_ACCESS_TOKEN` - Your personal Github token for making API calls > You can generate a Token Access Key from Github on this page: https://github.com/settings/tokens. diff --git a/haproxy/readme.md b/haproxy/readme.md index 76420f5a..72d4cf6a 100644 --- a/haproxy/readme.md +++ b/haproxy/readme.md @@ -21,19 +21,19 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf: `haproxy-config` - - 1 Dashboards: `haproxy` - - 1 Bucket: `haproxy` - - 1 Label: `haproxy` + - 1 Telegraf: `haproxy-config` + - 1 Dashboards: `haproxy` + - 1 Bucket: `haproxy` + - 1 Label: `haproxy` ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. - - `INFLUX_HOST` - The host where run InfluxDB +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The host where run InfluxDB In order to use this template, you need to specify the HAProxy instance, if you have one to monitor, you can pass as variable the name and port of the host. Ex ```$ export haproxy_stats=http://localhost:10000/stats``` If you have more instances of HAProxy I recommend you to edit the haproxy.yml to add others hosts. diff --git a/influxdb2_cardinality_now/cardinality_now.yml b/influxdb2_cardinality_now/cardinality_now.yml new file mode 100644 index 00000000..3087db07 --- /dev/null +++ b/influxdb2_cardinality_now/cardinality_now.yml @@ -0,0 +1,238 @@ +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: kind-spence-53b001 +spec: + color: '#fafafc' + name: cardinality +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: amazing-boyd-93b003 +spec: + associations: + - kind: Label + name: kind-spence-53b001 + language: flux + name: bucket + query: |- + buckets() + |> filter(fn: (r) => r.name !~ /^_/) + |> rename(columns: {name: "_value"}) + |> keep(columns: ["_value"]) + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: blissful-morse-53b003 +spec: + associations: + - kind: Label + name: kind-spence-53b001 + language: flux + name: measurement + query: |- + import "influxdata/influxdb/v1" + + v1.measurements(bucket: v.bucket) + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: nostalgic-einstein-13b001 +spec: + associations: + - kind: Label + name: kind-spence-53b001 + charts: + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#31C0F6' + id: 5b3aea53-4f8a-47e5-bda3-3e9299df5fa1 + name: Nineteen Eighty Four + type: scale + - hex: '#A500A5' + id: e8fec99c-5ff4-43aa-b30a-d3c88d343fb9 + name: Nineteen Eighty Four + type: scale + - hex: '#FF7E27' + id: 3f9c84bb-6dfa-42e3-aa35-2e87e6a88842 + name: Nineteen Eighty Four + type: scale + geom: line + height: 3 + hoverDimension: auto + kind: Xy + legendOpacity: 1 + legendOrientationThreshold: 10 + name: Cardinality Per Bucket + position: overlaid + queries: + - query: |- + import "influxdata/influxdb" + import "experimental" + + time = experimental.subDuration( + d: v.timeRangeStart, + from: now(), + ) + + buckets() + |> map(fn: (r) => { + cardinality = influxdb.cardinality(bucket: r.name, start: -7d) + |> findRecord(idx: 0, fn: (key) => + (true)) + return { + _time: time, + _measurement: "buckets", + bucket: r.name, + _field: "cardinality", + _value: cardinality._value, + } + }) + |> group(columns: ["bucket"], mode:"by") + width: 12 + xCol: _time + yCol: _value + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + fieldOptions: + - displayName: _field + fieldName: _field + - displayName: _measurement + fieldName: _measurement + - displayName: _time + fieldName: _time + - displayName: bucket + fieldName: bucket + visible: true + - displayName: _value + fieldName: _value + visible: true + height: 4 + kind: Table + name: Top 10 Buckets + queries: + - query: |- + import "influxdata/influxdb" + import "experimental" + + time = experimental.subDuration( + d: v.timeRangeStart, + from: now(), + ) + + buckets() + |> map(fn: (r) => { + cardinality = influxdb.cardinality(bucket: r.name, start: -7d) + |> findRecord(idx: 0, fn: (key) => + (true)) + return { + _time: time, + _measurement: "buckets", + bucket: r.name, + _field: "cardinality", + _value: cardinality._value, + } + }) + |> group(columns: ["bucket"], mode:"by") + |> last() + |> group() + |> top(n:10) + tableOptions: + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 12 + yPos: 3 + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + fieldOptions: + - displayName: bucket + fieldName: bucket + visible: true + - displayName: measurement + fieldName: measurement + visible: true + - displayName: cardinality + fieldName: _value + visible: true + height: 3 + kind: Table + name: Cardinality By Mesurement + queries: + - query: |- + import "influxdata/influxdb" + import "influxdata/influxdb/schema" + + + schema.measurements(bucket: v.bucket) + |> map(fn: (r) => { + m = r._value + return { + bucket: v.bucket, + measurement: m, + _value: (influxdb.cardinality(bucket: v.bucket, start: v.timeRangeStart, predicate: (r) => r._measurement == m) |> findRecord(idx:0, fn:(key) => true))._value + } + }) + |> sort(desc:true) + tableOptions: + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 12 + yPos: 7 + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + fieldOptions: + - displayName: tag + fieldName: tag + visible: true + - displayName: cardinality + fieldName: _value + visible: true + height: 3 + kind: Table + name: Cardinality By Tag + queries: + - query: |- + import "influxdata/influxdb/schema" + + cardinalityByTag = (bucket, measurement) => schema.tagKeys(bucket: bucket, predicate: (r) => r._measurement == measurement) + |> filter(fn: (r) => r._value != "_start" and r._value != "_stop") + |> map( + fn: (r) => ({ + tag: r._value, + _value: (schema.tagValues(bucket: bucket, tag: r._value) + |> count() + |> findRecord(fn: (key) => true, idx: 0))._value, + }), + ) + |> group(columns: ["tag"]) + |> sum() + + cardinalityByTag(bucket: v.bucket, measurement: v.measurement) + |> group() + |> top(n: 10) + tableOptions: + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 12 + yPos: 10 + name: Cardinality Now diff --git a/influxdb2_cardinality_now/img/cardinality-explorer-dashboard2.png b/influxdb2_cardinality_now/img/cardinality-explorer-dashboard2.png new file mode 100644 index 00000000..4202aee3 Binary files /dev/null and b/influxdb2_cardinality_now/img/cardinality-explorer-dashboard2.png differ diff --git a/influxdb2_cardinality_now/readme.md b/influxdb2_cardinality_now/readme.md new file mode 100644 index 00000000..beb5a74a --- /dev/null +++ b/influxdb2_cardinality_now/readme.md @@ -0,0 +1,39 @@ +# InfluxDB 2 Operational Monitoring + +Provided by: InfluxData + +This InfluxDB Template can be used to monitor your current InfluxDB cardinality on your already running InfluxDB 2 instance when you hit your cardinality limit. This template is only compatible with the Cloud version of InfluxDB 2. + +This is a slimmed down version of the [influxdb2_operational_monitoring](../influxdb2_operational_monitoring) template. This version allows you to monitor your currently cardinality without increasing the cardinality of your instance. + +This `Cardinality Now` dashboard can help you identify your source of runaway cardinality. +![Cardinality Explorer Screenshot](img/cardinality-explorer-dashboard2.png) +Use this dashboard to drill into the cardinality of specific measurements to identify the source of runaway cardinality. + +### Quick Install + +#### InfluxDB UI + +In the InfluxDB UI, go to Settings->Templates and enter this URL: https://raw.githubusercontent.com/influxdata/community-templates/master/influxdb2_operational_monitoring/influxdb2_operational_monitoring.yml + +#### Influx CLI +If you have your InfluxDB credentials [configured in the CLI](https://v2.docs.influxdata.com/v2.0/reference/cli/influx/config/), you can install this template with: + +``` +influx apply -u https://raw.githubusercontent.com/influxdata/community-templates/master/influxdb2_operational_monitoring/influxdb2_operational_monitoring.yml +``` + +## Included Resources + - 1 Labels: `cardinality` + - 1 Dashboard: `Cardinality Now` + +## Setup Instructions + + General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. + +## Contact + +- Author: Anais Dotis-Georgiou +- Email: anais@influxdata.com +- Github: [@Anaisdg](https://github.com/Anaisdg) +- Influx Slack: [@Anais](https://influxdata.com/slack) diff --git a/influxdb2_operational_monitoring/img/cardinality-explorer-dashboard1.png b/influxdb2_operational_monitoring/img/cardinality-explorer-dashboard1.png new file mode 100644 index 00000000..8941edb3 Binary files /dev/null and b/influxdb2_operational_monitoring/img/cardinality-explorer-dashboard1.png differ diff --git a/influxdb2_operational_monitoring/img/cardinality-explorer-dashboard2.png b/influxdb2_operational_monitoring/img/cardinality-explorer-dashboard2.png new file mode 100644 index 00000000..4202aee3 Binary files /dev/null and b/influxdb2_operational_monitoring/img/cardinality-explorer-dashboard2.png differ diff --git a/influxdb2_operational_monitoring/img/task-summary-dashboard.png b/influxdb2_operational_monitoring/img/task-summary-dashboard.png new file mode 100644 index 00000000..6afbfd34 Binary files /dev/null and b/influxdb2_operational_monitoring/img/task-summary-dashboard.png differ diff --git a/influxdb2_operational_monitoring/influxdb2_cardinality_now.yml b/influxdb2_operational_monitoring/influxdb2_cardinality_now.yml new file mode 100644 index 00000000..3087db07 --- /dev/null +++ b/influxdb2_operational_monitoring/influxdb2_cardinality_now.yml @@ -0,0 +1,238 @@ +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: kind-spence-53b001 +spec: + color: '#fafafc' + name: cardinality +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: amazing-boyd-93b003 +spec: + associations: + - kind: Label + name: kind-spence-53b001 + language: flux + name: bucket + query: |- + buckets() + |> filter(fn: (r) => r.name !~ /^_/) + |> rename(columns: {name: "_value"}) + |> keep(columns: ["_value"]) + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: blissful-morse-53b003 +spec: + associations: + - kind: Label + name: kind-spence-53b001 + language: flux + name: measurement + query: |- + import "influxdata/influxdb/v1" + + v1.measurements(bucket: v.bucket) + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: nostalgic-einstein-13b001 +spec: + associations: + - kind: Label + name: kind-spence-53b001 + charts: + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#31C0F6' + id: 5b3aea53-4f8a-47e5-bda3-3e9299df5fa1 + name: Nineteen Eighty Four + type: scale + - hex: '#A500A5' + id: e8fec99c-5ff4-43aa-b30a-d3c88d343fb9 + name: Nineteen Eighty Four + type: scale + - hex: '#FF7E27' + id: 3f9c84bb-6dfa-42e3-aa35-2e87e6a88842 + name: Nineteen Eighty Four + type: scale + geom: line + height: 3 + hoverDimension: auto + kind: Xy + legendOpacity: 1 + legendOrientationThreshold: 10 + name: Cardinality Per Bucket + position: overlaid + queries: + - query: |- + import "influxdata/influxdb" + import "experimental" + + time = experimental.subDuration( + d: v.timeRangeStart, + from: now(), + ) + + buckets() + |> map(fn: (r) => { + cardinality = influxdb.cardinality(bucket: r.name, start: -7d) + |> findRecord(idx: 0, fn: (key) => + (true)) + return { + _time: time, + _measurement: "buckets", + bucket: r.name, + _field: "cardinality", + _value: cardinality._value, + } + }) + |> group(columns: ["bucket"], mode:"by") + width: 12 + xCol: _time + yCol: _value + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + fieldOptions: + - displayName: _field + fieldName: _field + - displayName: _measurement + fieldName: _measurement + - displayName: _time + fieldName: _time + - displayName: bucket + fieldName: bucket + visible: true + - displayName: _value + fieldName: _value + visible: true + height: 4 + kind: Table + name: Top 10 Buckets + queries: + - query: |- + import "influxdata/influxdb" + import "experimental" + + time = experimental.subDuration( + d: v.timeRangeStart, + from: now(), + ) + + buckets() + |> map(fn: (r) => { + cardinality = influxdb.cardinality(bucket: r.name, start: -7d) + |> findRecord(idx: 0, fn: (key) => + (true)) + return { + _time: time, + _measurement: "buckets", + bucket: r.name, + _field: "cardinality", + _value: cardinality._value, + } + }) + |> group(columns: ["bucket"], mode:"by") + |> last() + |> group() + |> top(n:10) + tableOptions: + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 12 + yPos: 3 + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + fieldOptions: + - displayName: bucket + fieldName: bucket + visible: true + - displayName: measurement + fieldName: measurement + visible: true + - displayName: cardinality + fieldName: _value + visible: true + height: 3 + kind: Table + name: Cardinality By Mesurement + queries: + - query: |- + import "influxdata/influxdb" + import "influxdata/influxdb/schema" + + + schema.measurements(bucket: v.bucket) + |> map(fn: (r) => { + m = r._value + return { + bucket: v.bucket, + measurement: m, + _value: (influxdb.cardinality(bucket: v.bucket, start: v.timeRangeStart, predicate: (r) => r._measurement == m) |> findRecord(idx:0, fn:(key) => true))._value + } + }) + |> sort(desc:true) + tableOptions: + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 12 + yPos: 7 + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + fieldOptions: + - displayName: tag + fieldName: tag + visible: true + - displayName: cardinality + fieldName: _value + visible: true + height: 3 + kind: Table + name: Cardinality By Tag + queries: + - query: |- + import "influxdata/influxdb/schema" + + cardinalityByTag = (bucket, measurement) => schema.tagKeys(bucket: bucket, predicate: (r) => r._measurement == measurement) + |> filter(fn: (r) => r._value != "_start" and r._value != "_stop") + |> map( + fn: (r) => ({ + tag: r._value, + _value: (schema.tagValues(bucket: bucket, tag: r._value) + |> count() + |> findRecord(fn: (key) => true, idx: 0))._value, + }), + ) + |> group(columns: ["tag"]) + |> sum() + + cardinalityByTag(bucket: v.bucket, measurement: v.measurement) + |> group() + |> top(n: 10) + tableOptions: + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 12 + yPos: 10 + name: Cardinality Now diff --git a/influxdb2_operational_monitoring/influxdb2_operational_monitoring.yml b/influxdb2_operational_monitoring/influxdb2_operational_monitoring.yml new file mode 100644 index 00000000..b332b3db --- /dev/null +++ b/influxdb2_operational_monitoring/influxdb2_operational_monitoring.yml @@ -0,0 +1,592 @@ +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: affectionate-jackson-9aa001 +spec: + color: '#ebadf8' + description: tasks and cardinality monitoring + name: operational_monitoring +--- +apiVersion: influxdata.com/v2alpha1 +kind: Bucket +metadata: + name: sharp-thompson-daa003 +spec: + associations: + - kind: Label + name: affectionate-jackson-9aa001 + name: cardinality +--- +apiVersion: influxdata.com/v2alpha1 +kind: Task +metadata: + name: alerting-northcutt-daa003 +spec: + associations: + - kind: Label + name: affectionate-jackson-9aa001 + every: 1h + name: cardinality_by_bucket + query: |- + import "influxdata/influxdb" + + + + buckets() + |> map(fn: (r) => { + cardinality = influxdb.cardinality(bucket: r.name, start: -task.every) + |> findRecord(idx: 0, fn: (key) => + (true)) + + return { + _time: now(), + _measurement: "buckets", + bucket: r.name, + _field: "cardinality", + _value: cardinality._value, + } + }) + |> to(bucket: "cardinality") +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: eager-proskuriakova-9aa003 +spec: + associations: + - kind: Label + name: affectionate-jackson-9aa001 + language: flux + name: bucket + query: |- + buckets() + |> filter(fn: (r) => r.name !~ /^_/) + |> rename(columns: {name: "_value"}) + |> keep(columns: ["_value"]) + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: relaxed-archimedes-5aa003 +spec: + associations: + - kind: Label + name: affectionate-jackson-9aa001 + language: flux + name: measurement + query: |- + import "influxdata/influxdb/v1" + + v1.measurements(bucket: v.bucket) + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: magical-brahmagupta-5aa001 +spec: + associations: + - kind: Label + name: affectionate-jackson-9aa001 + charts: + - colors: + - hex: '#00C9FF' + id: base + name: laser + type: text + decimalPlaces: 2 + height: 4 + kind: Single_Stat + name: Total Runs + queries: + - query: |- + import "strings" + import "regexp" + + from(bucket: "_tasks") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "runs") + |> filter(fn: (r) => r["_field"] == "logs") + |> map(fn: (r) => { + ind = regexp.findStringIndex(r: /name: \\\\\\".+?\\\\\\"/, v: r._value) + finalstring = strings.trimSuffix(v: strings.trimPrefix(v: strings.substring(end: ind[1], start: ind[0], v: r._value ), prefix: "name: \\\\\\\""), suffix: "\\\\\\\"") + return {r with name: finalstring } + } + ) + |> group(columns: ["_start"]) + |> count() + |> rename(columns: {_start: "_time"}) + width: 4 + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + fieldOptions: + - displayName: name + fieldName: name + visible: true + - displayName: taskID + fieldName: taskID + visible: true + - displayName: success + fieldName: success + visible: true + - displayName: failed + fieldName: failed + visible: true + - displayName: errorRate + fieldName: errorRate + visible: true + height: 4 + kind: Table + name: Error Rates Per Task + queries: + - query: |- + import "influxdata/influxdb/v1" + import "strings" + import "regexp" + + + from(bucket: "_tasks") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "runs") + |> filter(fn: (r) => r["_field"] == "logs" or r._field == "name") + |> v1.fieldsAsCols() + |> group(columns: ["taskID", "name", "status"]) + |> count(column: "logs") + |> group() + |> pivot(rowKey:["taskID", "name"], columnKey: ["status"], valueColumn: "logs") + |> map(fn: (r) => ({r with success: if exists r.success then r.success else 0, failed: if exists r.failed then r.failed else 0})) + |> map(fn: (r)=> ({r with errorRate: float(v: r.failed)/float(v: r.success+r.failed) })) + tableOptions: + sortBy: failed + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 12 + yPos: 4 + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + fieldOptions: + - displayName: _time + fieldName: _time + visible: true + - displayName: name + fieldName: name + visible: true + - displayName: taskID + fieldName: taskID + visible: true + - displayName: runID + fieldName: runID + visible: true + - displayName: _value + fieldName: _value + - displayName: status + fieldName: status + - displayName: failed + fieldName: failed + - displayName: success + fieldName: success + - displayName: errorRate + fieldName: errorRate + - displayName: _measurement + fieldName: _measurement + - displayName: _start + fieldName: _start + - displayName: _stop + fieldName: _stop + - displayName: logs + fieldName: logs + - displayName: message + fieldName: message + visible: true + height: 3 + kind: Table + name: Error List + queries: + - query: |- + import "influxdata/influxdb/v1" + import "strings" + import "regexp" + + + from(bucket: "_tasks") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "runs" and r.status == "failed") + |> filter(fn: (r) => r["_field"] == "errorMessage" or r._field == "runID" or r._field == "name") + |> v1.fieldsAsCols() + |> group() + // |> group(columns: ["taskID", "runID", "name", "status"]) + tableOptions: + sortBy: failed + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 12 + yPos: 8 + - axes: + - base: "10" + name: x + scale: linear + - base: "2" + name: y + scale: linear + colors: + - hex: '#31C0F6' + id: fcee9956-f868-405d-8f3b-863ab456067d + name: Nineteen Eighty Four + type: scale + - hex: '#A500A5' + id: 613dba11-30be-4d41-91d8-ffeac3ee9458 + name: Nineteen Eighty Four + type: scale + - hex: '#FF7E27' + id: 4b074fc6-f802-4b2d-aad3-fe0624970891 + name: Nineteen Eighty Four + type: scale + geom: line + height: 4 + kind: Xy + name: Error Rates Over Time (All tasks) + position: overlaid + queries: + - query: |- + import "strings" + import "regexp" + + + from(bucket: "_tasks") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "runs") + |> filter(fn: (r) => r["_field"] == "logs") + |> map(fn: (r) => { + ind = regexp.findStringIndex(r: /name: \\\\\\".+?\\\\\\"/, v: r._value) + finalstring = strings.trimSuffix(v: strings.trimPrefix(v: strings.substring(end: ind[1], start: ind[0], v: r._value ), prefix: "name: \\\\\\\""), suffix: "\\\\\\\"") + return {r with name: finalstring } + } + ) + |> group(columns: ["status"]) + |> aggregateWindow(every: 1h, fn: count) + |> pivot(rowKey:["_time"], columnKey: ["status"], valueColumn: "_value") + |> filter(fn: (r) => r.failed + r.success != 0) + |> map(fn: (r)=> ({r with _value: float(v: r.failed)/float(v: r.success+r.failed) })) + width: 12 + xCol: _time + yCol: _value + yPos: 11 + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + - hex: '#513CC6' + id: c749e60f-b9a3-436b-996a-d3eca0dc87b7 + name: planet + type: text + value: 95 + fieldOptions: + - displayName: _start + fieldName: _start + - displayName: _stop + fieldName: _stop + - displayName: name + fieldName: name + visible: true + - displayName: taskID + fieldName: taskID + - displayName: _time + fieldName: _time + visible: true + - displayName: runID + fieldName: runID + - displayName: scheduledFor + fieldName: scheduledFor + visible: true + - displayName: requestedAt + fieldName: requestedAt + visible: true + - displayName: startedAt + fieldName: startedAt + visible: true + - displayName: finishedAt + fieldName: finishedAt + visible: true + height: 4 + kind: Table + name: Last Successful Run Per Task + queries: + - query: |- + import "influxdata/influxdb/v1" + + import "strings" + import "regexp" + + + from(bucket: "_tasks") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "runs") + |> filter(fn: (r) => r["status"] == "success") + |> group(columns: ["taskID"]) + |> v1.fieldsAsCols() + |> map(fn: (r) => { + ind = regexp.findStringIndex(r: /name: \\\\\\".+?\\\\\\"/, v: r.logs) + finalstring = strings.trimSuffix(v: strings.trimPrefix(v: strings.substring(end: ind[1], start: ind[0], v: r.logs ), prefix: "name: \\\\\\\""), suffix: "\\\\\\\"") + return {r with name: finalstring } + } + ) + |> group(columns: ["taskID", "name"]) + |> rename(columns: {logs: "_value"}) + |> last() + |> drop(columns: ["_value"]) + |> group() + tableOptions: + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 12 + yPos: 15 + - colors: + - hex: '#7CE490' + id: base + name: honeydew + type: text + decimalPlaces: 2 + height: 4 + kind: Single_Stat + name: Total Succeeding + queries: + - query: |- + import "strings" + import "regexp" + + from(bucket: "_tasks") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "runs") + |> filter(fn: (r) => r["_field"] == "logs") + |> filter(fn: (r) => r.status == "success") + |> group(columns: ["_start"]) + |> count() + |> rename(columns: {_start: "_time"}) + width: 5 + xPos: 4 + - colors: + - hex: '#BF3D5E' + id: base + name: ruby + type: text + decimalPlaces: 2 + height: 4 + kind: Single_Stat + name: Total Failing + queries: + - query: |- + import "strings" + import "regexp" + + from(bucket: "_tasks") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "runs") + |> filter(fn: (r) => r["_field"] == "logs") + |> filter(fn: (r) => r.status != "success") + |> group(columns: ["_start"]) + |> count() + |> rename(columns: {_start: "_time"}) + width: 3 + xPos: 9 + name: Task Summary Dashboard +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: nostalgic-kirch-1aa001 +spec: + associations: + - kind: Label + name: affectionate-jackson-9aa001 + charts: + - height: 2 + kind: Markdown + name: Name this Cell + note: |- + Explore the cardinality of the buckets in your account. + + Quickly find buckets that have high cardinality then drill down by bucket measurement and tag to understand the cause of high cardinality. + width: 12 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#31C0F6' + id: 75a22a5f-96e6-41ac-8fb0-d9d628b39311 + name: Nineteen Eighty Four + type: scale + - hex: '#A500A5' + id: ec7e8c82-2aef-4682-9c64-64298feca9cf + name: Nineteen Eighty Four + type: scale + - hex: '#FF7E27' + id: d94bc95b-d1cf-4493-b0a9-c3ccb62f9b57 + name: Nineteen Eighty Four + type: scale + geom: line + height: 4 + hoverDimension: auto + kind: Xy + legendOpacity: 1 + legendOrientationThreshold: 10 + name: Cardinality by Bucket + note: View trends over time of how cardinality is changing for each of your + buckets + position: overlaid + queries: + - query: |- + import "influxdata/influxdb" + + from(bucket: "cardinality") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r._measurement == "buckets" and r._field == "cardinality") + |> aggregateWindow(every:v.windowPeriod, fn: last) + width: 12 + xCol: _time + yCol: _value + yPos: 2 + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + fieldOptions: + - displayName: _start + fieldName: _start + - displayName: _stop + fieldName: _stop + - displayName: _time + fieldName: _time + - displayName: bucket + fieldName: bucket + visible: true + - displayName: cardinality + fieldName: _value + visible: true + - displayName: _field + fieldName: _field + - displayName: _measurement + fieldName: _measurement + height: 4 + kind: Table + name: Top Buckets + note: View the top 10 buckets with the highest cardinality + queries: + - query: |- + import "influxdata/influxdb" + + from(bucket: "cardinality") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r._measurement == "buckets" and r._field == "cardinality") + |> last() + |> group() + |> top(n:10) + tableOptions: + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 12 + yPos: 6 + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + fieldOptions: + - displayName: bucket + fieldName: bucket + visible: true + - displayName: measurement + fieldName: measurement + visible: true + - displayName: cardinality + fieldName: cardinality + visible: true + - displayName: tag + fieldName: tag + visible: true + - displayName: cardinality + fieldName: _value + visible: true + height: 4 + kind: Table + name: Cardinality by Measurment + queries: + - query: |- + import "influxdata/influxdb" + import "influxdata/influxdb/schema" + + + schema.measurements(bucket: v.bucket) + |> map(fn: (r) => { + m = r._value + return { + bucket: v.bucket, + measurement: m, + _value: (influxdb.cardinality(bucket: v.bucket, start: v.timeRangeStart, predicate: (r) => r._measurement == m) |> findRecord(idx:0, fn:(key) => true))._value + } + }) + |> sort(desc:true) + tableOptions: + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 12 + yPos: 10 + - colors: + - hex: '#ffffff' + id: base + name: white + type: text + fieldOptions: + - displayName: tag + fieldName: tag + visible: true + - displayName: cardinality + fieldName: _value + visible: true + - displayName: bucket + fieldName: bucket + visible: true + - displayName: measurement + fieldName: measurement + visible: true + height: 4 + kind: Table + name: Cardinality By Tag + queries: + - query: |- + import "influxdata/influxdb/schema" + + cardinalityByTag = (bucket, measurement) => schema.tagKeys(bucket: bucket, predicate: (r) => r._measurement == measurement) + |> filter(fn: (r) => r._value != "_start" and r._value != "_stop") + |> map( + fn: (r) => ({ + tag: r._value, + _value: (schema.tagValues(bucket: bucket, tag: r._value) + |> count() + |> findRecord(fn: (key) => true, idx: 0))._value, + }), + ) + |> group(columns: ["tag"]) + |> sum() + + cardinalityByTag(bucket: v.bucket, measurement: v.measurement) + |> group() + |> top(n: 10) + tableOptions: + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 12 + yPos: 14 + name: Cardinality Explorer diff --git a/influxdb2_operational_monitoring/readme.md b/influxdb2_operational_monitoring/readme.md new file mode 100644 index 00000000..92d19594 --- /dev/null +++ b/influxdb2_operational_monitoring/readme.md @@ -0,0 +1,83 @@ +# InfluxDB 2 Operational Monitoring + +Provided by: InfluxData + +This directory contains two InfluxDB Templates, [Operational Monitoring Template](./influxdb2_operational_monitoring.yml) and [Cardinality Now Template](./cardinality_now.yml), that can be used to monitor your InfluxDB tasks and cardinality on your already running InfluxDB 2 instance. These templates are only compatible with the Cloud version of InfluxDB 2. + +### Operational Monitoring Template + +Monitoring your tasks allows you to: +1) Verify that critical downsampling and data transforation tasks are succeeding. +2) Respond to task failures quickly. + +![Task Summary Dashboard Screenshot](img/task-summary-dashboard.png) + +Monitoring your cardinality allows you to: +1) Ensure that you're not experiencing runaway cardinality. +2) Verify that your downsampling tasks and retention policies are successfully reducing your series cardinality. + +This `Cardinality Explorer` dashboard is used by InfluxData to monitor the cardinality of InfluxData's own production monitoring InfluxDB instance. The `cardinality` bucket is populated by the `cardinality_by_bucket` task which summarizes schema data about your InfluxDB instance. +![Cardinality Explorer Screenshot](img/cardinality-explorer-dashboard1.png) +If the cardinality of a bucket is increasing exponentially, use this dashboard to drill into the cardinality of specific measurements to identify the source of runaway cardinality. +![Cardinality Explorer Screenshot](img/cardinality-explorer-dashboard2.png) + +#### Included Resources + + - 1 Bucket: `cardinality`, Forever + - 1 Task: `cardinality_by_bucket`, 1h + - 2 Labels: `operational_monitoring` + - 2 Dashboard: `Task Summary Dashboard`, `Cardinality Explorer` + +### Cardinality Now + +This is a slimmed down version of the [Operational Monitoring Template](../influxdb2_operational_monitoring). This version allows you to monitor your currently cardinality without adding additional buckets or tasks to your instance (i.e. without increasing the cardinality of your instance). + +This `Cardinality Now` dashboard can help you identify your source of runaway cardinality when you have reached your cardinality limit. +![Cardinality Explorer Screenshot](img/cardinality-explorer-dashboard2.png) +Use this dashboard to drill into the cardinality of specific measurements to identify the source of runaway cardinality. + +#### Included Resources + - 1 Labels: `cardinality` + - 1 Dashboard: `Cardinality Now` + +### Quick Install + +#### InfluxDB UI + +In the InfluxDB UI, go to Settings->Templates and enter the URL for the template you wish to apply. +- For the Operational Monitoring Template: https://raw.githubusercontent.com/influxdata/community-templates/master/influxdb2_operational_monitoring/influxdb2_operational_monitoring.yml +- For the Cardinality Now Template: https://raw.githubusercontent.com/influxdata/community-templates/master/influxdb2_operational_monitoring/influxdb2_cardinality_now.yml + +#### Influx CLI +If you have your InfluxDB credentials [configured in the CLI](https://v2.docs.influxdata.com/v2.0/reference/cli/influx/config/), you can install each template with their respective URLs: +- For the Operational Monitoring Template: + ``` + influx apply -u https://raw.githubusercontent.com/influxdata/community-templates/master/influxdb2_operational_monitoring/influxdb2_operational_monitoring.yml + ``` +- For the Cardinality Now Template: + ``` + influx apply -u https://raw.githubusercontent.com/influxdata/community-templates/master/influxdb2_operational_monitoring/influxdb2_cardinality_now.yml + ``` +### Setup Instructions + + General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. + +#### Setup Instruction for the Operational Monitoring Template + + The data for the `Task Summary Dashboard` is populated by data collected from the default `_tasks` bucket. The `Cardinality Explorer` dashboard is populated by data collected from the `cardinality` bucket. The best way to set this up is to first follow the instructions for installing any template. That will create the label, bucket, and dashboards for you. + + You must already have a task running in order to populate the `Task Summary Dashboard` with data. [Get started with InfluxDB tasks](https://docs.influxdata.com/influxdb/v2.0/process-data/get-started/) to create a task and populate the dashboard. + +#### Setup Instructions for the Cardinality Now Template + + You must at least one bucket and have written data to that bucket for the `Cardinality Now` dashboard to be populated with data. + + +## Contact + +- Author: Anais Dotis-Georgiou +- Email: anais@influxdata.com +- Github: [@Anaisdg](https://github.com/Anaisdg) +- Influx Slack: [@Anais](https://influxdata.com/slack) + + diff --git a/jboss_wildfly/readme.md b/jboss_wildfly/readme.md index bb035de9..c2784e45 100644 --- a/jboss_wildfly/readme.md +++ b/jboss_wildfly/readme.md @@ -21,20 +21,20 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf Configuration: 'jboss-wildfly-config' - - 1 Dashboards: 'jboss-wildfly' - - 1 Label: 'jboss-wildfly' - - 1 Bucket: 'jboss-wildfly' + - 1 Telegraf Configuration: 'jboss-wildfly-config' + - 1 Dashboards: 'jboss-wildfly' + - 1 Label: 'jboss-wildfly' + - 1 Bucket: 'jboss-wildfly' ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your master token to get started. - - `INFLUX_ORG` - The name of your Organization. - - `INFLUX_HOST` - The address of you InfluxDB - - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```export INFLUX_BUCKET=jboss-wildfly``` +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your master token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The address of you InfluxDB + - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```export INFLUX_BUCKET=jboss-wildfly``` In order to use this Dashboard, you need to specify the connection string to connect to the jBoss Wildfly / Jolokia instance as variable. diff --git a/jenkins/img/Dashboard-Jobs.png b/jenkins/img/Dashboard-Jobs.png new file mode 100644 index 00000000..d0ba2dfa Binary files /dev/null and b/jenkins/img/Dashboard-Jobs.png differ diff --git a/jenkins/jenkins.yml b/jenkins/jenkins.yml index 64e99bba..057948fb 100644 --- a/jenkins/jenkins.yml +++ b/jenkins/jenkins.yml @@ -17,9 +17,9 @@ spec: description: Availability via mem_total every: 5m0s level: CRIT - name: Node Health (mem_total) + name: Jenkins node health query: |- - from(bucket: "regency-aloha-one") + from(bucket: "jenkins") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r._measurement == "jenkins_node") |> filter(fn: (r) => r._field == "memory_total") @@ -37,9 +37,9 @@ spec: - kind: Label name: noshing-gagarin-8d2001 every: 15m0s - name: Job Duration Check + name: Jenkins job duration query: |- - from(bucket: "regency-aloha-one") + from(bucket: "jenkins") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r._measurement == "jenkins_job") |> filter(fn: (r) => r._field == "duration") @@ -68,10 +68,11 @@ spec: associations: - kind: Label name: noshing-gagarin-8d2001 + language: flux name: jenkinsHostnames - type: constant - values: - - ctzen-ctrl.local + query: "import \"influxdata/influxdb/v1\"\r\n\r\nv1.tagValues(\r\n bucket: v.jenkinsBucket,\r\n + \ tag: \"host\",\r\n predicate: (r) => true,\r\n)" + type: query --- apiVersion: influxdata.com/v2alpha1 kind: Variable @@ -84,8 +85,40 @@ spec: name: jenkinsBucket type: constant values: - - sgnl's Bucket - - regency-aloha-one + - jenkins +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: focused-shirley-506003 +spec: + associations: + - kind: Label + name: noshing-gagarin-8d2001 + language: flux + name: jenkinsJobNames + query: "import \"csv\"\r\nimport \"influxdata/influxdb/v1\"\r\n\r\nactual = v1.tagValues(\r\n + \ bucket: v.jenkinsBucket,\r\n tag: \"name\",\r\n predicate: (r) => true,\r\n)\r\n\r\ncsvData + = \"\r\n#datatype,string,long,string\r\n#group,false,false,false\r\n#default,,,\r\n,result,table,_value\r\n,,0,\r\n,,0,\r\n\"\r\n\r\nall = csv.from(csv: csvData)\r\nunion(tables: [all, + actual])" + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: rustling-chaplygin-d06003 +spec: + associations: + - kind: Label + name: noshing-gagarin-8d2001 + name: jenkinsExcludedJobs + selected: + - + type: map + values: + : "" + : deploy-tickstack-eprise-partial,ec2-cleanup weekly,ec2-terminator-all --- apiVersion: influxdata.com/v2alpha1 kind: Dashboard @@ -190,7 +223,7 @@ spec: from(bucket: v.jenkinsBucket) |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r._measurement == "jenkins_job") - |> filter(fn: (r) => r.host == "ctzen-ctrl.local") + |> filter(fn: (r) => r.host == v.jenkinsHostnames) |> filter(fn: (r) => r._field == "result_code") |> filter(fn: (r) => r.result == "SUCCESS" or r.result == "FAILURE") |> group(columns: ["result"]) @@ -339,7 +372,7 @@ spec: |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r._measurement == "jenkins_job") |> filter(fn: (r) => r._field == "result_code") - |> filter(fn: (r) => r.host == "ctzen-ctrl.local") + |> filter(fn: (r) => r.host == v.jenkinsHostnames) |> filter(fn: (r) => r.result == "FAILURE") |> group(columns: ["result"]) |> sort(columns: ["_time"]) @@ -356,7 +389,7 @@ spec: decimalPlaces: 2 height: 1 kind: Single_Stat - name: Volitile Job + name: Volatile Job note: no job failures reported noteOnEmpty: true queries: @@ -393,7 +426,7 @@ spec: |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r._measurement == "jenkins_job") |> filter(fn: (r) => r._field == "result_code") - |> filter(fn: (r) => r.host == "ctzen-ctrl.local") + |> filter(fn: (r) => r.host == v.jenkinsHostnames) |> filter(fn: (r) => r.result == "SUCCESS") |> group(columns: ["result"]) |> sort(columns: ["_time"]) @@ -459,7 +492,7 @@ spec: from(bucket: v.jenkinsBucket) |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r._measurement == "jenkins_node") - |> filter(fn: (r) => r.host == "ctzen-ctrl.local") + |> filter(fn: (r) => r.host == v.jenkinsHostnames) |> filter(fn: (r) => r._field == "disk_available" or r._field == "swap_total" or r._field == "swap_available" or r._field == "temp_available") shade: true width: 2 @@ -522,7 +555,7 @@ spec: from(bucket: v.jenkinsBucket) |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r._measurement == "jenkins_node") - |> filter(fn: (r) => r.host == "ctzen-ctrl.local") + |> filter(fn: (r) => r.host == v.jenkinsHostnames) |> filter(fn: (r) => r._field == "response_time") width: 3 xCol: _time @@ -539,7 +572,7 @@ spec: Recent Failed/Success Job - Shows the most recent failing or successful jobs by name - Volitile/Stable Job - Shows the most failing or successful build from the "duration" specificied in the dashboard + Volatile/Stable Job - Shows the most failing or successful build from the "duration" specified in the dashboard "Gauge" cell - Something I was messing around with. note: I wish I could use a variable as a value for "customize - maximum threshold" @@ -567,6 +600,9 @@ kind: Dashboard metadata: name: zen-taussig-127001 spec: + associations: + - kind: Label + name: noshing-gagarin-8d2001 charts: - colors: - hex: '#00C9FF' @@ -581,7 +617,7 @@ spec: from(bucket: v.jenkinsBucket) |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r._measurement == "jenkins_job") - |> filter(fn: (r) => r.name == v.jenkinJobNames) + |> filter(fn: (r) => r.name == v.jenkinsJobNames) |> keep(columns: ["name"]) |> rename(columns: {name: "_value"}) |> first() @@ -614,7 +650,7 @@ spec: |> filter(fn: (r) => r.host == v.jenkinsHostnames) |> filter(fn: (r) => r._field == "result_code") |> filter(fn: (r) => r.result == "FAILURE") - |> filter(fn: (r) => r.name == v.jenkinJobNames) + |> filter(fn: (r) => r.name == v.jenkinsJobNames) |> group(columns: ["result"]) |> count() |> yield(name: "count") @@ -648,7 +684,7 @@ spec: |> filter(fn: (r) => r.host == v.jenkinsHostnames) |> filter(fn: (r) => r._field == "result_code") |> filter(fn: (r) => r.result == "SUCCESS") - |> filter(fn: (r) => r.name == v.jenkinJobNames) + |> filter(fn: (r) => r.name == v.jenkinsJobNames) |> group(columns: ["result"]) |> count() |> yield(name: "count") @@ -686,7 +722,7 @@ spec: |> filter(fn: (r) => r._measurement == "jenkins_job") |> filter(fn: (r) => r._field == "duration") |> filter(fn: (r) => r.host == v.jenkinsHostnames) - |> filter(fn: (r) => r.name == v.jenkinJobNames) + |> filter(fn: (r) => r.name == v.jenkinsJobNames) |> map(fn: (r) => ({ r with _value: r._value / 1000 @@ -708,10 +744,478 @@ spec: name: Jenkins - Single Job --- apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: alerting-morse-fcb001 +spec: + associations: + - kind: Label + name: noshing-gagarin-8d2001 + charts: + - colors: + - hex: '#F95F53' + name: curacao + type: min + - hex: '#4ED8A0' + name: rainforest + type: max + value: 100 + decimalPlaces: 0 + height: 4 + kind: Gauge + name: Success ratio + queries: + - query: "import \"influxdata/influxdb/v1\"\r\nimport \"strings\"\r\n\r\nfrom(bucket: + v.jenkinsBucket)\r\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\r\n + \ |> filter(fn: (r) => r._measurement == \"jenkins_job\")\r\n |> + filter(fn: (r) => r.host == v.jenkinsHostnames)\r\n |> filter(fn: + (r) => \r\n if v.jenkinsJobNames == \"\" then true\r\n else + if v.jenkinsJobNames == \"\" then not contains(set: + strings.split(v: v.jenkinsExcludedJobs, t: \",\"), value: r.name)\r\n + \ else r.name == v.jenkinsJobNames\r\n )\r\n |> v1.fieldsAsCols()\r\n + \ |> duplicate(column: \"result_code\", as: \"_value\")\r\n |> group()\r\n + \ |> reduce(\r\n fn: (r, accumulator) => ({\r\n total: + accumulator.total + 1,\r\n success: accumulator.success + + (if r.result_code == 0 then 1 else 0),\r\n }),\r\n identity: + {total: 0, success: 0}\r\n ) \r\n |> map(fn: (r) => ({r with + _value: (float(v: 100 * r.success) / float(v: r.total))}))" + suffix: '%' + width: 5 + - colors: + - hex: '#4ED8A0' + id: base + name: rainforest + type: background + value: 0 + - hex: '#F95F53' + id: 2ad18e5d-818e-4e5f-aa81-c38b746dfaa6 + name: curacao + type: background + value: 1 + - hex: '#545667' + id: 28ebafd3-cf8e-4572-bf14-a02b0e543f39 + name: graphite + type: background + value: 2 + - hex: '#FFD255' + id: e510204d-b629-43c9-acad-89dbd5148287 + name: thunder + type: background + value: 3 + - hex: '#545667' + id: 0b210208-6a4c-4bcd-835c-83ee9a154b11 + name: graphite + type: background + value: 4 + decimalPlaces: 0 + fieldOptions: + - displayName: time + fieldName: _time + visible: true + - displayName: _start + fieldName: _start + visible: true + - displayName: _stop + fieldName: _stop + visible: true + - displayName: _measurement + fieldName: _measurement + visible: true + - displayName: host + fieldName: host + - displayName: job name + fieldName: name + visible: true + - displayName: port + fieldName: port + - displayName: source + fieldName: source + visible: true + - displayName: duration + fieldName: duration + - displayName: result code + fieldName: result_code + visible: true + - displayName: result + fieldName: _value + visible: true + - displayName: build + fieldName: link + visible: true + - displayName: _zlink + fieldName: _zlink + height: 4 + kind: Table + name: Last Builds + queries: + - query: "import \"influxdata/influxdb/v1\"\r\nimport \"strings\"\r\n\r\nfrom(bucket: + v.jenkinsBucket)\r\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\r\n + \ |> filter(fn: (r) => r._measurement == \"jenkins_job\")\r\n |> + filter(fn: (r) => r.host == v.jenkinsHostnames)\r\n |> filter(fn: + (r) => \r\n if v.jenkinsJobNames == \"\" then true\r\n else + if v.jenkinsJobNames == \"\" then not contains(set: + strings.split(v: v.jenkinsExcludedJobs, t: \",\"), value: r.name)\r\n + \ else r.name == v.jenkinsJobNames\r\n )\r\n |> v1.fieldsAsCols()\r\n + \ |> duplicate(column: \"result_code\", as: \"_value\")\r\n |> group()\r\n + \ |> keep(columns: [\"_time\", \"host\", \"port\", \"name\", \"result\", + \"result_code\", \"duration\"])\r\n |> map(fn: (r) => ({r with link: + \"https://\" + r.host + \":\" + r.port + \"/job/\" + r.name + \"/\"}))" + tableOptions: + sortBy: _time + verticalTimeAxis: true + timeFormat: YYYY/MM/DD HH:mm:ss + width: 11 + yPos: 4 + - height: 2 + kind: Markdown + name: Name this Cell + note: |- + --- + Last 7 days statistics + --- + width: 11 + yPos: 8 + - colors: + - hex: '#BF3D5E' + id: base + name: ruby + type: background + value: 0 + - hex: '#DC4E58' + id: 3fe676af-09e5-47ab-a32e-d1b99335c5f3 + name: fire + type: background + value: 33 + - hex: '#F48D38' + id: 94ece9fa-23bf-43aa-910f-e5e1c56cec77 + name: tiger + type: background + value: 50 + - hex: '#FFB94A' + id: 82593475-2ea4-437c-9ffe-3ab2fa57bb3c + name: pineapple + type: background + value: 75 + - hex: '#4ED8A0' + id: 8b045ebd-007e-46ff-9c1e-bd60d4bf8517 + name: rainforest + type: background + value: 100 + decimalPlaces: 0 + fieldOptions: + - displayName: name + fieldName: name + visible: true + - displayName: ratio (%) + fieldName: _value + visible: true + - displayName: success + fieldName: success + - displayName: total + fieldName: total + height: 4 + kind: Table + name: Success Rate (last 7 days) + queries: + - query: "import \"influxdata/influxdb/v1\"\r\nimport \"strings\"\r\n\r\nfrom(bucket: + v.jenkinsBucket)\r\n |> range(start: -1w)\r\n |> filter(fn: (r) + => r._measurement == \"jenkins_job\")\r\n |> filter(fn: (r) => r.host + == v.jenkinsHostnames)\r\n |> filter(fn: (r) => \r\n if v.jenkinsJobNames + == \"\" then true\r\n else if v.jenkinsJobNames == \"\" then not contains(set: strings.split(v: v.jenkinsExcludedJobs, + t: \",\"), value: r.name)\r\n else r.name == v.jenkinsJobNames\r\n + \ )\r\n |> v1.fieldsAsCols()\r\n |> duplicate(column: \"result_code\", + as: \"_value\")\r\n |> group(columns: [\"name\"])\r\n |> reduce(\r\n + \ fn: (r, accumulator) => ({\r\n total: accumulator.total + + 1,\r\n success: accumulator.success + (if r.result_code + == 0 then 1 else 0),\r\n }),\r\n identity: {total: 0, + success: 0}\r\n ) \r\n |> map(fn: (r) => ({r with _value: (float(v: + 100 * r.success) / float(v: r.total))}))\r\n |> group()" + tableOptions: + sortBy: _value + verticalTimeAxis: true + timeFormat: YYYY-MM-DD HH:mm:ss + width: 3 + yPos: 10 + - height: 2 + kind: Markdown + name: Name this Cell + note: |- + --- + Last builds of each job irrespective of time window and job filter. + --- + width: 11 + yPos: 14 + - colors: + - hex: '#F95F53' + name: curacao + type: min + - hex: '#4ED8A0' + name: rainforest + type: max + value: 100 + decimalPlaces: 0 + height: 4 + kind: Gauge + name: Success ratio (Distinct) + queries: + - query: "import \"influxdata/influxdb/v1\"\r\n\r\nfrom(bucket: v.jenkinsBucket)\r\n + \ |> range(start: -1y)\r\n |> filter(fn: (r) => r._measurement == + \"jenkins_job\")\r\n |> filter(fn: (r) => r.host == v.jenkinsHostnames)\r\n + \ |> v1.fieldsAsCols()\r\n |> duplicate(column: \"result_code\", + as: \"_value\")\r\n |> group(columns: [\"name\"])\r\n |> last()\r\n + \ |> group()\r\n |> reduce(\r\n fn: (r, accumulator) => ({\r\n + \ total: accumulator.total + 1,\r\n success: + accumulator.success + (if r.result_code == 0 then 1 else 0),\r\n }),\r\n + \ identity: {total: 0, success: 0}\r\n ) \r\n |> map(fn: + (r) => ({r with _value: (float(v: 100 * r.success) / float(v: r.total))}))" + suffix: '%' + width: 5 + yPos: 16 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#FD7A5D' + name: Delorean + type: scale + - hex: '#5F1CF2' + name: Delorean + type: scale + - hex: '#4CE09A' + name: Delorean + type: scale + geom: line + height: 4 + kind: Xy + name: Builds Duration (last 7 days) + position: overlaid + queries: + - query: "import \"influxdata/influxdb/v1\"\r\nimport \"strings\"\r\n\r\nfrom(bucket: + v.jenkinsBucket)\r\n |> range(start: -1w)\r\n |> filter(fn: (r) + => r._measurement == \"jenkins_job\")\r\n |> filter(fn: (r) => r.host + == v.jenkinsHostnames)\r\n |> filter(fn: (r) => \r\n if v.jenkinsJobNames + == \"\" then true\r\n else if v.jenkinsJobNames == \"\" then not contains(set: strings.split(v: v.jenkinsExcludedJobs, + t: \",\"), value: r.name)\r\n else r.name == v.jenkinsJobNames\r\n + \ )\r\n |> v1.fieldsAsCols()\r\n |> filter(fn: (r) => r.result_code + == 0)\r\n |> duplicate(column: \"duration\", as: \"_value\")\r\n + \ |> keep(columns: [\"name\", \"_time\", \"_value\"])" + width: 8 + xCol: _time + xPos: 3 + yCol: _value + yPos: 10 + - colors: + - hex: '#7A65F2' + id: base + name: star + type: text + value: 0 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Projects + queries: + - query: "import \"influxdata/influxdb/v1\"\nimport \"strings\"\n\nfrom(bucket: + v.jenkinsBucket)\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n + \ |> filter(fn: (r) => r._measurement == \"jenkins_job\")\n |> filter(fn: + (r) => r.host == v.jenkinsHostnames)\n |> filter(fn: (r) => \n if + v.jenkinsJobNames == \"\" then true\n else if v.jenkinsJobNames + == \"\" then not contains(set: strings.split(v: + v.jenkinsExcludedJobs, t: \",\"), value: r.name)\n else r.name + == v.jenkinsJobNames\n )\n |> v1.fieldsAsCols()\n |> duplicate(column: + \"result_code\", as: \"_value\")\n |> group()\n |> distinct(column: + \"name\")\n |> count()" + width: 3 + xPos: 5 + - colors: + - hex: '#4ED8A0' + id: base + name: rainforest + type: text + value: 0 + decimalPlaces: 0 + height: 3 + kind: Single_Stat + name: Successful Builds + queries: + - query: "import \"influxdata/influxdb/v1\"\nimport \"strings\"\n\nfrom(bucket: + v.jenkinsBucket)\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n + \ |> filter(fn: (r) => r._measurement == \"jenkins_job\")\n |> filter(fn: + (r) => r.host == v.jenkinsHostnames)\n |> filter(fn: (r) => \n if + v.jenkinsJobNames == \"\" then true\n else if v.jenkinsJobNames + == \"\" then not contains(set: strings.split(v: + v.jenkinsExcludedJobs, t: \",\"), value: r.name)\n else r.name + == v.jenkinsJobNames\n )\n |> v1.fieldsAsCols()\n |> filter(fn: + (r) => r.result_code == 0)\n |> duplicate(column: \"result_code\", + as: \"_value\")\n |> group()\n |> count()" + width: 2 + xPos: 5 + yPos: 1 + - colors: + - hex: '#4ED8A0' + id: base + name: rainforest + type: background + value: 0 + - hex: '#F95F53' + id: 2ad18e5d-818e-4e5f-aa81-c38b746dfaa6 + name: curacao + type: background + value: 1 + - hex: '#545667' + id: 28ebafd3-cf8e-4572-bf14-a02b0e543f39 + name: graphite + type: background + value: 2 + - hex: '#FFD255' + id: e510204d-b629-43c9-acad-89dbd5148287 + name: thunder + type: background + value: 3 + - hex: '#545667' + id: 0b210208-6a4c-4bcd-835c-83ee9a154b11 + name: graphite + type: background + value: 4 + decimalPlaces: 0 + fieldOptions: + - displayName: time + fieldName: _time + visible: true + - displayName: _start + fieldName: _start + visible: true + - displayName: _stop + fieldName: _stop + visible: true + - displayName: _measurement + fieldName: _measurement + visible: true + - displayName: host + fieldName: host + visible: true + - displayName: job name + fieldName: name + visible: true + - displayName: port + fieldName: port + visible: true + - displayName: source + fieldName: source + visible: true + - displayName: duration + fieldName: duration + - displayName: result code + fieldName: result_code + visible: true + - displayName: _value + fieldName: _value + visible: true + height: 4 + kind: Table + name: Last Jobs Builds (Distinct) + queries: + - query: "import \"influxdata/influxdb/v1\"\r\n\r\nfrom(bucket: v.jenkinsBucket)\r\n + \ |> range(start: -1y)\r\n |> filter(fn: (r) => r._measurement == + \"jenkins_job\")\r\n |> filter(fn: (r) => r.host == v.jenkinsHostnames)\r\n + \ |> v1.fieldsAsCols()\r\n |> duplicate(column: \"result_code\", + as: \"_value\")\r\n |> group(columns: [\"name\"])\r\n |> last()\r\n + \ |> group()\r\n |> keep(columns: [\"name\", \"_time\", \"result\", + \"result_code\", \"duration\"])" + tableOptions: + sortBy: _time + verticalTimeAxis: true + timeFormat: YYYY/MM/DD HH:mm:ss + width: 6 + xPos: 5 + yPos: 16 + - colors: + - hex: '#FFD255' + id: base + name: thunder + type: text + value: 0 + decimalPlaces: 0 + height: 3 + kind: Single_Stat + name: Unstable Builds + queries: + - query: "import \"influxdata/influxdb/v1\"\nimport \"strings\"\n\nfrom(bucket: + v.jenkinsBucket)\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n + \ |> filter(fn: (r) => r._measurement == \"jenkins_job\")\n |> filter(fn: + (r) => r.host == v.jenkinsHostnames)\n |> filter(fn: (r) => \n if + v.jenkinsJobNames == \"\" then true\n else if v.jenkinsJobNames + == \"\" then not contains(set: strings.split(v: + v.jenkinsExcludedJobs, t: \",\"), value: r.name)\n else r.name + == v.jenkinsJobNames\n )\n |> v1.fieldsAsCols()\n |> filter(fn: + (r) => r.result_code == 3)\n |> duplicate(column: \"result_code\", + as: \"_value\")\n |> group()\n |> count()" + width: 2 + xPos: 7 + yPos: 1 + - colors: + - hex: '#00C9FF' + id: base + name: laser + type: text + value: 0 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Builds + queries: + - query: "import \"influxdata/influxdb/v1\"\nimport \"strings\"\n\nfrom(bucket: + v.jenkinsBucket)\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n + \ |> filter(fn: (r) => r._measurement == \"jenkins_job\")\n |> filter(fn: + (r) => r.host == v.jenkinsHostnames)\n |> filter(fn: (r) => \n if + v.jenkinsJobNames == \"\" then true\n else if v.jenkinsJobNames + == \"\" then not contains(set: strings.split(v: + v.jenkinsExcludedJobs, t: \",\"), value: r.name)\n else r.name + == v.jenkinsJobNames\n )\n |> v1.fieldsAsCols()\n |> duplicate(column: + \"result_code\", as: \"_value\")\n |> group()\n |> count()" + width: 3 + xPos: 8 + - colors: + - hex: '#F95F53' + id: base + name: curacao + type: text + value: 0 + decimalPlaces: 0 + height: 3 + kind: Single_Stat + name: Failed Builds + queries: + - query: "import \"influxdata/influxdb/v1\"\nimport \"strings\"\n\nfrom(bucket: + v.jenkinsBucket)\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n + \ |> filter(fn: (r) => r._measurement == \"jenkins_job\")\n |> filter(fn: + (r) => r.host == v.jenkinsHostnames)\n |> filter(fn: (r) => \n if + v.jenkinsJobNames == \"\" then true\n else if v.jenkinsJobNames + == \"\" then not contains(set: strings.split(v: + v.jenkinsExcludedJobs, t: \",\"), value: r.name)\n else r.name + == v.jenkinsJobNames\n )\n |> v1.fieldsAsCols()\n |> filter(fn: + (r) => r.result_code == 1)\n |> duplicate(column: \"result_code\", + as: \"_value\")\n |> group()\n |> count()" + width: 2 + xPos: 9 + yPos: 1 + description: Jenkins jobs dashboard + name: Jenkins - Jobs +--- +apiVersion: influxdata.com/v2alpha1 kind: Telegraf metadata: name: wiggitty-huh-127001 spec: + associations: + - kind: Label + name: noshing-gagarin-8d2001 config: | # Telegraf Configuration # @@ -901,4 +1405,5 @@ spec: ############################################################################### # SERVICE INPUT PLUGINS # ############################################################################### - name: Website Monitor + description: Telegraf config for Jenkins monitoring + name: Jenkins Monitor diff --git a/jenkins/readme.md b/jenkins/readme.md index 8560142d..d19d4751 100644 --- a/jenkins/readme.md +++ b/jenkins/readme.md @@ -1,6 +1,6 @@ ## jenkins Monitoring Template -This InfluxDB Template can be used to montior a Jenkins instance +This InfluxDB Template can be used to monitor a Jenkins instance ![Website Monitoring Dashboard Screenshot](img/Dashboard.png) @@ -20,11 +20,12 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ### Included Resources - `Jenkins - High-Resolution` Dashboard (see screenshot above) +- `Jenkins - Jobs` Dashboard - `Jenkins - Single Job` Dashboard - `Jenkins` Telegraf configuration -- Variables for `jenkinJobNames`, `jenkinsBucket`, and `jenkinsHostnames`. You will need to update the value for these thru the InfluxDB UI `Settings` > `Variables` after you've installed this dashboard template +- Variables for `jenkinsJobNames`, `jenkinsBucket`, `jenkinsHostnames` and `jenkinsExcludedJobs`. You will need to update the value for these thru the InfluxDB UI `Settings` > `Variables` after you've installed this dashboard template - Bucket handled by variable `jenkinsBucket` -- Label `jenkins` applied to all resources +- Label `Jenkins` applied to all resources - `Job Duration Check` a **threshold** alert - `Node Health` a **deadman** alert based on the `mem_total` @@ -63,11 +64,19 @@ No configuration is necessary to gather Jenkins data other than having valid cre ## Customizations -Be sure to update the values for these variables: `jenkinJobNames`, `jenkinsBucket`, and `jenkinsHostnames` +Be sure to update the values for these variables: `jenkinsBucket`, `jenkinsExcludedJobs`. +These variables are populated dynamically: `jenkinsHostnames`, `jenkinJobNames` ## Contact - Author: Ray Farias - Email: ray@sudokrew.com - Github: [@sgnl](https://github.com/sgnl) -- Influx Slack: [@Ray Farias](https://influxdata.com/slack) \ No newline at end of file +- Influx Slack: [@Ray Farias](https://influxdata.com/slack) + +--- + +- Contributor: Ales Pour +- Email: ales.pour@bonitoo.io +- Github: [@alespour](https://github.com/alespour) +- Influx Slack: [@Ales Pour](https://influxdata.com/slack) diff --git a/kafka/readme.md b/kafka/readme.md index b9d93d29..6186f48d 100644 --- a/kafka/readme.md +++ b/kafka/readme.md @@ -126,268 +126,269 @@ Zookeeper metrics: https://github.com/influxdata/telegraf/tree/master/plugins/in
kafka_controller -* Tags - * jolokia_agent_url -* Fields - * ActiveControllerCount - * ControllerState - * GlobalPartitionCount - * GlobalTopicCount - * OfflinePartitionsCount - * PreferredReplicaImbalanceCount - * ReplicasIneligibleToDeleteCount - * ReplicasToDeleteCount - * TopicsIneligibleToDeleteCount - * TopicsToDeleteCount + * Tags + * jolokia_agent_url + * Fields + * ActiveControllerCount + * ControllerState + * GlobalPartitionCount + * GlobalTopicCount + * OfflinePartitionsCount + * PreferredReplicaImbalanceCount + * ReplicasIneligibleToDeleteCount + * ReplicasToDeleteCount + * TopicsIneligibleToDeleteCount + * TopicsToDeleteCount +
kafka_network -* Tags - * jolokia_agent_url - * request -* Fields - * FENCED_LEADER_EPOCH - * LEADER_NOT_AVAILABLE - * NONE + * Tags + * jolokia_agent_url + * request + * Fields + * FENCED_LEADER_EPOCH + * LEADER_NOT_AVAILABLE + * NONE +
kafka_partition -* Tags - * jolokia_agent_url - * partition - * topic -* Fields - * LogEndOffset - * LogStartOffset - * NumLogSegments - * Size - * UnderReplicatedPartitions + * Tags + * jolokia_agent_url + * partition + * topic + * Fields + * LogEndOffset + * LogStartOffset + * NumLogSegments + * Size + * UnderReplicatedPartitions
kafka_replica_manager -* Tags - * jolokia_agent_url -* Fields - * FailedIsrUpdatesPerSec - * IsrExpandsPerSec - * IsrShrinksPerSec - * LeaderCount - * PartitionCount - * UnderReplicatedPartitions + * Tags + * jolokia_agent_url + * Fields + * FailedIsrUpdatesPerSec + * IsrExpandsPerSec + * IsrShrinksPerSec + * LeaderCount + * PartitionCount + * UnderReplicatedPartitions +
kafka_requests -* Tags - * jolokia_agent_url - * request -* Fields - * Bytes50thPercentile - * Bytes75thPercentile - * Bytes95thPercentile - * Bytes98thPercentile - * Bytes99thPercentile - * Bytes999thPercentile - * BytesCount - * BytesMax - * BytesMean - * BytesMin - * BytesStdDev - * Count - * QueueTime50thPercentile - * QueueTime75thPercentile - * QueueTime95thPercentile - * QueueTime98thPercentile - * QueueTime99thPercentile - * Queuetime999thPercentile - * QueueTimeCount - * QueueTimeMax - * QueueTimeMean - * QueueTimeMin - * QueueTimeStdDev -
- -
-kafka_topics + * Tags + * jolokia_agent_url + * request + * Fields + * Bytes50thPercentile + * Bytes75thPercentile + * Bytes95thPercentile + * Bytes98thPercentile + * Bytes99thPercentile + * Bytes999thPercentile + * BytesCount + * BytesMax + * BytesMean + * BytesMin + * BytesStdDev + * Count + * QueueTime50thPercentile + * QueueTime75thPercentile + * QueueTime95thPercentile + * QueueTime98thPercentile + * QueueTime99thPercentile + * Queuetime999thPercentile + * QueueTimeCount + * QueueTimeMax + * QueueTimeMean + * QueueTimeMin + * QueueTimeStdDev -* Tags - * jolokia_agent_url - * topic -* Fields - * BytesInPerSec - * MessagesInPerSec - * ProduceMessageConversionsPerSec - * TotalFetchRequestsPerSec - * TotalProduceRequestsPerSec
kafka_topics -* Tags - * jolokia_agent_url - * topic -* Fields - * BytesInPerSec - * MessagesInPerSec - * ProduceMessageConversionsPerSec - * TotalFetchRequestsPerSec - * TotalProduceRequestsPerSec + * Tags + * jolokia_agent_url + * topic + * Fields + * BytesInPerSec + * MessagesInPerSec + * ProduceMessageConversionsPerSec + * TotalFetchRequestsPerSec + * TotalProduceRequestsPerSec +
kafka_consumer -* Tags - * jolokia_agent_url - * client-id - * partition - * topic -* Fields - * bytes-consumed-rate - * fetch-rate_fetch-rate - * records-consumed-rate - * records-lag-avg_records-lag-avg + * Tags + * jolokia_agent_url + * client-id + * partition + * topic + * Fields + * bytes-consumed-rate + * fetch-rate_fetch-rate + * records-consumed-rate + * records-lag-avg_records-lag-avg +
kafka_producer -* Tags - * jolokia_agent_url - * client-id -* Fields - * batch-size-avg - * compression-rate-avg - * io-wait-ratio - * outgoing-byte-rate - * response-rate - * request-latency-avg - * request-rate + * Tags + * jolokia_agent_url + * client-id + * Fields + * batch-size-avg + * compression-rate-avg + * io-wait-ratio + * outgoing-byte-rate + * response-rate + * request-latency-avg + * request-rate +
zookeeper -* Tags - * port - * server - * state -* Fields - * approximate_data_size - * avg_latency - * aphemerals_count - * max_file_descriptor_count - * max_latency - * min_latency - * num_alive_connections - * open_file_descriptor_count - * outstanding_requests - * packets_received - * packets_sent - * version - * watch_count - * znode_count + * Tags + * port + * server + * state + * Fields + * approximate_data_size + * avg_latency + * aphemerals_count + * max_file_descriptor_count + * max_latency + * min_latency + * num_alive_connections + * open_file_descriptor_count + * outstanding_requests + * packets_received + * packets_sent + * version + * watch_count + * znode_count +
jvm_runtime -* Tags - * jolokia_agent_url -* Fields - * SpecVersion - * StartTime - * Uptime - * VmName + * Tags + * jolokia_agent_url + * Fields + * SpecVersion + * StartTime + * Uptime + * VmName +
jvm_os -* Tags - * jolokia_agent_url -* Fields - * AvailableProcessors - * CommittedVirtualMemorySize - * FreePhysicalMemorySize - * Name - * ProcessCpuLoad - * SystemCpuLoad - * SystemLoadAverage - * TotalPhysicalMemorySize - * Version + * Tags + * jolokia_agent_url + * Fields + * AvailableProcessors + * CommittedVirtualMemorySize + * FreePhysicalMemorySize + * Name + * ProcessCpuLoad + * SystemCpuLoad + * SystemLoadAverage + * TotalPhysicalMemorySize + * Version +
jvm_mem -* Tags - * jolokia_agent_url -* Fields - * HeapMemoryUsage_committed - * HeapMemoryUsage_init - * HeapMemoryUsage_max - * HeapMemoryUsage_used - * NonHeapMemoryUsage_committed - * NonHeapMemoryUsage_init - * NonHeapMemoryUsage_max - * NonHeapMemoryUsage_used + * Tags + * jolokia_agent_url + * Fields + * HeapMemoryUsage_committed + * HeapMemoryUsage_init + * HeapMemoryUsage_max + * HeapMemoryUsage_used + * NonHeapMemoryUsage_committed + * NonHeapMemoryUsage_init + * NonHeapMemoryUsage_max + * NonHeapMemoryUsage_used +
jvm_threading -* Tags - * jolokia_agent_url -* Fields - * DaemonThreadCount - * PeakThreadCount - * ThreadCount - * TotalStartedThreadCount + * Tags + * jolokia_agent_url + * Fields + * DaemonThreadCount + * PeakThreadCount + * ThreadCount + * TotalStartedThreadCount +
java_gc -* Tags - * jolokia_agent_url - * name -* Fields - * CollectionCount - * CollectionTime + * Tags + * jolokia_agent_url + * name + * Fields + * CollectionCount + * CollectionTime +
java_classes -* Tags - * jolokia_agent_url -* Fields - * LoadedClassCount - * TotalLoadedClassCount - * UnloadedClassCount + * Tags + * jolokia_agent_url + * Fields + * LoadedClassCount + * TotalLoadedClassCount + * UnloadedClassCount +
java_mem_pool -* Tags - * jolokia_agent_url - * name -* Fields - * Usage_Usage_committed - * Usage_Usage_init - * Usage_Usage_max - * Usage_Usage_used + * Tags + * jolokia_agent_url + * name + * Fields + * Usage_Usage_committed + * Usage_Usage_init + * Usage_Usage_max + * Usage_Usage_used +
## Contact diff --git a/minio/readme.md b/minio/readme.md index ddf2a60f..4fe12c7f 100644 --- a/minio/readme.md +++ b/minio/readme.md @@ -21,9 +21,9 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Dashboards: 'minio' - - 1 Label: 'minio' - - 1 Bucket: 'minio' + - 1 Dashboards: 'minio' + - 1 Label: 'minio' + - 1 Bucket: 'minio' ## Setup Instructions diff --git a/mongodb/readme.md b/mongodb/readme.md index f1a429fa..cbd0b720 100644 --- a/mongodb/readme.md +++ b/mongodb/readme.md @@ -21,20 +21,20 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf Configuration: 'mongodb-config' - - 1 Dashboards: 'MongoDB' - - 1 bucket: 'mongodb' - - 1 label: 'mongodb' + - 1 Telegraf Configuration: 'mongodb-config' + - 1 Dashboards: 'MongoDB' + - 1 bucket: 'mongodb' + - 1 label: 'mongodb' ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_HOST` - Your InfluxDB host (ex:http://localhost:9999) - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. - - `INFLUX_BUCKET` - The name of the bucket you will store the data. +Telegraf Configuration requires the following environment variables + - `INFLUX_HOST` - Your InfluxDB host (ex:http://localhost:9999) + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_BUCKET` - The name of the bucket you will store the data. In order to use this Dashboard, you need to specify the string connection to MongoDB instance as variable. Ex: diff --git a/mssql/readme.md b/mssql/readme.md index a0bde4ed..cf238672 100644 --- a/mssql/readme.md +++ b/mssql/readme.md @@ -21,21 +21,21 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf Configuration: 'mssql-config' - - 1 Dashboards: 'MSSQL' - - 1 Label: 'mssql' - - 1 Bucket: 'mssql' - - 1 Variable: 'mssqlType' + - 1 Telegraf Configuration: 'mssql-config' + - 1 Dashboards: 'MSSQL' + - 1 Label: 'mssql' + - 1 Bucket: 'mssql' + - 1 Variable: 'mssqlType' ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. - - `INFLUX_HOST` - The address of you InfluxDB - - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```INFLUX_BUCKET=mssql``` +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The address of you InfluxDB + - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```INFLUX_BUCKET=mssql``` ## Additional Instructions For Azure DB diff --git a/mysql_mariadb/readme.md b/mysql_mariadb/readme.md index 06e35216..b5ff0980 100644 --- a/mysql_mariadb/readme.md +++ b/mysql_mariadb/readme.md @@ -21,20 +21,20 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf Configuration: 'mysql-mariadb' - - 1 Dashboards: 'MySQL - MariaDB' - - 1 Label: 'mysql' - - 1 Bucket: 'mariadb' + - 1 Telegraf Configuration: 'mysql-mariadb' + - 1 Dashboards: 'MySQL - MariaDB' + - 1 Label: 'mysql' + - 1 Bucket: 'mariadb' ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. - - `INFLUX_HOST` - The address of you InfluxDB - - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```export INFLUX_BUCKET=mariadb``` +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The address of you InfluxDB + - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```export INFLUX_BUCKET=mariadb``` In order to use this Dashboard, you need to specify the connection string to the mySQL/MariaDB instance as variable. The same needs to define user and password (read only recommended) diff --git a/network_interface_performance/network_interface_performance.yml b/network_interface_performance/network_interface_performance.yml index 45c5336c..fcb5094e 100644 --- a/network_interface_performance/network_interface_performance.yml +++ b/network_interface_performance/network_interface_performance.yml @@ -460,7 +460,7 @@ spec: ## Multiple URLs can be specified for a single cluster, only ONE of the ## urls will be written to each interval. ## ex: urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"] - urls = ["$INFLUX_URL"] + urls = ["$INFLUX_HOST"] ## Token for authentication. token = "$INFLUX_TOKEN" ## Organization is the name of the organization you wish to write to; must exist. diff --git a/network_interface_performance/readme.md b/network_interface_performance/readme.md index 1b358648..a060b891 100644 --- a/network_interface_performance/readme.md +++ b/network_interface_performance/readme.md @@ -34,7 +34,7 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - `INFLUX_ORG` - The name of your Organization (this will be your email address on the InfluxDB Cloud free tier) - - `INFLUX_URL` - The URL of your InfluxDB host (this can your localhost, a remote instance, or InfluxDB Cloud) + - `INFLUX_HOST` - The URL of your InfluxDB host (this can your localhost, a remote instance, or InfluxDB Cloud) You **MUST** set these environment variables before running Telegraf using something similar to the following commands diff --git a/nextcloud/nextcloud.yml b/nextcloud/nextcloud.yml new file mode 100644 index 00000000..1f53043a --- /dev/null +++ b/nextcloud/nextcloud.yml @@ -0,0 +1,787 @@ +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: distracted-taussig-630001 +spec: + color: '#066fc5' + name: nextcloud +--- +apiVersion: influxdata.com/v2alpha1 +kind: Bucket +metadata: + name: bettering-leakey-630005 +spec: + associations: + - kind: Label + name: distracted-taussig-630001 + name: nextcloud +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: upbeat-blackburn-230001 +spec: + associations: + - kind: Label + name: distracted-taussig-630001 + charts: + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + - hex: '#34BB55' + name: Solid Green + type: scale + - hex: '#34BB55' + name: Solid Green + type: scale + - hex: '#34BB55' + name: Solid Green + type: scale + decimalPlaces: 2 + height: 2 + kind: Single_Stat_Plus_Line + name: CPU Load + position: overlaid + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_system_cpuload_0") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 2 + xCol: _time + yCol: _value + - height: 1 + kind: Markdown + name: Name this Cell + note: '## Storage' + width: 1 + yPos: 2 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Local Drives + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_storage_num_storages_local") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 1 + yPos: 3 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 2 + kind: Single_Stat + name: Apps Installed + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_system_apps_num_installed") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 1 + yPos: 4 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Drives + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_storage_num_storages") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 1 + xPos: 1 + yPos: 2 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Others + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_storage_num_users") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 1 + xPos: 1 + yPos: 3 + - colors: + - hex: '#00C9FF' + name: laser + type: text + - hex: '#DC4E58' + name: fire + type: text + value: 1 + decimalPlaces: 0 + height: 2 + kind: Single_Stat + name: Updates Available + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_system_apps_num_updates_available") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 1 + xPos: 1 + yPos: 4 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 2 + kind: Single_Stat + name: Total Memory (GB) + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_system_mem_total") + |> map(fn: (r) => ({r with _value: r._value / 1024.00})) + suffix: ' ' + width: 2 + xPos: 2 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + - hex: '#DC4E58' + name: Solid Red + type: scale + - hex: '#DC4E58' + name: Solid Red + type: scale + - hex: '#DC4E58' + name: Solid Red + type: scale + decimalPlaces: 0 + height: 2 + kind: Single_Stat_Plus_Line + name: Files + position: overlaid + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_storage_num_files") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + shade: true + width: 3 + xCol: _time + xPos: 2 + yCol: _value + yPos: 2 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + - hex: '#74D495' + name: Atlantis + type: scale + - hex: '#3F3FBA' + name: Atlantis + type: scale + - hex: '#FF4D9E' + name: Atlantis + type: scale + decimalPlaces: 0 + height: 2 + kind: Single_Stat_Plus_Line + name: Users Active Last five minutes + position: overlaid + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_activeUsers_last5minutes") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + shade: true + width: 2 + xCol: _time + xPos: 2 + yCol: _value + yPos: 4 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + - hex: '#DA6FF1' + name: Ectoplasm + type: scale + - hex: '#00717A' + name: Ectoplasm + type: scale + - hex: '#ACFF76' + name: Ectoplasm + type: scale + decimalPlaces: 0 + height: 2 + kind: Single_Stat_Plus_Line + name: Free Memory (GB) + position: overlaid + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_system_mem_free") + |> map(fn: (r) => ({r with _value: r._value / 1024.00})) + shade: true + width: 2 + xCol: _time + xPos: 4 + yCol: _value + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + - hex: '#74D495' + name: Atlantis + type: scale + - hex: '#3F3FBA' + name: Atlantis + type: scale + - hex: '#FF4D9E' + name: Atlantis + type: scale + decimalPlaces: 0 + height: 2 + kind: Single_Stat_Plus_Line + name: Users Active Last Hour + position: overlaid + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_activeUsers_last1hour") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + shade: true + width: 2 + xCol: _time + xPos: 4 + yCol: _value + yPos: 4 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + - hex: '#FDC44F' + name: Cthulhu + type: scale + - hex: '#007C76' + name: Cthulhu + type: scale + - hex: '#8983FF' + name: Cthulhu + type: scale + decimalPlaces: 0 + height: 2 + kind: Single_Stat_Plus_Line + name: Free Space (GB) + position: overlaid + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_system_freespace") + |> map(fn: (r) => ({r with _value: r._value / 1000024.00})) + shade: true + width: 3 + xCol: _time + xPos: 5 + yCol: _value + yPos: 2 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#31C0F6' + name: Nineteen Eighty Four + type: scale + - hex: '#A500A5' + name: Nineteen Eighty Four + type: scale + - hex: '#FF7E27' + name: Nineteen Eighty Four + type: scale + geom: line + height: 2 + kind: Xy + name: Swap Total + position: overlaid + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_system_swap_total") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 2 + xCol: _time + xPos: 6 + yCol: _value + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + - hex: '#8F8AF4' + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#A51414' + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#F4CF31' + name: Do Androids Dream of Electric Sheep? + type: scale + decimalPlaces: 0 + height: 2 + kind: Single_Stat_Plus_Line + name: Users Active Last 24 hours + position: overlaid + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_activeUsers_last24hours") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + shade: true + width: 2 + xCol: _time + xPos: 6 + yCol: _value + yPos: 4 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#31C0F6' + name: Nineteen Eighty Four + type: scale + - hex: '#A500A5' + name: Nineteen Eighty Four + type: scale + - hex: '#FF7E27' + name: Nineteen Eighty Four + type: scale + geom: line + height: 2 + kind: Xy + name: Swap Free + position: overlaid + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_system_swap_free") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 2 + xCol: _time + xPos: 8 + yCol: _value + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 2 + height: 2 + kind: Single_Stat_Plus_Line + name: Response Time + position: overlaid + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "response_time") + |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false) + |> yield(name: "last") + shade: true + width: 2 + xCol: _time + xPos: 8 + yCol: _value + yPos: 2 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Database Size (MB) + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_server_database_size") + |> map(fn: (r) => ({r with _value: r._value / 1000024.0})) + width: 2 + xPos: 8 + yPos: 4 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: HTTP - Status Code + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_meta_statuscode") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 2 + xPos: 8 + yPos: 5 + - height: 1 + kind: Markdown + name: Name this Cell + note: '## Sharing Stats' + width: 2 + xPos: 10 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Received + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 1 + xPos: 10 + yPos: 1 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Links + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_shares_num_shares_link") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 1 + xPos: 10 + yPos: 2 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: By email + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_shares_num_shares_mail") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 1 + xPos: 10 + yPos: 3 + - colors: + - hex: '#7CE490' + name: honeydew + type: text + - hex: '#BF3D5E' + name: ruby + type: text + value: 1 + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Without password + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_shares_num_shares_link_no_password") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 2 + xPos: 10 + yPos: 4 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: PHP Max Exec Time + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_server_php_max_execution_time") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 2 + xPos: 10 + yPos: 5 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Sent + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_shares_num_fed_shares_sent") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 1 + xPos: 11 + yPos: 1 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Group + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_shares_num_shares_groups") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 1 + xPos: 11 + yPos: 2 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Room (Chats) + queries: + - query: |- + from(bucket: "nextcloud") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "httpjson_nextcloud_stats") + |> filter(fn: (r) => r["_field"] == "ocs_data_nextcloud_shares_num_shares_room") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 1 + xPos: 11 + yPos: 3 + name: Nextcloud +--- +apiVersion: influxdata.com/v2alpha1 +kind: Telegraf +metadata: + name: nextcloud-config +spec: + associations: + - kind: Label + name: distracted-taussig-630001 + config: | + [[outputs.influxdb_v2]] + ## The URLs of the InfluxDB cluster nodes. + ## + ## Multiple URLs can be specified for a single cluster, only ONE of the + ## urls will be written to each interval. + ## urls exp: http://127.0.0.1:8086 + urls = ["$INFLUX_HOST"] + + ## Token for authentication. + token = "$INFLUX_TOKEN" + + ## Organization is the name of the organization you wish to write to; must exist. + organization = "$INFLUX_ORG" + + ## Destination bucket to write into. + bucket = "nextcloud" + + [agent] + interval = "5m" + + [[inputs.httpjson]] + ## NOTE This plugin only reads numerical measurements, strings and booleans + ## will be ignored. + ## Name for the service being polled. Will be appended to the name of the + ## measurement e.g. "httpjson_webserver_stats". + ## + ## Deprecated (1.3.0): Use name_override, name_suffix, name_prefix instead. + name = "nextcloud_stats" + + ## URL of each server in the service's cluster + servers = [ + "$NEXTCLOUDURL/ocs/v2.php/apps/serverinfo/api/v1/info?format=json", + ] + ## Set response_timeout (default 5 seconds) + response_timeout = "10s" + + ## HTTP method to use: GET or POST (case-sensitive) + method = "GET" + + ## Tags to extract from top-level of JSON server response. + # tag_keys = [ + # "my_tag_1", + # "my_tag_2" + # ] + + ## Optional TLS Config + # tls_ca = "/etc/telegraf/ca.pem" + # tls_cert = "/etc/telegraf/cert.pem" + # tls_key = "/etc/telegraf/key.pem" + + ## Use TLS but skip chain & host verification + # insecure_skip_verify = false + + ## HTTP Request Parameters (all values must be strings). For "GET" requests, data + ## will be included in the query. For "POST" requests, data will be included + ## in the request body as "x-www-form-urlencoded". + + # [inputs.httpjson.parameters] + # event_type = "cpu_spike" + # threshold = "0.75" + + ## HTTP Request Headers (all values must be strings). + # [inputs.httpjson.headers] + # X-Auth-Token = "" + # apiVersion = "v1" \ No newline at end of file diff --git a/nextcloud/readme.md b/nextcloud/readme.md new file mode 100644 index 00000000..73bf4d9a --- /dev/null +++ b/nextcloud/readme.md @@ -0,0 +1,50 @@ +# Nextcloud template for InfluxDB v2 + +Provided by: Ignacio Van Droogenbroeck + +This Dashboard offers you information about your Nextcloud instance. CPU Load, Storage, Users, Sharing files stats and more. + +![Dashboard Screenshot](screenshot.png) + +### Quick Install + +#### InfluxDB UI + +In the InfluxDB UI, go to Settings->Templates and enter this URL: https://raw.githubusercontent.com/influxdata/community-templates/master/nextcloud/nextcloud.yml + +#### Influx CLI +If you have your InfluxDB credentials [configured in the CLI](https://v2.docs.influxdata.com/v2.0/reference/cli/influx/config/), you can install this template with: + +``` +influx apply -u https://raw.githubusercontent.com/influxdata/community-templates/master/nextcloud/nextcloud.yml +``` + +## Included Resources + + - 1 Telegraf Configuration: 'nextcloud' + - 1 Dashboards: 'nextcloud' + - 1 Label: 'nextcloud' + - 1 Bucket: 'nextcloud' + +## Setup Instructions + +General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. + +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The address of you InfluxDB + +In order to use this Dashboard, you need to pass as variable the information about your user and password and Nextcloud server. + +ex: ```$ export NEXTCLOUDURL=https://user:password@your-server.com``` + +## Contact + +Author: Ignacio Van Droogenbroeck + +Email: ignacio[at]vandroogenbroeck[dot]net + +Github and Gitlab user: @xe-nvdk + +Influx Slack: Ignacio Van Droogenbroeck diff --git a/nextcloud/screenshot.png b/nextcloud/screenshot.png new file mode 100644 index 00000000..1a0a8bcb Binary files /dev/null and b/nextcloud/screenshot.png differ diff --git a/postgresql/readme.md b/postgresql/readme.md index e90277f1..60ea1c53 100644 --- a/postgresql/readme.md +++ b/postgresql/readme.md @@ -21,20 +21,20 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf Configuration: 'postgres-config - - 1 Dashboards: 'Postgres' - - 1 Label: 'postgres' - - 1 Bucket: 'postgres' + - 1 Telegraf Configuration: 'postgres-config + - 1 Dashboards: 'Postgres' + - 1 Label: 'postgres' + - 1 Bucket: 'postgres' ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. - - `INFLUX_HOST` - The address of you InfluxDB - - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```export INFLUX_BUCKET=postgres``` +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The address of you InfluxDB + - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```export INFLUX_BUCKET=postgres``` In order to use this Dashboard, you need to specify the connection string to the Postgres instance as variable. The same needs to define user and password (read only recommended) diff --git a/raspberry-pi/img/raspberry_system_dashboard.png b/raspberry-pi/img/raspberry_system_dashboard.png new file mode 100644 index 00000000..02d26d2a Binary files /dev/null and b/raspberry-pi/img/raspberry_system_dashboard.png differ diff --git a/raspberry-pi/raspberry-pi-system.yml b/raspberry-pi/raspberry-pi-system.yml new file mode 100755 index 00000000..f7c64767 --- /dev/null +++ b/raspberry-pi/raspberry-pi-system.yml @@ -0,0 +1,538 @@ +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: eloquent-lovelace-776001 +spec: + color: '#7A65F2' + name: Linux System Template +--- +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: tender-wu-ecc001 +spec: + color: '#E30B5D' + name: raspberry-pi +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: gifted-chatterjee-776004 +spec: + associations: + - kind: Label + name: eloquent-lovelace-776001 + - kind: Label + name: tender-wu-ecc001 + language: flux + name: bucket + query: |- + buckets() + |> filter(fn: (r) => r.name !~ /^_/) + |> rename(columns: {name: "_value"}) + |> keep(columns: ["_value"]) + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: trusting-carson-b76004 +spec: + associations: + - kind: Label + name: eloquent-lovelace-776001 + - kind: Label + name: tender-wu-ecc001 + language: flux + name: linux_host + query: |- + import "influxdata/influxdb/v1" + v1.measurementTagValues(bucket: v.bucket, measurement: "cpu", tag: "host") + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: objective-booth-2cc001 +spec: + associations: + - kind: Label + name: tender-wu-ecc001 + charts: + - height: 1 + kind: Markdown + name: Name this Cell + note: | + The Raspberry Pi dashboard gives you an overview of Raspbian system metrics with metrics from `system`, `temp`, `mem`, `diskio`, `swap` and `net` measurements. See the [Telegraf Documentation](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/system) for help configuring these plugins. + width: 12 + - colors: + - hex: '#00C9FF' + name: laser + type: text + height: 1 + kind: Single_Stat + name: System Uptime + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r._measurement == "system") + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r._field == "uptime") + |> last() + |> toFloat() + |> map(fn: (r) => ({r with _value: r._value / 86400.0})) + |> yield(name: "last") + suffix: ' days' + width: 2 + yPos: 1 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + suffix: '%' + - base: "10" + name: y2 + scale: linear + geom: line + height: 3 + kind: Xy + name: Disk Usage + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r._measurement == "disk") + |> filter(fn: (r) => r._field == "used_percent") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 3 + xCol: _time + yCol: _value + yPos: 2 + - axes: + - base: "10" + name: x + scale: linear + - base: "2" + label: Bytes + name: y + scale: linear + - base: "10" + name: y2 + scale: linear + geom: line + height: 3 + kind: Xy + name: Disk IO + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r._measurement == "diskio") + |> filter(fn: (r) => r._field == "read_bytes" or r._field == "write_bytes") + |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false) + |> derivative(unit: 1s, nonNegative: false) + |> yield(name: "derivative") + width: 3 + xCol: _time + yCol: _value + yPos: 5 + - colors: + - hex: '#4ED8A0' + name: rainforest + type: background + - hex: '#4591ED' + name: ocean + type: background + value: 40 + - hex: '#4ED8A0' + name: rainforest + type: background + value: 60 + - hex: '#FFD255' + name: thunder + type: background + value: 80 + - hex: '#DC4E58' + name: fire + type: background + value: 120 + decimalPlaces: 1 + height: 1 + kind: Single_Stat + name: CPU Temp + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "temp") + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r["_field"] == "temp") + |> last() + |> yield(name: "last") + suffix: ' °C' + width: 1 + xPos: 2 + yPos: 1 + - colors: + - hex: '#00C9FF' + name: laser + type: background + - hex: '#4591ED' + name: ocean + type: background + value: 40 + - hex: '#4ED8A0' + name: rainforest + type: background + value: 60 + - hex: '#FFD255' + name: thunder + type: background + value: 80 + - hex: '#DC4E58' + name: fire + type: background + value: 120 + decimalPlaces: 1 + height: 1 + kind: Single_Stat + name: GPU Temp + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r["_measurement"] == "gpu_temperature") + |> filter(fn: (r) => r["_field"] == "value") + |> last() + |> yield(name: "last") + suffix: ' °C' + width: 1 + xPos: 3 + yPos: 1 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + suffix: '%' + - base: "10" + name: y2 + scale: linear + geom: line + height: 3 + kind: Xy + name: CPU Usage + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r._field == "usage_user" or r._field == "usage_system" or r._field == "usage_idle") + |> filter(fn: (r) => r.cpu == "cpu-total") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 3 + xCol: _time + xPos: 3 + yCol: _value + yPos: 2 + - axes: + - base: "10" + name: x + scale: linear + - base: "2" + label: Bytes + name: y + scale: linear + - base: "10" + name: y2 + scale: linear + geom: line + height: 3 + kind: Xy + name: Network + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r._measurement == "net") + |> filter(fn: (r) => r._field == "bytes_recv" or r._field == "bytes_sent") + |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false) + |> derivative(unit: 1s, nonNegative: false) + |> yield(name: "derivative") + width: 3 + xCol: _time + xPos: 3 + yCol: _value + yPos: 5 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: nCPUs + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r._measurement == "system") + |> filter(fn: (r) => r._field == "n_cpus") + |> last() + |> yield(name: "last") + suffix: ' ' + width: 1 + xPos: 4 + yPos: 1 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: System Load + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart) + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r._measurement == "system") + |> filter(fn: (r) => r._field == "load1") + |> last() + |> yield(name: "mean") + width: 2 + xPos: 5 + yPos: 1 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + label: Load + name: y + scale: linear + - base: "10" + name: y2 + scale: linear + geom: line + height: 3 + kind: Xy + name: System Load + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r._measurement == "system") + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r._field == "load1" or r._field == "load5" or r._field == "load15") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 3 + xCol: _time + xPos: 6 + yCol: _value + yPos: 2 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + - base: "10" + name: y2 + scale: linear + geom: line + height: 3 + kind: Xy + name: Processes + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r._measurement == "processes") + |> filter(fn: (r) => r._field == "running" or r._field == "blocked" or r._field == "idle" or r._field == "unknown") + |> aggregateWindow(every: v.windowPeriod, fn: max) + |> yield(name: "max") + width: 3 + xCol: _time + xPos: 6 + yCol: _value + yPos: 5 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 2 + height: 1 + kind: Single_Stat + name: Total Memory + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r._measurement == "mem") + |> filter(fn: (r) => r._field == "total") + |> last() + |> map(fn: (r) => ({r with _value: float(v: r._value) / 1024.0 / 1024.0 / 1024.0})) + |> yield(name: "last") + suffix: ' GB' + width: 2 + xPos: 7 + yPos: 1 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + suffix: '%' + - base: "10" + name: y2 + scale: linear + colors: + - hex: '#00C9FF' + name: laser + type: text + - hex: '#8F8AF4' + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#A51414' + name: Do Androids Dream of Electric Sheep? + type: scale + - hex: '#F4CF31' + name: Do Androids Dream of Electric Sheep? + type: scale + decimalPlaces: 1 + height: 4 + kind: Single_Stat_Plus_Line + name: Memory Usage + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r._measurement == "mem") + |> filter(fn: (r) => r._field == "used_percent") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + suffix: '%' + width: 3 + xCol: _time + xPos: 9 + yCol: _value + yPos: 1 + - axes: + - base: "10" + name: x + scale: linear + - base: "2" + name: y + scale: linear + - base: "10" + name: y2 + scale: linear + geom: line + height: 3 + kind: Xy + name: Swap + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["host"] == v.linux_host) + |> filter(fn: (r) => r._measurement == "swap") + |> filter(fn: (r) => r._field == "total" or r._field == "used") + |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) + |> yield(name: "mean") + width: 3 + xCol: _time + xPos: 9 + yCol: _value + yPos: 5 + description: A collection of useful visualizations for monitoring your system + stats + name: Raspberry Pi System +--- +apiVersion: influxdata.com/v2alpha1 +kind: Telegraf +metadata: + name: happy-wing-ecc001 +spec: + associations: + - kind: Label + name: tender-wu-ecc001 + config: | + # Configuration for telegraf agent + [agent] + interval = "10s" + round_interval = true + metric_batch_size = 1000 + metric_buffer_limit = 10000 + collection_jitter = "0s" + flush_interval = "10s" + flush_jitter = "0s" + precision = "" + debug = false + quiet = false + logfile = "" + ## Override default hostname, if empty use os.Hostname() + hostname = "" + ## If set to true, do no set the "host" tag in the telegraf agent. + omit_hostname = false + [[outputs.influxdb_v2]] + urls = ["$INFLUX_HOST"] + ## Token for authentication. + token = "$INFLUX_TOKEN" + ## Organization is the name of the organization you wish to write to; must exist. + organization = "$INFLUX_ORG" + ## Destination bucket to write into. + bucket = "$INFLUX_BUCKET" + [[inputs.cpu]] + ## Whether to report per-cpu stats or not + percpu = true + ## Whether to report total system cpu stats or not + totalcpu = true + ## If true, collect raw CPU time metrics. + collect_cpu_time = false + ## If true, compute and report the sum of all non-idle CPU states. + report_active = false + [[inputs.disk]] + ## By default stats will be gathered for all mount points. + ## Set mount_points will restrict the stats to only the specified mount points. + # mount_points = ["/"] + ## Ignore mount points by filesystem type. + ignore_fs = ["tmpfs", "devtmpfs", "devfs", "overlay", "aufs", "squashfs"] + [[inputs.diskio]] + [[inputs.mem]] + [[inputs.net]] + [[inputs.processes]] + [[inputs.swap]] + [[inputs.system]] + + [[inputs.exec]] + commands = ["/opt/vc/bin/vcgencmd measure_temp"] + name_override = "gpu_temperature" + data_format = "grok" + grok_patterns = ["%{NUMBER:value:float}"] + + [[inputs.temp]] + description: rpi monitoring + name: rpi diff --git a/raspberry-pi/readme.md b/raspberry-pi/readme.md new file mode 100644 index 00000000..99b36114 --- /dev/null +++ b/raspberry-pi/readme.md @@ -0,0 +1,86 @@ +# Raspberry Pi System Template + +Provided by: @bonitoo.io + +This InfluxDB template can be used to monitor your Raspberry Pi Linux system. + +> The Raspberry Pi dashboard gives you an overview of system metrics with +> metrics from `system`, `temp`, `mem`, `diskio`, `swap` and `net` measurements. +> **Raspbian** is supported. + +![Example Dashboard Screenshot](img/raspberry_system_dashboard.png) + +## Quick Install + +If you have your InfluxDB credentials +[configured in the CLI](Vhttps://v2.docs.influxdata.com/v2.0/reference/cli/influx/config/), +you can install this template with: + +```sh +influx apply -u https://raw.githubusercontent.com/influxdata/community-templates/master/raspberry-pi/raspberry-pi-system.yml +``` + +## Included Resources + +- 1 Bucket: `telegraf`, 7d retention +- Labels: `raspberry-pi` + Telegraf Plugin Labels +- 1 Telegraf Configuration +- 1 Dashboard: `Raspberry Pi System` +- 2 Variables: `bucket` and `linux_host` + +## Setup Instructions + +General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. + +- **Install Telegraf** on your Raspberry Pi device. You can use + [Raspbian installation instructions](https://community.influxdata.com/t/raspberry-pi-installation-instructions/5515). +- To enable GPU temperature measurement, **add the telegraf user to the video + group** and test the command. + + ```sh + sudo usermod -a -G video telegraf + sudo -u telegraf vcgencmd measure_temp + ``` + +Telegraf configuration requires the following environment variables: + +- `INFLUX_HOST` - The URL of your influxdb server. +- `INFLUX_ORG` - The name of your Organization. Your Organization name can be + found on the Settings page in your browser. In the **Influxdata Cloud 2** the + organization is your username. +- `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and + write data to the `telegraf` bucket. You can just use your operator token to + get started. This can be found on the `Load Data` > `Tokens` page in your + browser + +### Command Line Setup + +```sh +export INFLUX_HOST=host +export INFLUX_TOKEN=token +export INFLUX_ORG=my_org +``` + +### SystemD Setup + +To **create environment variables** for the systemd **telegraf.service**, define +the variables `INFLUX_TOKEN`, `INFLUX_ORG` and `INFLUX_HOST` in the +`/etc/default/telegraf` file. + +```sh +# write this with values to the /etc/default/telegraf file +INFLUX_HOST=host +INFLUX_TOKEN=token +INFLUX_ORG=my_org +``` + +## Customizations + +You can run the provided Telegraf configuration on multiple Linux machines, and +switch between them using the `linux_host` filter at the top of the dashboard. + +## Contact + +- Author: Ivan Kudibal, Bonitoo s.r.o. +- Github: [Bonitoo.io](https://github.com/bonitoo-io) +- Influx Slack: [@ivankudibal](https://influxdata.com/slack) diff --git a/redis/redis.yml b/redis/redis.yml index fb33acce..c008152e 100644 --- a/redis/redis.yml +++ b/redis/redis.yml @@ -678,7 +678,7 @@ spec: ## Multiple URLs can be specified for a single cluster, only ONE of the ## urls will be written to each interval. ## ex: urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"] - urls = ["$INFLUX_URL"] + urls = ["$INFLUX_HOST"] ## Token for authentication. token = "$INFLUX_TOKEN" ## Organization is the name of the organization you wish to write to; must exist. diff --git a/sensu_go/readme.md b/sensu_go/readme.md new file mode 100644 index 00000000..d68785a4 --- /dev/null +++ b/sensu_go/readme.md @@ -0,0 +1,60 @@ +## Sensu Go Monitoring Template + +This InfluxDB Template can be used to monitor the performance of your Sensu Go observability tool with Telegraf and Prometheus. + +![Sensu Go Dashboard Screenshot](sensu_go_dashboard.png) + +### Quick Install + +#### InfluxDB UI + +In the InfluxDB UI, go to Settings->Templates and enter this URL: https://raw.githubusercontent.com/influxdata/community-templates/master/sensu_go/sensu_go.yml + +#### Influx CLI + +If you have your InfluxDB credentials [configured in the CLI](https://v2.docs.influxdata.com/v2.0/reference/cli/influx/config/), you can install this template with: + +``` +influx apply -u https://raw.githubusercontent.com/influxdata/community-templates/master/sensu_go/sensu_go.yml +``` + +### Included Resources + +- 1 Bucket: `telegraf`, 7d retention +- Labels: `Sensu Go Template` + Telegraf Plugin Labels +- 1 Telegraf Configuration +- 1 Dashboard: `Sensu Go` +- 1 Variable: `bucket` + +## Setup Instructions + + General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. + + The data for the dashboard is populated by the included Telegraf configuration. The Telegraf Configuration requires the following environment variables + + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization (this will be your email address on the InfluxDB Cloud free tier) + - `INFLUX_HOST` - The URL of your InfluxDB host (this can your localhost, a remote instance, or InfluxDB Cloud) + + You **MUST** set these environment variables before running Telegraf using something similar to the following commands + + - This can be found on the `Load Data` > `Tokens` page in your browser: `export INFLUX_TOKEN=TOKEN` + - Your Organization name can be found on the Settings page in your browser: `export INFLUX_ORG=my_org` + +## Running Telegraf + + To get resource data from your Linux hosts, [download and install Telegraf](https://portal.influxdata.com/downloads/) on those hosts. InfluxData provides native packages for a number of distributions as well as binaries that can be executed directly. + + Start Telegraf using the instructions from the `Load Data` > `Telegraf` > `Setup Instructions` link in the UI. + +## Customizations + +You can customize it based on your Sensu installation. More information can be found in the Sensu Go [backend configuration](https://docs.sensu.io/sensu-go/latest/observability-pipeline/observe-schedule/backend/#configuration-summary) and [/metrics endpoint](https://docs.sensu.io/sensu-go/latest/api/metrics) documentation. + +## Contact + +- Author: Nikki Attea +- Email: contact@nikki.dev +- Github: [@nikkictl](https://github.com/nikkictl) +- Influx Slack: [@nikki](https://influxdata.com/slack) +- Website: https://nikki.dev \ No newline at end of file diff --git a/sensu_go/sensu_go.yml b/sensu_go/sensu_go.yml new file mode 100644 index 00000000..262b92c5 --- /dev/null +++ b/sensu_go/sensu_go.yml @@ -0,0 +1,367 @@ +--- +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: sensu-go-template +spec: + name: Sensu Go Template + color: '#7A65F2' +--- +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: outputs-influxdb-v2 +spec: + name: outputs.influxdb_v2 + color: '#108174' +--- +apiVersion: influxdata.com/v2alpha1 +kind: Label +metadata: + name: inputs-prometheus +spec: + name: inputs.prometheus + color: '#326BBA' +--- +apiVersion: influxdata.com/v2alpha1 +kind: Bucket +metadata: + name: telegraf +spec: + name: telegraf + retentionRules: + - everySeconds: 604800 + type: expire +--- +apiVersion: influxdata.com/v2alpha1 +kind: Variable +metadata: + name: bucket +spec: + associations: + - kind: Label + name: sensu-go-template + language: flux + name: bucket + query: |- + buckets() + |> filter(fn: (r) => r.name !~ /^_/) + |> rename(columns: {name: "_value"}) + |> keep(columns: ["_value"]) + type: query +--- +apiVersion: influxdata.com/v2alpha1 +kind: Telegraf +metadata: + name: sensu-go-monitoring +spec: + name: Sensu Go Monitoring + associations: + - kind: Label + name: outputs-influxdb-v2 + - kind: Label + name: inputs-prometheus + - kind: Label + name: sensu-go-template + config: | + # Configuration for telegraf agent + [agent] + interval = "10s" + round_interval = true + metric_batch_size = 1000 + metric_buffer_limit = 10000 + collection_jitter = "0s" + flush_interval = "10s" + flush_jitter = "0s" + precision = "" + hostname = "" + omit_hostname = false + # Configuration for sending metrics to InfluxDB + [[outputs.influxdb_v2]] + urls = ["$INFLUX_HOST"] + token = "$INFLUX_TOKEN" + organization = "$INFLUX_ORG" + bucket = "telegraf" + # Read metrics from one or many prometheus clients + [[inputs.prometheus]] + ## An array of urls to scrape metrics from. + urls = ["http://localhost:8080/metrics"] +--- +apiVersion: influxdata.com/v2alpha1 +kind: Dashboard +metadata: + name: sensu-go +spec: + charts: + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + label: agent sessions + name: y + scale: linear + colors: + - hex: '#FDC44F' + name: Cthulhu + type: scale + - hex: '#007C76' + name: Cthulhu + type: scale + - hex: '#8983FF' + name: Cthulhu + type: scale + geom: monotoneX + height: 3 + hoverDimension: auto + kind: Xy + name: Agent sessions per namespace + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "sensu_go_agent_sessions") + |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: true) + |> yield(name: "last") + shade: true + width: 5 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#DA6FF1' + name: Ectoplasm + type: scale + - hex: '#00717A' + name: Ectoplasm + type: scale + - hex: '#ACFF76' + name: Ectoplasm + type: scale + geom: line + height: 3 + hoverDimension: auto + kind: Xy + name: Message publish latency distributions + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "sensu_go_bus_message_duration") + |> filter(fn: (r) => r["_field"] == "0.5" or r["_field"] == "0.9" or r["_field"] == "0.99") + |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false) + |> yield(name: "last") + shade: true + width: 6 + xCol: _time + yCol: _value + yPos: 3 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + name: y + scale: linear + colors: + - hex: '#FD7A5D' + name: Delorean + type: scale + - hex: '#5F1CF2' + name: Delorean + type: scale + - hex: '#4CE09A' + name: Delorean + type: scale + geom: line + height: 3 + hoverDimension: auto + kind: Xy + name: Event handler latency distribution + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "sensu_go_event_handler_duration") + |> filter(fn: (r) => r["_field"] == "0.5" or r["_field"] == "0.9" or r["_field"] == "0.99") + |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false) + |> yield(name: "last") + shade: true + width: 6 + xCol: _time + yCol: _value + yPos: 6 + - colors: + - hex: '#7CE490' + name: honeydew + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Total etcd servers + queries: + - query: |- + from(bucket: "telegraf") + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "etcd_server_id") + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) + |> yield(name: "sum") + width: 2 + xPos: 5 + - colors: + - hex: '#00C9FF' + name: laser + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Total agent sessions + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "sensu_go_agent_sessions") + |> group() + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: true) + |> yield(name: "sum") + width: 2 + xPos: 5 + yPos: 1 + - colors: + - hex: '#F48D38' + name: tiger + type: text + decimalPlaces: 0 + height: 1 + kind: Single_Stat + name: Total check schedulers + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "sensu_go_cron_schedulers" or r["_measurement"] == "sensu_go_interval_schedulers" or r["_measurement"] == "sensu_go_round_robin_cron_schedulers" or r["_measurement"] == "sensu_go_round_robin_interval_schedulers") + |> group() + |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: true) + |> yield(name: "sum") + width: 2 + xPos: 5 + yPos: 2 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + label: messages published + name: y + scale: linear + colors: + - hex: '#74D495' + name: Atlantis + type: scale + - hex: '#3F3FBA' + name: Atlantis + type: scale + - hex: '#FF4D9E' + name: Atlantis + type: scale + geom: line + height: 3 + hoverDimension: auto + kind: Xy + name: Messages published per second + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "sensu_go_bus_messages_published") + |> derivative(unit: 1s, nonNegative: true) + |> yield(name: "nonnegative derivative") + shade: true + width: 6 + xCol: _time + xPos: 6 + yCol: _value + yPos: 3 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + label: processed events + name: y + scale: linear + colors: + - hex: '#FD7A5D' + name: Delorean + type: scale + - hex: '#5F1CF2' + name: Delorean + type: scale + - hex: '#4CE09A' + name: Delorean + type: scale + geom: line + height: 3 + hoverDimension: auto + kind: Xy + name: Processed events per second + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "sensu_go_events_processed") + |> derivative(unit: 1s, nonNegative: true) + |> yield(name: "nonnegative derivative") + shade: true + width: 6 + xCol: _time + xPos: 6 + yCol: _value + yPos: 6 + - axes: + - base: "10" + name: x + scale: linear + - base: "10" + label: check schedulers + name: y + scale: linear + colors: + - hex: '#FD7A5D' + name: Delorean + type: scale + - hex: '#5F1CF2' + name: Delorean + type: scale + - hex: '#4CE09A' + name: Delorean + type: scale + geom: monotoneX + height: 3 + hoverDimension: auto + kind: Xy + name: Check schedulers per type + position: overlaid + queries: + - query: |- + from(bucket: v.bucket) + |> range(start: v.timeRangeStart, stop: v.timeRangeStop) + |> filter(fn: (r) => r["_measurement"] == "sensu_go_round_robin_interval_schedulers" or r["_measurement"] == "sensu_go_round_robin_cron_schedulers" or r["_measurement"] == "sensu_go_interval_schedulers" or r["_measurement"] == "sensu_go_cron_schedulers") + |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: true) + |> yield(name: "last") + shade: true + width: 5 + xCol: _time + xPos: 7 + yCol: _value + description: Monitoring your Sensu Go observability tool with Telegraf and Prometheus + name: Sensu Go diff --git a/sensu_go/sensu_go_dashboard.png b/sensu_go/sensu_go_dashboard.png new file mode 100644 index 00000000..8fd280c1 Binary files /dev/null and b/sensu_go/sensu_go_dashboard.png differ diff --git a/sflow/sflow.yml b/sflow/sflow.yml index 7dc596e9..1f36e3bd 100644 --- a/sflow/sflow.yml +++ b/sflow/sflow.yml @@ -1057,7 +1057,7 @@ spec: ## Multiple URLs can be specified for a single cluster, only ONE of the ## urls will be written to each interval. ## ex: urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"] - urls = ["$INFLUX_URL"] + urls = ["$INFLUX_HOST"] ## Token for authentication. token = "$INFLUX_TOKEN" ## Organization is the name of the organization you wish to write to; must exist. diff --git a/snmp/readme.md b/snmp/readme.md index 6d2c213a..dac8efce 100644 --- a/snmp/readme.md +++ b/snmp/readme.md @@ -25,12 +25,12 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates This template includes the following: - - 1 Bucket: `snmp`, 1d retention - - 3 Labels: `inputs.net`,`inputs.system`,`SNMP` - - 3 Telegraf Configurations: `Linux SNMP`, `Cisco SNMP`, `Mikrotik SNMP` - - 0 Checks: - - 3 Dashboards: `SNMP System Monitoring`, `SNMP Mikrotik Monitoring`, `SNMP Network Monitoring` - - 2 Variables: `snmp_host`, and `snmp_ifName` + - 1 Bucket: `snmp`, 1d retention + - 3 Labels: `inputs.net`,`inputs.system`,`SNMP` + - 3 Telegraf Configurations: `Linux SNMP`, `Cisco SNMP`, `Mikrotik SNMP` + - 0 Checks: + - 3 Dashboards: `SNMP System Monitoring`, `SNMP Mikrotik Monitoring`, `SNMP Network Monitoring` + - 2 Variables: `snmp_host`, and `snmp_ifName` ## Setup Instructions diff --git a/speedtest/readme.md b/speedtest/readme.md index 34e05f7d..a9456bf3 100644 --- a/speedtest/readme.md +++ b/speedtest/readme.md @@ -21,19 +21,19 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf Configuration: 'speedtest-config' - - 1 Dashboards: 'speedtest' - - 1 Label: 'speedtest' - - 1 Bucket: 'speedtest' + - 1 Telegraf Configuration: 'speedtest-config' + - 1 Dashboards: 'speedtest' + - 1 Label: 'speedtest' + - 1 Bucket: 'speedtest' ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. - - `INFLUX_HOST` - The address of you InfluxDB +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The address of you InfluxDB In order to use this template, you must install speedtest-cli. If you're using Debian based distributions, you need to run ```sudo apt install speedtest-cli``` diff --git a/telegraf/readme.md b/telegraf/readme.md index 94e5d4ec..a7af953a 100644 --- a/telegraf/readme.md +++ b/telegraf/readme.md @@ -72,9 +72,9 @@ ___example___ export TELEGRAF_MANIFEST_URL=https://raw.githubusercontent.com/influxdata/community-templates/master/telegraf/manifest.yml influx apply --url $TELEGRAF_MANIFEST_URL \ - --org $INFLUX_ORG \ - --host $INFLUX_HOST \ - --token $INFLUX_TOKEN + --org $INFLUX_ORG \ + --host $INFLUX_HOST \ + --token $INFLUX_TOKEN ``` ## Contact diff --git a/tomcat/readme.md b/tomcat/readme.md index cdee306e..387d27b6 100644 --- a/tomcat/readme.md +++ b/tomcat/readme.md @@ -21,19 +21,19 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf: 'Tomcat Inputs' - - 1 Dashboards: 'Apache Tomcat' - - 1 Bucket: 'tomcat' - - 1 Label: 'tomcat' + - 1 Telegraf: 'Tomcat Inputs' + - 1 Dashboards: 'Apache Tomcat' + - 1 Bucket: 'tomcat' + - 1 Label: 'tomcat' ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_HOST` - The host running InfluxDB - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. +Telegraf Configuration requires the following environment variables + - `INFLUX_HOST` - The host running InfluxDB + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. As the bucket was provided in this template, you need to set the following environment data export INFLUX_BUCKET=tomcat. Also is required to specify trough variables the connection string of Tomcat using export TOMCAT_CONNECTION_STRING="you-server:8080//manager/status/all?XML=true" and the user and password. diff --git a/vsphere/readme.md b/vsphere/readme.md index 2c1de127..7c640641 100644 --- a/vsphere/readme.md +++ b/vsphere/readme.md @@ -21,20 +21,20 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf Configuration - - 1 Dashboards: vsphere.yml - - 1 bucket: 'vsphere' - - 1 label: 'vsphere' + - 1 Telegraf Configuration + - 1 Dashboards: vsphere.yml + - 1 bucket: 'vsphere' + - 1 label: 'vsphere' ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. - - `INFLUX_HOST` - The address of you InfluxDB - - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```export INFLUX_BUCKET=vsphere``` +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The address of you InfluxDB + - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```export INFLUX_BUCKET=vsphere``` In order to use this Dashboard, you need to specify as variable the address to the vSphere (EX: https://vsphere/sdk), also, you need to provide the username and password as variable too. diff --git a/windows_system/readme.md b/windows_system/readme.md index 6d5be45d..56814b2e 100644 --- a/windows_system/readme.md +++ b/windows_system/readme.md @@ -33,7 +33,7 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - `INFLUX_ORG` - The name of your Organization (this will be your email address on the InfluxDB Cloud free tier) - - `INFLUX_URL` - The URL of your InfluxDB host (this can your localhost, a remote instance, or InfluxDB Cloud) + - `INFLUX_HOST` - The URL of your InfluxDB host (this can your localhost, a remote instance, or InfluxDB Cloud) You **MUST** set these environment variables before running Telegraf using something similar to the following commands diff --git a/windows_system/windows_system.yml b/windows_system/windows_system.yml index 880a2afb..14a31b1d 100644 --- a/windows_system/windows_system.yml +++ b/windows_system/windows_system.yml @@ -187,7 +187,7 @@ spec: ## Multiple URLs can be specified for a single cluster, only ONE of the ## urls will be written to each interval. ## ex: urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"] - urls = ["$INFLUX_URL"] + urls = ["$INFLUX_HOST"] ## Token for authentication. token = "$INFLUX_TOKEN" ## Organization is the name of the organization you wish to write to; must exist. diff --git a/x509/readme.md b/x509/readme.md index 17d01b0e..81a302f6 100644 --- a/x509/readme.md +++ b/x509/readme.md @@ -21,20 +21,20 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf Configuration - - 1 Dashboards: x509.yml - - 1 bucket: 'x509' - - 1 label: 'x509' + - 1 Telegraf Configuration + - 1 Dashboards: x509.yml + - 1 bucket: 'x509' + - 1 label: 'x509' ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. - - `INFLUX_HOST` - The address of you InfluxDB - - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```export INFLUX_BUCKET=x509``` +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The address of you InfluxDB + - `INFLUX_BUCKET` - The name of the Bucket. If you going to use the bucket included, you need to export the variable. Ex: ```export INFLUX_BUCKET=x509``` In order to use this template, before import, you need to specify the certificates you want monitor in the x509.yml file. diff --git a/zookeeper/readme.md b/zookeeper/readme.md index 97e1f4fd..a545d8ba 100644 --- a/zookeeper/readme.md +++ b/zookeeper/readme.md @@ -21,20 +21,20 @@ influx apply -u https://raw.githubusercontent.com/influxdata/community-templates ## Included Resources - - 1 Telegraf Configuration - - 1 Dashboards: 'zookeeper' - - 1 Bucket: 'zookeeper' - - 1 Label: 'zookeeper' + - 1 Telegraf Configuration + - 1 Dashboards: 'zookeeper' + - 1 Bucket: 'zookeeper' + - 1 Label: 'zookeeper' ## Setup Instructions General instructions on using InfluxDB Templates can be found in the [use a template](../docs/use_a_template.md) document. - Telegraf Configuration requires the following environment variables - - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. - - `INFLUX_ORG` - The name of your Organization. - - `INFLUX_HOST` - The address of you InfluxDB - - `INFLUX_BUCKET` - The name of the Bucket. In this case, the bucket was included, so, you need to specify 'INFLUX_BUCKET=zookeeper' +Telegraf Configuration requires the following environment variables + - `INFLUX_TOKEN` - The token with the permissions to read Telegraf configs and write data to the `telegraf` bucket. You can just use your operator token to get started. + - `INFLUX_ORG` - The name of your Organization. + - `INFLUX_HOST` - The address of you InfluxDB + - `INFLUX_BUCKET` - The name of the Bucket. In this case, the bucket was included, so, you need to specify 'INFLUX_BUCKET=zookeeper' In order to use this Dashboard, you need to specify the address to the ZooKeeper client (EX: https://localhost:2181) as variable 'ZOOKEEPER_HOST'