diff --git a/docs/assets/monitoring_grafana_alert_rules_5xx_1.png b/docs/assets/monitoring_grafana_alert_rules_5xx_1.png new file mode 100644 index 00000000..cadaa3b8 Binary files /dev/null and b/docs/assets/monitoring_grafana_alert_rules_5xx_1.png differ diff --git a/docs/assets/monitoring_grafana_alert_rules_5xx_2.png b/docs/assets/monitoring_grafana_alert_rules_5xx_2.png new file mode 100644 index 00000000..50f7fc15 Binary files /dev/null and b/docs/assets/monitoring_grafana_alert_rules_5xx_2.png differ diff --git a/docs/assets/monitoring_grafana_dashboards_Errors_1.png b/docs/assets/monitoring_grafana_dashboards_Errors_1.png new file mode 100644 index 00000000..6b436ae7 Binary files /dev/null and b/docs/assets/monitoring_grafana_dashboards_Errors_1.png differ diff --git a/docs/assets/monitoring_grafana_dashboards_Errors_2.png b/docs/assets/monitoring_grafana_dashboards_Errors_2.png new file mode 100644 index 00000000..e736ae4d Binary files /dev/null and b/docs/assets/monitoring_grafana_dashboards_Errors_2.png differ diff --git a/docs/assets/monitoring_grafana_dashboards_Performance_1.png b/docs/assets/monitoring_grafana_dashboards_Performance_1.png new file mode 100644 index 00000000..0cfcaeb6 Binary files /dev/null and b/docs/assets/monitoring_grafana_dashboards_Performance_1.png differ diff --git a/docs/assets/monitoring_grafana_dashboards_Performance_2.png b/docs/assets/monitoring_grafana_dashboards_Performance_2.png new file mode 100644 index 00000000..00041238 Binary files /dev/null and b/docs/assets/monitoring_grafana_dashboards_Performance_2.png differ diff --git a/docs/assets/monitoring_grafana_folder.png b/docs/assets/monitoring_grafana_folder.png new file mode 100644 index 00000000..1291399d Binary files /dev/null and b/docs/assets/monitoring_grafana_folder.png differ diff --git a/docs/blocks/monitoring.md b/docs/blocks/monitoring.md new file mode 100644 index 00000000..0f736a3e --- /dev/null +++ b/docs/blocks/monitoring.md @@ -0,0 +1,53 @@ +# Monitoring Block + +This block sets up the monitoring stack for Self Host Blocks. It is composed of: + +- Grafana as the dashboard frontend. +- Prometheus as the database for metrics. +- Loki as the database for logs. + +## Provisioning + +Self Host Blocks will create automatically the following resources: + +- For Grafana: + - datasources + - dashboards + - contact points + - notification policies + - alerts +- For Prometheus, the following exporters and related scrapers: + - node + - smartctl + - nginx +- For Loki, the following exporters and related scrapers: + - systemd + +Those resources are namespaced as appropriate under the Self Host Blocks namespace: + +![](../assets/monitoring_grafana_folder.png) + +## Errors Dashboard + +This dashboard is meant to be the first stop to understand why a service is misbehaving. + +![](../assets/monitoring_grafana_dashboards_Errors_1.png) +![](../assets/monitoring_grafana_dashboards_Errors_2.png) + +The yellow and red dashed vertical bars correspond to the [Requests Error Budget +Alert](#requests-error-budge-alert) firing. + +## Performance Dashboard + +This dashboard is meant to be the first stop to understand why a service is performing poorly. + +![](../assets/monitoring_grafana_dashboards_Performance_1.png) +![](../assets/monitoring_grafana_dashboards_Performance_2.png) + +## Requests Error Budget Alert + +This alert will fire when the ratio between number of requests getting a 5XX response from a service +and the total requests to that service exceeds 1%. + +![](../assets/monitoring_grafana_alert_rules_5xx_1.png) +![](../assets/monitoring_grafana_alert_rules_5xx_2.png)