Skip to content

Commit

Permalink
feat: Too many for title (prodzilla#16)
Browse files Browse the repository at this point in the history
* chore: Fix clippy warnings

In order to clean the code up a bit, I fixed everything that `cargo
clippy` complained about. The warnings were of types:

- redundant field names in struct initialization
- unused imports
- `&String` being used instead of `&str`, fixing this saves a copy
- unnecessary `return` statements
- use of `or_insert_with` instead of `or_default` for constructing
  default values
- some references which are immediately dereferenced by the compiler
- unneeded late initializations before `match` expressions
- single-character string constants being used instead of a char
- module inception in `test_utils::test_utils`. I renamed the inner
  module to `probe_test_utils` to facilitate other types of test util
  modules being added to that parent module.

Hopefully this should make the code a little bit more idiomatic,
although I'm far from a Rust expert so take the changes with a grain of
salt.

* feat: Add root span to more deeply instrument stories

* chore: Move otel setup to otel module

* fix: Add nested spans

* feat: Switch to tokio-tracing with tracing_opentelemetry

* feat: Add support for specifying other files through CLI

* chore: Update RAM usage in README

The changes thus far have increased RAM usage from 8 to around 14 MB on
my machine when running the default `prozilla.yml`. Hopefully this isn't
too bad, if necessary I could probably profile where the memory usage is
coming from but I'm fairly certain it's from OpenTelemetry since there's
a proper pipeline going on now.

* chore: move otel resource init to module root

* feat: Otel metrics, pick exporters with env

Adds some basic OpenTelemetry metrics support using
tracing_opentelemetry as well as support for choosing between stdout and
otlp exporters for both traces and metrics using the standard
OpenTelemetry environment variable conventions. Both traces and metrics
are disabled if the corresponding environment variables are not set.

* feat: Support environment variable substitution

* feat: Add trace ID to alerts

* Revert "feat: Switch to tokio-tracing with tracing_opentelemetry"

This reverts commit 3bbb42d.

* fix: Fix broken test

* chore: Run cargo fmt on entire project

* chore: Add .git-blame-ignore-revs to hide chore commits

* feat: Add 'Matches' expectation which tests a regular expression

* feat: Set up OTLP HTTP, remove tracing-opentelemetry

* feat: Trim variables to support whitespace in substitutions

* chore: Refactor metrics

* fix: Add 0 to error metrics on success

This is done so that the error metrics are initialized for that
particular set of labels to 0. By doing this, any backend that receives
the data will receive a time series that starts at 0 instead of
eventually receiving a new time series that starts at 1 once the first
error arrives. In the latter case, a promql query for `rate` e.g. would
fail to detect the transition from 0 -> 1 as the transition is actually
from non-existant to 1.

* feat: Add support for Slack webhooks

* feat: Enable use of env vars globally in config

The previous implementation hooked in to the same substitution logic as
for step outputs and generated values. This had the consequence that it
would only apply during probes, and can not be used for other parts of
the configuration file such as webhook URLs.

This commit moves the enivronment variable substitution to the
configuration loading step of Prodzilla's initialization in order to
allow environment variables to be used anywhere in the configuration.

* feat: Add basic dockerfile

* feat: Add GHA to build and publish docker image

* fix: Support multiplatform build through QEMU

* chore: Track Cargo.lock

It seems that the guidance used to be that Cargo.lock should be tracked
for binaries (like Prodzilla) but not for libraries. It should have been
 tracked from the start according to this guidance.

 Updated guidance is available at
 https://blog.rust-lang.org/2023/08/29/committing-lockfiles.html , where
 the new recommendation is to simply do what is best for the project but
 defualt to tracking Cargo.lock.

For Prodzilla, tracking Cargo.lock helps make the Docker and binary
builds reproducible so I believe that it is beneficial to this project.

Further reference:
- https://doc.rust-lang.org/nightly/cargo/faq.html#why-have-cargolock-in-version-control
- rust-lang/cargo#8728

* fix: Add more build dependencies

* fix: Set prodzilla as docker entrypoint

* fix: Fix dockerfile

* fix: Make whitespace around env vars optional

* fix: Drastically simplify dockerfile

* chore: Add test to catch earlier bug with whitespace in env vars

* chore: Document slack_webhook parameter

* docs: Update feature roadmap

* chore: revert accidental change to TOC format

* docs: Document new features

* fix: Fix regression with missing parent span IDs

Previously, opentelemetry tracing wouldn't be initialized at all if an
exporter isn't configured. This leads to parent trace IDs not being
available. This commit re-introduces the earlier Prodzilla behvaiour of
including a parent trace ID which is propagated in outgoing web
requests even if the root spans aren't exported.

* fix: Vendor openssl to enable cross-compilation

* feat: Add release workflow

* chore: Only build docker image on tagged releases

* chore: Update package version to v0.0.3

* feat: Add musl build targets for Alpine Linux

* fix: Improve error message on missing config file

* chore: add missing space in readme

* fix: Correct error metrics for probes

* feat: Include error message in webhook alerts

* feat!: Remove slack_webhook config parameter and route based on url instead

* chore: Update package version to v0.0.4

* chore: Cleanup error unwrapping

* feat: Include status code and body in alerts, style Slack

* feat: Add support for marking steps as sensitive

This leads to logs and alerts being redacted so that the sensitive
response bodies aren't included.

* fix: Report span status on probe error

* chore: Remove unused import

* fix: escape newlines in logged bodies

* feat: Make request timeout configurable

* chore: Use tidier Option unwrapping

* fix: Remove openssl vendoring, binary builds

OpenSSL vendoring was causing issues with builds on Windows so I think
for now it's probably best to just remove it and the binary builds that
required it. This means that releases will no longer include
executables, but the Docker images are still published and the source
code can be compiled for the target platform by the user.

It might be worth revisiting binary builds in the future with static
linking, but I don't have the time (or the use-case at $WORK) to do that
at the moment and would like to avoid this being a blocker.
  • Loading branch information
stevensdavid authored Jul 21, 2024
1 parent 6325147 commit c720d78
Show file tree
Hide file tree
Showing 30 changed files with 3,889 additions and 361 deletions.
32 changes: 32 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Include any files or directories that you don't want to be copied to your
# container here (e.g., local build artifacts, temporary files, etc.).
#
# For more help, visit the .dockerignore file reference guide at
# https://docs.docker.com/go/build-context-dockerignore/

**/.DS_Store
**/.classpath
**/.dockerignore
**/.env
**/.git
**/.gitignore
**/.project
**/.settings
**/.toolstarget
**/.vs
**/.vscode
**/*.*proj.user
**/*.dbmdl
**/*.jfm
**/charts
**/docker-compose*
**/compose*
**/Dockerfile*
**/node_modules
**/npm-debug.log
**/secrets.dev.yaml
**/values.dev.yaml
/bin
/target
LICENSE
README.md
2 changes: 2 additions & 0 deletions .git-blame-ignore-revs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# cargo fmt
c4cad928287973eb223d31177af10deaaf84ef3d
66 changes: 66 additions & 0 deletions .github/workflows/docker.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
#
name: Create and publish a Docker image

# Configures this workflow to run every time a change is pushed to the branch called `release`.
on:
push:
tags: ["v*"]

# Defines two custom environment variables for the workflow. These are used for the Container registry domain, and a name for the Docker image that this workflow builds.
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}

# There is a single job in this workflow. It's configured to run on the latest available version of Ubuntu.
jobs:
build-and-push-image:
runs-on: ubuntu-latest
# Sets the permissions granted to the `GITHUB_TOKEN` for the actions in this job.
permissions:
contents: read
packages: write
attestations: write
id-token: write
#
steps:
- name: Checkout repository
uses: actions/checkout@v4
# Uses the `docker/login-action` action to log in to the Container registry registry using the account and password that will publish the packages. Once published, the packages are scoped to the account defined here.
- name: Log in to the Container registry
uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# This step uses [docker/metadata-action](https://github.com/docker/metadata-action#about) to extract tags and labels that will be applied to the specified image. The `id` "meta" allows the output of this step to be referenced in a subsequent step. The `images` value provides the base name for the tags and labels.
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# These two steps configure multi-platform building
- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# This step uses the `docker/build-push-action` action to build the image, based on your repository's `Dockerfile`. If the build succeeds, it pushes the image to GitHub Packages.
# It uses the `context` parameter to define the build's context as the set of files located in the specified path. For more information, see "[Usage](https://github.com/docker/build-push-action#usage)" in the README of the `docker/build-push-action` repository.
# It uses the `tags` and `labels` parameters to tag and label the image with the output from the "meta" step.
- name: Build and push Docker image
id: push
uses: docker/build-push-action@f2a1d5e99d037542a71f64918e516c093c6f3fc4
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
platforms: linux/amd64,linux/arm64

# This step generates an artifact attestation for the image, which is an unforgeable statement about where and how it was built. It increases supply chain security for people who consume the image. For more information, see "[AUTOTITLE](/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds)."
- name: Generate artifact attestation
uses: actions/attest-build-provenance@v1
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
subject-digest: ${{ steps.push.outputs.digest }}
push-to-registry: true
20 changes: 20 additions & 0 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: Release

permissions:
contents: write

on:
push:
tags:
- v[0-9]+.*

jobs:
create-release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# v1 tag as of 2024-06-13
- uses: taiki-e/create-gh-release-action@72d65cee1f8033ef0c8b5d79eaf0c45c7c578ce3
with:
# (required) GitHub token for creating GitHub Releases.
token: ${{ secrets.GITHUB_TOKEN }}
4 changes: 0 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,6 @@
debug/
target/

# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
Cargo.lock

# These are backup files generated by rustfmt
**/*.rs.bk

Expand Down
Loading

0 comments on commit c720d78

Please sign in to comment.