agent: introduce status HTTP API and OpenAPI docs

Adds the first public HTTP endpoint to agent, for fetching the status for a
list of live specs.  This includes some boilerplate for generating an OpenAPI
spec from our axum handlers, and API docs that are rendered from that spec
using Scalar.

I tried to establish some repeatable patterns for writing API handlers with
generated documentation and consistent error handling. The objective was to
make it pretty straight forward to add more endpoints. Part of this was
introducing a separation between the "public" APIs that we wish to make
available to all clients, and the private APIs for which we don't want to
guarantee backward compatibility. The idea is to only add apis to the `public`
module when we want to support it for use by end users (which implies at least
some level of backward compatibility). Only the public APIs are covered by the
OpenAPI spec.

The `aide` crate is used for generating the OpenAPI spec. It's not _great_,
but seems to be better than the alternative, `utoipa`, which requires
macros for defining routes and isn't directly compatible with `schemars`.

Cargo.toml

@@ -15,6 +15,14 @@ license = "BSL"
 
 [workspace.dependencies]
 addr = { version = "0.15.4", default-features = false, features = ["std"] }
+aide = { version = "0.13", features = [
+    "axum",
+    "macros",
+    "scalar",
+    "redoc",
+    "axum-extra",
+    "axum-extra-query",
+] }
 anyhow = "1.0"
 async-compression = { version = "0.3", features = [
     "futures-io",
@@ -227,10 +235,11 @@ pbjson-build = "0.7"
 prost-build = "0.13"
 tonic-build = "0.12"
 
-warp = "0.3.3"                                                # TODO(johnny) remove me in favor of axum
+warp = "0.3.3" # TODO(johnny) remove me in favor of axum
+# Used for the agent http server
 axum = { version = "0.7", features = ["macros"] }
 axum-server = { version = "0.7", features = ["tls-rustls"] }
-axum-extra = { version = "0.9", features = ["typed-header"] }
+axum-extra = { version = "0.9", features = ["typed-header", "query"] }
 
 [profile.release]
 incremental = true

crates/agent/Cargo.toml

@@ -30,6 +30,7 @@ sources = { path = "../sources" }
 tables = { path = "../tables", features = ["persist"] }
 validation = { path = "../validation" }
 
+aide = { workspace = true }
 anyhow = { workspace = true }
 async-trait = { workspace = true }
 axum = { workspace = true }
@@ -60,6 +61,7 @@ thiserror = { workspace = true }
 tokio = { workspace = true }
 tokio-util = { workspace = true }
 tonic = { workspace = true }
+tower = { workspace = true }
 tower-http = { workspace = true }
 tracing = { workspace = true }
 tracing-subscriber = { workspace = true }