Skip to content

Commit

Permalink
feat(plugins): use RegisterDirectiveOrder in caddy (#520)
Browse files Browse the repository at this point in the history 
* feat(plugins): use RegisterDirectiveOrder in caddy

* feat(documentation): iso with the actual behaviour/codebase

* feat(chore): bump and prepare release v1.6.49

* fix(workflow): temporary remove Tyk and Roadrunner
  • Loading branch information
@darkweak
darkweak authored May 30, 2024
1 parent 308e8d5 commit 97c7255
Show file tree
Hide file tree
Showing 45 changed files with 414 additions and 530 deletions.
16 changes: 1 addition & 15 deletions .github/workflows/plugins.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
run: go install github.com/caddyserver/xcaddy/cmd/xcaddy@latest
-
name: Build Souin as caddy module
run: cd plugins/caddy && xcaddy build --with github.com/darkweak/souin/plugins/caddy=./ --with github.com/darkweak/souin@latest=../..
run: cd plugins/caddy && xcaddy build --with github.com/darkweak/souin/plugins/caddy=./ --with github.com/darkweak/souin=../..
-
name: Run Caddy tests
run: cd plugins/caddy && go test -v ./...
Expand Down Expand Up @@ -208,13 +208,6 @@ jobs:
CAPITALIZED_NAME: Kratos
LOWER_NAME: kratos
GO_VERSION: '1.21'
build-roadrunner-validator:
uses: ./.github/workflows/plugin_template.yml
secrets: inherit
with:
CAPITALIZED_NAME: Roadrunner
LOWER_NAME: roadrunner
GO_VERSION: '1.21'
build-souin-validator:
uses: ./.github/workflows/plugin_template.yml
secrets: inherit
Expand All @@ -229,13 +222,6 @@ jobs:
CAPITALIZED_NAME: Traefik
LOWER_NAME: traefik
GO_VERSION: '1.21'
build-tyk-validator:
uses: ./.github/workflows/plugin_template.yml
secrets: inherit
with:
CAPITALIZED_NAME: Tyk
LOWER_NAME: tyk
GO_VERSION: '1.21'
build-webgo-validator:
uses: ./.github/workflows/plugin_template.yml
secrets: inherit
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/workflow_plugins_generator.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# Disabled go-zero and hertz temporary
# plugins=("beego" "chi" "dotweb" "echo" "fiber" "gin" "goa" "go-zero" "hertz" "kratos" "roadrunner" "souin" "traefik" "tyk" "webgo")
plugins=("beego" "chi" "dotweb" "echo" "fiber" "gin" "goa" "kratos" "roadrunner" "souin" "traefik" "tyk" "webgo")
plugins=("beego" "chi" "dotweb" "echo" "fiber" "gin" "goa" "kratos" "souin" "traefik" "webgo")
go_version=1.21

IFS= read -r -d '' tpl <<EOF
Expand Down Expand Up @@ -53,7 +53,7 @@ jobs:
run: go install github.com/caddyserver/xcaddy/cmd/xcaddy@latest
-
name: Build Souin as caddy module
run: cd plugins/caddy && xcaddy build --with github.com/darkweak/souin/plugins/caddy=./ --with github.com/darkweak/souin@latest=../..
run: cd plugins/caddy && xcaddy build --with github.com/darkweak/souin/plugins/caddy=./ --with github.com/darkweak/souin=../..
-
name: Run Caddy tests
run: cd plugins/caddy && go test -v ./...
Expand Down
31 changes: 21 additions & 10 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,7 @@ surrogate_keys:
| `default_cache.badger` | Configure the Badger cache storage | |
| `default_cache.badger.path` | Configure Badger with a file | `/anywhere/badger_configuration.json` |
| `default_cache.badger.configuration` | Configure Badger directly in the Caddyfile or your JSON caddy configuration | [See the Badger configuration for the options](https://dgraph.io/docs/badger/get-started/) |
| `default_cache.default_cache_control` | Set the default value of `Cache-Control` response header if not set by upstream (Souin treats empty `Cache-Control` as `public` if omitted) | `no-store` |
| `default_cache.etcd` | Configure the Etcd cache storage | |
| `default_cache.etcd.configuration` | Configure Etcd directly in the Caddyfile or your JSON caddy configuration | [See the Etcd configuration for the options](https://pkg.go.dev/go.etcd.io/etcd/clientv3#Config) |
| `default_cache.etcd` | Configure the Etcd cache storage | |
Expand All @@ -210,22 +211,24 @@ surrogate_keys:
| `default_cache.key.headers` | Add headers to the key matching the regexp | `- Authorization`<br/><br/>`- Content-Type`<br/><br/>`- X-Additional-Header` |
| `default_cache.key.hide` | Prevent the key from being exposed in the `Cache-Status` HTTP response header | `true`<br/><br/>`(default: false)` |
| `default_cache.key.template` | Use caddy placeholders to create the key (when this option is enabled, disable_* directives are skipped) | [Placeholders documentation](https://caddyserver.com/docs/caddyfile/concepts#placeholders) |
| `default_cache.max_cacheable_body_bytes` | Set the maximum size (in bytes) for a response body to be cached (unlimited if omited) | `1048576` (1MB) |
| `default_cache.mode` | RFC respect tweaking | One of `bypass` `bypass_request` `bypass_response` `strict` (default `strict`) |
| `default_cache.nuts` | Configure the Nuts cache storage | |
| `default_cache.nuts.path` | Set the Nuts file path storage | `/anywhere/nuts/storage` |
| `default_cache.nuts.configuration` | Configure Nuts directly in the Caddyfile or your JSON caddy configuration | [See the Nuts configuration for the options](https://github.com/nutsdb/nutsdb#default-options) |
| `default_cache.olric` | Configure the Olric cache storage | |
| `default_cache.olric.path` | Configure Olric with a file | `/anywhere/olric_configuration.json` |
| `default_cache.olric.configuration` | Configure Olric directly in the Caddyfile or your JSON caddy configuration | [See the Olric configuration for the options](https://github.com/buraksezer/olric/blob/master/cmd/olricd/olricd.yaml/) |
| `default_cache.otter` | Configure the Otter cache storage | |
| `default_cache.otter.configuration` | Configure Otter directly in the Caddyfile or your JSON caddy configuration | |
| `default_cache.otter.configuration.size` | Set the size of the pool in Otter | `999999` (default `10000`) |
| `default_cache.port.{web,tls}` | The device's local HTTP/TLS port that Souin should be listening on | Respectively `80` and `443` |
| `default_cache.regex.exclude` | The regex used to prevent paths being cached | `^[A-z]+.*$` |
| `default_cache.stale` | The stale duration | `25m` |
| `default_cache.timeout` | The timeout configuration | |
| `default_cache.timeout.backend` | The timeout duration to consider the backend as unreachable | `10s` |
| `default_cache.timeout.cache` | The timeout duration to consider the cache provider as unreachable | `10ms` |
| `default_cache.ttl` | The TTL duration | `120s` |
| `default_cache.default_cache_control` | Set the default value of `Cache-Control` response header if not set by upstream (Souin treats empty `Cache-Control` as `public` if omitted) | `no-store` |
| `default_cache.max_cachable_body_bytes` | Set the maximum size (in bytes) for a response body to be cached (unlimited if omited) | `1048576` (1MB) |
| `log_level` | The log level | `One of DEBUG, INFO, WARN, ERROR, DPANIC, PANIC, FATAL it's case insensitive` |
| `reverse_proxy_url` | The reverse-proxy's instance URL (Apache, Nginx, Træfik...) | - `http://yourservice` (Container way)<br/>`http://localhost:81` (Local way)<br/>`http://yourdomain.com:81` (Network way) |
| `ssl_providers` | List of your providers handling certificates | `- traefik`<br/><br/>`- nginx`<br/><br/>`- apache` |
Expand All @@ -251,12 +254,12 @@ The base path for the prometheus API is `/metrics`.
|:--------|:---------|:----------------------------------------|
| `GET` | `/` | Expose the different keys listed below. |

| Key | Definition |
|:-----------------------------------|:--------------------------------|
| `souin_request_counter` | Count the incoming requests |
| `souin_no_cached_response_counter` | Count the uncacheable responses |
| `souin_cached_response_counter` | Count the cacheable responses |
| `souin_avg_response_time` | Average response time |
| Key | Definition |
|:-----------------------------------|:----------------------------------------------------|
| `souin_request_upstream_counter` | Count the incoming requests that go to the upstream |
| `souin_no_cached_response_counter` | Count the uncacheable responses |
| `souin_cached_response_counter` | Count the cacheable responses |
| `souin_avg_response_time` | Average response time |

### Souin API
Souin API allow users to manage the cache.
Expand Down Expand Up @@ -325,6 +328,15 @@ The cache system sits on top of three providers at the moment. It provides two i
}
```

**The Otter provider**: you can tune its configuration using the otter configuration inside your Souin configuration. In order to do that, you have to declare the `otter` block. See the following json example.
```json
"otter": {
"configuration": {
"size": 9999999
}
}
```

**The Olric provider**: you can tune its configuration using the olric configuration inside your Souin configuration and declare Souin has to use the distributed provider. In order to do that, you have to declare the `olric` block and the `distributed` directive. See the following json example.
```json
"distributed": true,
Expand Down Expand Up @@ -454,7 +466,6 @@ xcaddy build --with github.com/darkweak/souin/plugins/caddy
There is the fully configuration below
```caddy
{
order cache before rewrite
log {
level debug
}
Expand Down Expand Up @@ -987,7 +998,7 @@ experimental:
plugins:
souin:
moduleName: github.com/darkweak/souin
version: v1.6.48
version: v1.6.49
```
After that you can declare either the whole configuration at once in the middleware block or by service. See the examples below.
```yaml
Expand Down
2 changes: 1 addition & 1 deletion configurationtypes/types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ type DefaultCache struct {
Timeout Timeout `json:"timeout" yaml:"timeout"`
TTL Duration `json:"ttl" yaml:"ttl"`
DefaultCacheControl string `json:"default_cache_control" yaml:"default_cache_control"`
MaxBodyBytes uint64 `json:"max_cachable_body_bytes" yaml:"max_cachable_body_bytes"`
MaxBodyBytes uint64 `json:"max_cacheable_body_bytes" yaml:"max_cacheable_body_bytes"`
}

// GetAllowedHTTPVerbs returns the allowed verbs to cache
Expand Down
1 change: 0 additions & 1 deletion docs/cache-tests/cache-tests-caddyfile
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
{
order cache before rewrite
cache {
api {
prometheus
Expand Down
2 changes: 1 addition & 1 deletion docs/e2e/Souin E2E.postman_collection.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -375,7 +375,7 @@
"",
"pm.test(\"Prometheus is populated.\", function () {",
" const result = pm.response.text();",
" pm.expect(result).to.include('souin_request_counter 7');",
" pm.expect(result).to.include('souin_request_upstream_counter 7');",
"// pm.expect(result).to.include('souin_no_cached_response_counter 7');",
"// pm.expect(result).to.include('souin_cached_response_counter 3');",
"// pm.expect(result).to.include('souin_avg_response_time_count 7');",
Expand Down
27 changes: 27 additions & 0 deletions docs/website/content/docs/configuration.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,12 @@ default: `false`
* **disable_query**: Prevent the URL query to be part of the generated key.
default: `false`

* **disable_scheme**: Prevent the scheme to be part of the generated key.
default: `false`

* **hash**: Hash the key in the storage.
default: `false`

* **headers**: Add specific headers to the generated key.

* **hide**: Prevent the key from being exposed in the `Cache-Status` HTTP response header.
Expand Down Expand Up @@ -147,11 +153,22 @@ default: `false`
* **disable_query**: Prevent the URL query to be part of the generated key.
default: `false`

* **disable_scheme**: Prevent the scheme to be part of the generated key.
default: `false`

* **hash**: Hash the key in the storage.
default: `false`

* **headers**: Add specific headers to the generated key.

* **hide**: Prevent the key from being exposed in the `Cache-Status` HTTP response header.
default: `false`

#### Max cacheable body bytes
Limit to define if the body size is allowed to be cached. (e.g. `1048576` (1MB)).
If a limit is set, your streamed/chunk responses won't be cached.
default: `unlimited`

#### Mode
The mode prefix allow you to bypass some RFC requirements. (e.g. `default_cache.mode`).
default: `strict`
Expand Down Expand Up @@ -179,6 +196,12 @@ The olric prefix configure the olric storage. (e.g. `default_cache.olric`).
* **configuration**: Configure the Embedded Olric instance directly in your configuration file. It won't connect to an external olric instance.
[See the Embedded Olric configuration for the options]({{% relref "/docs/storages/embedded-olric" %}})

#### Otter
The otter prefix configure the otter storage. (e.g. `default_cache.otter`).

* **configuration**: Configure Otter directly in your configuration file.
[See the Otter configuration for the options]({{% relref "/docs/storages/otter" %}})

#### Regex
The regex prefix configure the actions to do on URL that match the regex. (e.g. `default_cache.regex`).

Expand All @@ -189,6 +212,10 @@ example: `^[A-z]+.*$`
The stale prefix configure the duration to keep the stale responses in the storage. (e.g. `default_cache.stale`).
example: `1d`

#### Storers
The storers prefix configure the order to use the storages, with that you'll be able to chain them, use a local in-memory and fallback to a redis or distributed one that is slower. (e.g. `default_cache.storers`).
example: `[nuts otter badger]`

#### Timeout
The timeout prefix configure the timeouts. (e.g. `default_cache.timeout`).

Expand Down
1 change: 0 additions & 1 deletion docs/website/content/docs/uses-cases/wordpress-with-caddy.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ The following Caddyfile will enable Souin as cache system in caddy. We set dynam
```
{
default_sni {$SERVER_NAME}
order cache before rewrite
cache {
api {
souin
Expand Down
6 changes: 3 additions & 3 deletions pkg/api/prometheus/prometheus.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ const (
counter = "counter"
average = "average"

RequestCounter = "souin_request_counter"
RequestCounter = "souin_request_upstream_counter"
RequestRevalidationCounter = "souin_request_revalidation_counter"
NoCachedResponseCounter = "souin_no_cached_response_counter"
CachedResponseCounter = "souin_cached_response_counter"
Expand Down Expand Up @@ -99,8 +99,8 @@ func push(promType, name, help string) {
// Run populate and prepare the map with the default values.
func run() {
registered = make(map[string]interface{})
push(counter, RequestCounter, "Total request counter")
push(counter, RequestRevalidationCounter, "Total request revalidation counter")
push(counter, RequestCounter, "Total upstream request counter")
push(counter, RequestRevalidationCounter, "Total revalidation request revalidation counter")
push(counter, NoCachedResponseCounter, "No cached response counter")
push(counter, CachedResponseCounter, "Cached response counter")
push(average, AvgResponseTime, "Average response time")
Expand Down
Loading

0 comments on commit 97c7255

Please sign in to comment.