Skip to content

Commit

Permalink
fix: subdomain-url handling
Browse files Browse the repository at this point in the history
- removed implicit tests based on SubdomainLocalhostGatewayURL
- subdomain tests are now run only against origins defined via
  `--subdomain-url`
- DNSLink gateway tests are no longer tied to origin passed via
  `--subdomain-url`
  • Loading branch information
lidel committed May 24, 2024
1 parent c0dafe1 commit 3e74a1a
Show file tree
Hide file tree
Showing 15 changed files with 681 additions and 700 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,4 @@ dist

# reports
reports
report.json
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.6.0] - 2024-05-27
### Changed
- The `--gateway-url` now defaults to `http://127.0.0.1:8080` to ensure no confusion with subdomain gateway feature.
- The `--subdomain-url` now defaults to `http://localhost:8080`, making it more friendly for local development.
- We also simplified the way `--subdomain-url` works. We no longer run implicit tests
against `http://localhost` in addition to the URL passed via
`--subdomain-url`. To test more than one domain, run test multiple times.
- DNSLink fixtures no longer depend on `--subdomain-url` and use unrelated `*.example.org` domains instead.

## [0.5.2] - 2024-05-20
### Changed
- Fixed: relaxed dag-cbor error check ([#205](https://github.com/ipfs/gateway-conformance/pull/205))
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ test-cargateway: provision-cargateway fixtures.car gateway-conformance

test-kubo-subdomains: provision-kubo gateway-conformance
./kubo-config.example.sh
./gateway-conformance test --json reports/output.json --gateway-url http://127.0.0.1:8080 --subdomain-url http://example.com:8080
./gateway-conformance test --json reports/output.json --gateway-url http://127.0.0.1:8080 --subdomain-url http://localhost:8080

test-kubo: provision-kubo gateway-conformance
./gateway-conformance test --json reports/output.json --gateway-url http://127.0.0.1:8080 --specs -subdomain-gateway
Expand Down
12 changes: 7 additions & 5 deletions cmd/gateway-conformance/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -90,16 +90,18 @@ func main() {
Flags: []cli.Flag{
&cli.StringFlag{
Name: "gateway-url",
EnvVars: []string{"GATEWAY_URL"},
Aliases: []string{"url", "g"},
Usage: "The URL of the IPFS Gateway implementation to be tested.",
Value: "http://localhost:8080",
Value: "http://127.0.0.1:8080",
Destination: &gatewayURL,
},
&cli.StringFlag{
Name: "subdomain-url",
Usage: "The Subdomain URL of the IPFS Gateway implementation to be tested.",
Value: "http://example.com",
Destination: &subdomainGatewayURL,
Name: "subdomain-url",
Aliases: []string{"sg"},
EnvVars: []string{"SUBDOMAIN_GATEWAY_URL"},
Usage: "URL of the HTTP Host that should be used when testing https://specs.ipfs.tech/http-gateways/subdomain-gateway/ functionality",
Value: "http://localhost:8080", // TODO: ideally, make these empty by default, and opt-in
},
&cli.StringFlag{
Name: "json-output",
Expand Down
15 changes: 8 additions & 7 deletions docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ The `test` command is the main command of the tool. It is used to test a given I

| Input | Availability | Description | Default |
|---|---|---|---|
| gateway-url | Both | The URL of the IPFS Gateway implementation to be tested. | http://localhost:8080 |
| subdomain-url | Both | The URL to be used in Subdomain feature tests based on Host HTTP header. | http://example.com |
| gateway-url | Both | The URL of the IPFS Gateway implementation to be tested. | http://127.0.0.1:8080 |
| subdomain-url | Both | The URL to be used in Subdomain feature tests based on Host HTTP header. | http://localhost:8080 |
| json | Both | The path where the JSON test report should be generated. | `./report.json` |
| xml | GitHub Action | The path where the JUnit XML test report should be generated. | `./report.xml` |
| html | GitHub Action | The path where the one-page HTML test report should be generated. | `./report.html` |
Expand Down Expand Up @@ -61,18 +61,19 @@ A few examples:

| Use Case | gateway-url | subdomain-url |
|----------|-------------|---------------|
| CI & Dev | http://127.0.0.1:8080 | http://example.com |
| Production | https://dweb.link | https://dweb.link |
| CI & Dev | `http://127.0.0.1:8080` | `http://localhost:8080` |
| Production | `https://ipfs.io` | `https://dweb.link` |

#### Usage

##### GitHub Action

```yaml
- name: Run gateway-conformance tests
uses: ipfs/gateway-conformance/.github/actions/test@v1
uses: ipfs/gateway-conformance/.github/actions/test@v6 # TODO make sure to use latest
with:
gateway-url: http://localhost:8080
gateway-url: http://127.0.0.1:8080
subdomain-url: http://localhost:8080
specs: +subdomain-gateway,-path-gateway
json: report.json
xml: report.xml
Expand All @@ -84,7 +85,7 @@ A few examples:
##### Docker
```bash
docker run --network host -v "${PWD}:/workspace" -w "/workspace" ghcr.io/ipfs/gateway-conformance test --gateway-url http://localhost:8080 --json report.json --specs +subdomain-gateway,-path-gateway -- -timeout 30m
docker run --network host -v "${PWD}:/workspace" -w "/workspace" ghcr.io/ipfs/gateway-conformance test --gateway-url http://127.0.0.1:8080 --subdomain-url http://localhost:8080 --json report.json --specs +subdomain-gateway,-path-gateway -- -timeout 30m
```

### extract-fixtures
Expand Down
4 changes: 2 additions & 2 deletions fixtures/dir_listing/dnslink.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# yaml-language-server: $schema=../fixture.schema.json
dnslinks:
website:
subdomain: website
dir-listing-website:
domain: dnslink-dir-listing.example.org
path: /ipfs/bafybeig6ka5mlwkl4subqhaiatalkcleo4jgnr3hqwvpmsqfca27cijp3i # ./rootDir/
4 changes: 2 additions & 2 deletions fixtures/redirects_file/dnslink.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# yaml-language-server: $schema=fixture.schema.json
dnslinks:
custom-dnslink:
subdomain: dnslink-enabled-on-fqdn
domain: dnslink-enabled-on-fqdn.example.org
# cid of ./redirects.car:/examples/
path: /ipfs/QmYBhLYDwVFvxos9h8CGU2ibaY66QNgv8hpfewxaQrPiZj
dnslink-spa:
subdomain: dnslink-enabled-with-spa
domain: dnslink-enabled-with-spa.example.org
# cid of ./redirects-spa.car
path: /ipfs/bafybeib5lboymwd6p2eo4qb2lkueaine577flvsjjeuevmp2nlio72xv5q
6 changes: 3 additions & 3 deletions fixtures/subdomain_gateway/dnslink.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# yaml-language-server: $schema=../fixture.schema.json
dnslinks:
wikipedia:
domain: dnslink-subdomain-gw-test.example.org
domain: en.wikipedia-on-ipfs.example.org
# Wikipedia CID
path: /ipfs/bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze
test:
domain: dnslink-test.example.com
domain: dnslink-test.example.org
# CIDv1=$(echo "hello" | ipfs add --cid-version 1 -Q)
path: /ipfs/bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am
path: /ipfs/bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am
7 changes: 3 additions & 4 deletions kubo-config.example.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,16 @@ ipfs config --json Gateway.PublicGateways '{
"example.com": {
"UseSubdomains": true,
"InlineDNSLink": true,
"Paths": ["/ipfs", "/ipns", "/api"]
"Paths": ["/ipfs", "/ipns"]
},
"localhost": {
"UseSubdomains": true,
"InlineDNSLink": true,
"Paths": ["/ipfs", "/ipns", "/api"]
"Paths": ["/ipfs", "/ipns"]
}
}'

export IPFS_NS_MAP=$(cat "${FIXTURES_PATH}/dnslinks.json" | jq -r '.subdomains | to_entries | map("\(.key).example.com:\(.value)") | join(",")')
export IPFS_NS_MAP="$(cat "${FIXTURES_PATH}/dnslinks.json" | jq -r '.domains | to_entries | map("\(.key):\(.value)") | join(",")'),${IPFS_NS_MAP}"
export IPFS_NS_MAP="$(cat "${FIXTURES_PATH}/dnslinks.json" | jq -r '.domains | to_entries | map("\(.key):\(.value)") | join(",")')"

echo "Set the following IPFS_NS_MAP before starting the kubo daemon:"
echo "IPFS_NS_MAP=${IPFS_NS_MAP}"
21 changes: 13 additions & 8 deletions tests/dnslink_gateway_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,25 +20,28 @@ func TestDNSLinkGatewayUnixFSDirectoryListing(t *testing.T) {
fixture := car.MustOpenUnixfsCar("dir_listing/fixtures.car")
file := fixture.MustGetNode("ą", "ę", "file-źł.txt")

// DNSLink domain and fixture we will be using for Host headerthis test
dnsLinkHostname := "dnslink-website.example.org"
dnsLinks := dnslink.MustOpenDNSLink("dir_listing/dnslink.yml")
dnsLink := dnsLinks.MustGet("website")

gatewayURL := SubdomainGatewayURL
dnsLink := dnsLinks.MustGet("dir-listing-website")

tests := SugarTests{}

u, err := url.Parse(gatewayURL)
// Sent requests to endpoint defined by --gateway-url
u, err := url.Parse(GatewayURL)
if err != nil {
t.Fatal(err)
}

dnsLinkHostname := tmpl.Fmt("{{dnslink}}.{{host}}", dnsLink, u.Host)
// Host header should use dnslink domain with the same scheme as --gateway-url
hostHdr := tmpl.Fmt("{{scheme}}://{{dnslink}}", u.Scheme, dnsLink)

tests = append(tests, SugarTests{
{
Name: "Backlink on root CID should be hidden (TODO: cleanup Kubo-specifics)",
Request: Request().
URL(`{{scheme}}://{{hostname}}/`, u.Scheme, dnsLinkHostname),
Header("Host", hostHdr).
URL(GatewayURL),
Response: Expect().
Body(
And(
Expand All @@ -50,7 +53,8 @@ func TestDNSLinkGatewayUnixFSDirectoryListing(t *testing.T) {
{
Name: "Redirect dir listing to URL with trailing slash",
Request: Request().
URL(`{{scheme}}://{{hostname}}/ą/ę`, u.Scheme, dnsLinkHostname),
Header("Host", hostHdr).
URL(GatewayURL + "/ą/ę"),
Response: Expect().
Status(301).
Headers(
Expand All @@ -60,7 +64,8 @@ func TestDNSLinkGatewayUnixFSDirectoryListing(t *testing.T) {
{
Name: "Regular dir listing (TODO: cleanup Kubo-specifics)",
Request: Request().
URL(`{{scheme}}://{{hostname}}/ą/ę/`, u.Scheme, dnsLinkHostname),
Header("Host", hostHdr).
URL(GatewayURL + "/ą/ę"),
Response: Expect().
Headers(
Header("Etag").Contains(`"DirIndex-`),
Expand Down
Loading

0 comments on commit 3e74a1a

Please sign in to comment.