Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Document the retain feature #2744

Merged
merged 8 commits into from
Apr 23, 2024
Merged

docs: Document the retain feature #2744

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a6d983b
docs: Document the retain feature
mhan83 Apr 22, 2024
0b01d0c
docs: Document retain feature
mhan83 Apr 22, 2024
ed10d3f
Describe the V2 assets approach
mhan83 Apr 23, 2024
af29aaf
link via path instead of url
mhan83 Apr 23, 2024
9b0eca8
Update docs/web-apps/automated-testing/_partials/_advanced.md
mhan83 Apr 23, 2024
c512ba5
note v2 method is only for cypress and playwright
mhan83 Apr 23, 2024
99670d0
Update docs/web-apps/automated-testing/_partials/_advanced.md
mhan83 Apr 23, 2024
f4afe3d
Merge branch 'main' into devx-2835
mhan83 Apr 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 26 additions & 3 deletions docs/web-apps/automated-testing/_partials/_advanced.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,12 +153,35 @@ This feature is highly experimental.

## Attaching Test Assets

Any test assets created by your tests at runtime (such as logs, screenshots or reports) you wish to retain along with your test results must be placed in the `__assets__` directory of your project root folder. On Sauce Labs VMs, this path is relative to the current working directory.
By default, any test assets created by your tests at runtime (such as logs, screenshots or reports) you wish to retain along with your test results must be placed in the `__assets__` directory of your project root folder. On Sauce Labs VMs, this path is relative to the current working directory.

### Attaching entire directories

In situations where you want to preserve the file structure of your assets (e.g. a multi-page HTML report),
you can use the `retain` feature to define a directory to archive and store as a test asset.

### Handling nested assets

:::note Nested Paths
Nested assets are stored **flat** in Sauce Labs. A test asset like `__assets__/mylogs/log.txt` would therefore be stored and available for download as `log.txt`.
Please keep that in mind when creating custom assets, as examples like `__assets__/mylogs/log.txt` and `__assets__/myotherlogs/log.txt` would eventually collide when persisted.
:::

For cypress and playwright, there are cases where you may want to override this default behaviour; e.g. your test framework generates
mhan83 marked this conversation as resolved.
Show resolved Hide resolved
an HTML report and you want to preserve the entire report directory and don't want the individual files to
be flattened and automatically attached as described above. In that case, you can set an environment variable
in your saucectl config to opt out of the default behaviour. When set, the configured output directory
for the test run will be honoured; e.g. it won't be overridden to `__assets__/`.

```yaml title= "example configuration"
artifacts:
retain:
report-directory: archived-report.zip

env:
SAUCE_SYNC_WEB_ASSETS: "true"
tianfeng92 marked this conversation as resolved.
Show resolved Hide resolved
```

When configured this way, the directory named `report-directory` will be archived as `archived-report.zip`.
To maintain backwards compatibility with our UI, some asset types (e.g. images, logs, etc.) in `report-directory` will still be automatically copied over to `__assets__/` and attached to the test results.

## Setting up a Proxy

Expand Down
29 changes: 29 additions & 0 deletions docs/web-apps/automated-testing/cucumberjs-playwright/yaml.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -587,6 +587,35 @@ artifacts:

---

### `retain`

<p><small>| OPTIONAL | OBJECT |</small></p>

Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can
be downloaded automatically using the `download` configuration, via the
[REST API](/dev/api/jobs/#get-a-job-asset-file), or through the test details page.

```yaml
artifacts:
retain:
source-directory: destination-archive.zip
download:
when: always
match:
- destination-archive.zip
directory: ./artifacts/
```

:::note
The source and destination will be relative to the `rootDir` defined in your configuration.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(observation) Sorta kinda weird to say it that way 😅 It's true, but technically wrong at the same time? After all, it's relative to the workspace directory on the VM. Which does mirror rootDir. I don't know which way is less of a confusing phrase to use. Might be fine the way it is and we can change it later if we see folks tripping over this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yah, I was waffling on this. I thought this was the most succinct way to describe it while being close enough to the truth.

:::

:::note
The destination archive must have a .zip file extension.
:::

---

### `download`

<p><small>| OPTIONAL | OBJECT |</small></p>
Expand Down
32 changes: 32 additions & 0 deletions docs/web-apps/automated-testing/cypress/yaml/v1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -613,10 +613,13 @@ Specifies how to manage test artifacts, such as logs, videos, and screenshots.
```yaml
artifacts:
cleanup: true
retain:
generated-report-dir: report.zip
download:
when: always
match:
- junit.xml
- report.zip
directory: ./artifacts/
```

Expand All @@ -635,6 +638,35 @@ artifacts:

---

### `retain`

<p><small>| OPTIONAL | OBJECT |</small></p>

Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can
be downloaded automatically using the `download` configuration, via the
[REST API](/dev/api/jobs/#get-a-job-asset-file), or through the test details page.

```yaml
artifacts:
retain:
source-directory: destination-archive.zip
download:
when: always
match:
- destination-archive.zip
directory: ./artifacts/
```

:::note
The source and destination will be relative to the `rootDir` defined in your configuration.
:::

:::note
The destination archive must have a .zip file extension.
:::

---

### `download`

<p><small>| OPTIONAL | OBJECT |</small></p>
Expand Down
29 changes: 29 additions & 0 deletions docs/web-apps/automated-testing/playwright/yaml.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -614,6 +614,35 @@ artifacts:

---

### `retain`

<p><small>| OPTIONAL | OBJECT |</small></p>

Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can
be downloaded automatically using the `download` configuration, via the
[REST API](/dev/api/jobs/#get-a-job-asset-file), or through the test details page.

```yaml
artifacts:
retain:
source-directory: destination-archive.zip
download:
when: always
match:
- destination-archive.zip
directory: ./artifacts/
```

:::note
The source and destination will be relative to the `rootDir` defined in your configuration.
:::

:::note
The destination archive must have a .zip file extension.
:::

---

### `download`

<p><small>| OPTIONAL | OBJECT |</small></p>
Expand Down
29 changes: 29 additions & 0 deletions docs/web-apps/automated-testing/replay/yaml.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -415,6 +415,35 @@ artifacts:

---

### `retain`

<p><small>| OPTIONAL | OBJECT |</small></p>

Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can
be downloaded automatically using the `download` configuration, via the
[REST API](https://docs.saucelabs.com/dev/api/jobs/#get-a-job-asset-file), or through the test details page.

```yaml
artifacts:
retain:
source-directory: destination-archive.zip
download:
when: always
match:
- destination-archive.zip
directory: ./artifacts/
```

:::note
The source and destination will be relative to the `rootDir` defined in your configuration.
:::

:::note
The destination archive must have a .zip file extension.
:::

---

### `download`

<p><small>| OPTIONAL | OBJECT |</small></p>
Expand Down
29 changes: 29 additions & 0 deletions docs/web-apps/automated-testing/testcafe/yaml.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -618,6 +618,35 @@ artifacts:

---

### `retain`

<p><small>| OPTIONAL | OBJECT |</small></p>

Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can
be downloaded automatically using the `download` configuration, via the
[REST API](/dev/api/jobs/#get-a-job-asset-file), or through the test details page.

```yaml
artifacts:
retain:
source-directory: destination-archive.zip
download:
when: always
match:
- destination-archive.zip
directory: ./artifacts/
```

:::note
The source and destination will be relative to the `rootDir` defined in your configuration.
:::

:::note
The destination archive must have a .zip file extension.
:::

---

### `download`

<p><small>| OPTIONAL | OBJECT |</small></p>
Expand Down
Loading