From a6d983b101062a4e3806eb4e6794f02a4368cfe0 Mon Sep 17 00:00:00 2001 From: Mike Han Date: Mon, 22 Apr 2024 15:13:27 -0600 Subject: [PATCH 1/7] docs: Document the retain feature --- .../automated-testing/cypress/yaml/v1.md | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/docs/web-apps/automated-testing/cypress/yaml/v1.md b/docs/web-apps/automated-testing/cypress/yaml/v1.md index 179a81829c..8aa05b6ff6 100644 --- a/docs/web-apps/automated-testing/cypress/yaml/v1.md +++ b/docs/web-apps/automated-testing/cypress/yaml/v1.md @@ -613,10 +613,13 @@ Specifies how to manage test artifacts, such as logs, videos, and screenshots. ```yaml artifacts: cleanup: true + retain: + generated-report-dir: report.zip download: when: always match: - junit.xml + - report.zip directory: ./artifacts/ ``` @@ -635,6 +638,35 @@ artifacts: --- +### `retain` + +

| OPTIONAL | OBJECT |

+ +Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can +be downloaded automatically using the `download` configuration or via the +[REST API](https://docs.saucelabs.com/dev/api/jobs/#get-a-job-asset-file) or through the test details page. + +```yaml +artifacts: + retain: + source-directory: destination-archive.zip + download: + when: always + match: + - destination-archive.zip + directory: ./artifacts/ +``` + +:::note +The source and destination will be relative to the `rootDir` defined in your configuration. +::: + +:::note +The destination archive must have a .zip file extension. +::: + +--- + ### `download`

| OPTIONAL | OBJECT |

From 0b01d0c859d0215481613a5ab7fbbc062c72d311 Mon Sep 17 00:00:00 2001 From: Mike Han Date: Mon, 22 Apr 2024 15:38:42 -0600 Subject: [PATCH 2/7] docs: Document retain feature --- .../cucumberjs-playwright/yaml.md | 29 +++++++++++++++++++ .../automated-testing/cypress/yaml/v1.md | 4 +-- .../automated-testing/playwright/yaml.md | 29 +++++++++++++++++++ .../web-apps/automated-testing/replay/yaml.md | 29 +++++++++++++++++++ .../automated-testing/testcafe/yaml.md | 29 +++++++++++++++++++ 5 files changed, 118 insertions(+), 2 deletions(-) diff --git a/docs/web-apps/automated-testing/cucumberjs-playwright/yaml.md b/docs/web-apps/automated-testing/cucumberjs-playwright/yaml.md index 1b18cdcd91..d8a534c9cf 100644 --- a/docs/web-apps/automated-testing/cucumberjs-playwright/yaml.md +++ b/docs/web-apps/automated-testing/cucumberjs-playwright/yaml.md @@ -587,6 +587,35 @@ artifacts: --- +### `retain` + +

| OPTIONAL | OBJECT |

+ +Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can +be downloaded automatically using the `download` configuration, via the +[REST API](https://docs.saucelabs.com/dev/api/jobs/#get-a-job-asset-file), or through the test details page. + +```yaml +artifacts: + retain: + source-directory: destination-archive.zip + download: + when: always + match: + - destination-archive.zip + directory: ./artifacts/ +``` + +:::note +The source and destination will be relative to the `rootDir` defined in your configuration. +::: + +:::note +The destination archive must have a .zip file extension. +::: + +--- + ### `download`

| OPTIONAL | OBJECT |

diff --git a/docs/web-apps/automated-testing/cypress/yaml/v1.md b/docs/web-apps/automated-testing/cypress/yaml/v1.md index 8aa05b6ff6..8bb7c7216e 100644 --- a/docs/web-apps/automated-testing/cypress/yaml/v1.md +++ b/docs/web-apps/automated-testing/cypress/yaml/v1.md @@ -643,8 +643,8 @@ artifacts:

| OPTIONAL | OBJECT |

Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can -be downloaded automatically using the `download` configuration or via the -[REST API](https://docs.saucelabs.com/dev/api/jobs/#get-a-job-asset-file) or through the test details page. +be downloaded automatically using the `download` configuration, via the +[REST API](https://docs.saucelabs.com/dev/api/jobs/#get-a-job-asset-file), or through the test details page. ```yaml artifacts: diff --git a/docs/web-apps/automated-testing/playwright/yaml.md b/docs/web-apps/automated-testing/playwright/yaml.md index bc8f72782c..d6b848b68d 100644 --- a/docs/web-apps/automated-testing/playwright/yaml.md +++ b/docs/web-apps/automated-testing/playwright/yaml.md @@ -614,6 +614,35 @@ artifacts: --- +### `retain` + +

| OPTIONAL | OBJECT |

+ +Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can +be downloaded automatically using the `download` configuration, via the +[REST API](https://docs.saucelabs.com/dev/api/jobs/#get-a-job-asset-file), or through the test details page. + +```yaml +artifacts: + retain: + source-directory: destination-archive.zip + download: + when: always + match: + - destination-archive.zip + directory: ./artifacts/ +``` + +:::note +The source and destination will be relative to the `rootDir` defined in your configuration. +::: + +:::note +The destination archive must have a .zip file extension. +::: + +--- + ### `download`

| OPTIONAL | OBJECT |

diff --git a/docs/web-apps/automated-testing/replay/yaml.md b/docs/web-apps/automated-testing/replay/yaml.md index 685e3503ec..ee90c21065 100644 --- a/docs/web-apps/automated-testing/replay/yaml.md +++ b/docs/web-apps/automated-testing/replay/yaml.md @@ -415,6 +415,35 @@ artifacts: --- +### `retain` + +

| OPTIONAL | OBJECT |

+ +Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can +be downloaded automatically using the `download` configuration, via the +[REST API](https://docs.saucelabs.com/dev/api/jobs/#get-a-job-asset-file), or through the test details page. + +```yaml +artifacts: + retain: + source-directory: destination-archive.zip + download: + when: always + match: + - destination-archive.zip + directory: ./artifacts/ +``` + +:::note +The source and destination will be relative to the `rootDir` defined in your configuration. +::: + +:::note +The destination archive must have a .zip file extension. +::: + +--- + ### `download`

| OPTIONAL | OBJECT |

diff --git a/docs/web-apps/automated-testing/testcafe/yaml.md b/docs/web-apps/automated-testing/testcafe/yaml.md index e5da868150..3392e6fa64 100644 --- a/docs/web-apps/automated-testing/testcafe/yaml.md +++ b/docs/web-apps/automated-testing/testcafe/yaml.md @@ -618,6 +618,35 @@ artifacts: --- +### `retain` + +

| OPTIONAL | OBJECT |

+ +Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can +be downloaded automatically using the `download` configuration, via the +[REST API](https://docs.saucelabs.com/dev/api/jobs/#get-a-job-asset-file), or through the test details page. + +```yaml +artifacts: + retain: + source-directory: destination-archive.zip + download: + when: always + match: + - destination-archive.zip + directory: ./artifacts/ +``` + +:::note +The source and destination will be relative to the `rootDir` defined in your configuration. +::: + +:::note +The destination archive must have a .zip file extension. +::: + +--- + ### `download`

| OPTIONAL | OBJECT |

From ed10d3fa15bf368f859bc2d28a1c5b4995116e81 Mon Sep 17 00:00:00 2001 From: Mike Han Date: Tue, 23 Apr 2024 09:54:23 -0600 Subject: [PATCH 3/7] Describe the V2 assets approach --- .../automated-testing/_partials/_advanced.md | 30 +++++++++++++++++-- .../automated-testing/cypress/yaml/v1.md | 2 +- 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/docs/web-apps/automated-testing/_partials/_advanced.md b/docs/web-apps/automated-testing/_partials/_advanced.md index e3ac28f694..39266ae05a 100644 --- a/docs/web-apps/automated-testing/_partials/_advanced.md +++ b/docs/web-apps/automated-testing/_partials/_advanced.md @@ -153,12 +153,36 @@ This feature is highly experimental. ## Attaching Test Assets -Any test assets created by your tests at runtime (such as logs, screenshots or reports) you wish to retain along with your test results must be placed in the `__assets__` directory of your project root folder. On Sauce Labs VMs, this path is relative to the current working directory. +By default, any test assets created by your tests at runtime (such as logs, screenshots or reports) you wish to retain along with your test results must be placed in the `__assets__` directory of your project root folder. On Sauce Labs VMs, this path is relative to the current working directory. + +### Attaching entire directories + +In situations where you want to preserve the file structure of your assets (e.g. a multi-page HTML report), +you can use the `retain` feature to define a directory to archive and store as a test asset. + +### Handling nested assets -:::note Nested Paths Nested assets are stored **flat** in Sauce Labs. A test asset like `__assets__/mylogs/log.txt` would therefore be stored and available for download as `log.txt`. Please keep that in mind when creating custom assets, as examples like `__assets__/mylogs/log.txt` and `__assets__/myotherlogs/log.txt` would eventually collide when persisted. -::: + +There are cases where you may want to override this default behaviour; e.g. your test framework generates +an HTML report and you want to preserve the entire report directory and don't want the individual files to +be flattened and automatically attached as described above. In that case, you can set an environment variable +in your saucectl config to opt out of the default behaviour. When set, the configured output directory +for the test run will be honoured; e.g. it won't be overridden to `__assets__/`. + +```yaml title= "example configuration" +artifacts: + retain: + report-directory: archived-report.zip + +env: + SAUCE_SYNC_WEB_ASSETS: "true" +``` + +When configured this way, the directory named `report-directory` will be archived as `archived-report.zip`. +To maintain backwards compatibility, some asset types (e.g. images, logs, etc.) in `report-directory` will be +automatically attached to the test results. ## Setting up a Proxy diff --git a/docs/web-apps/automated-testing/cypress/yaml/v1.md b/docs/web-apps/automated-testing/cypress/yaml/v1.md index 8bb7c7216e..7c3a99d91a 100644 --- a/docs/web-apps/automated-testing/cypress/yaml/v1.md +++ b/docs/web-apps/automated-testing/cypress/yaml/v1.md @@ -644,7 +644,7 @@ artifacts: Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can be downloaded automatically using the `download` configuration, via the -[REST API](https://docs.saucelabs.com/dev/api/jobs/#get-a-job-asset-file), or through the test details page. +[REST API](/dev/api/jobs/#get-a-job-asset-file), or through the test details page. ```yaml artifacts: From af29aaf33bedcb823d6540a04a049a968fd1082e Mon Sep 17 00:00:00 2001 From: Mike Han Date: Tue, 23 Apr 2024 10:06:12 -0600 Subject: [PATCH 4/7] link via path instead of url --- docs/web-apps/automated-testing/cucumberjs-playwright/yaml.md | 2 +- docs/web-apps/automated-testing/playwright/yaml.md | 2 +- docs/web-apps/automated-testing/testcafe/yaml.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/web-apps/automated-testing/cucumberjs-playwright/yaml.md b/docs/web-apps/automated-testing/cucumberjs-playwright/yaml.md index d8a534c9cf..891da9c816 100644 --- a/docs/web-apps/automated-testing/cucumberjs-playwright/yaml.md +++ b/docs/web-apps/automated-testing/cucumberjs-playwright/yaml.md @@ -593,7 +593,7 @@ artifacts: Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can be downloaded automatically using the `download` configuration, via the -[REST API](https://docs.saucelabs.com/dev/api/jobs/#get-a-job-asset-file), or through the test details page. +[REST API](/dev/api/jobs/#get-a-job-asset-file), or through the test details page. ```yaml artifacts: diff --git a/docs/web-apps/automated-testing/playwright/yaml.md b/docs/web-apps/automated-testing/playwright/yaml.md index d6b848b68d..4a9fb90e96 100644 --- a/docs/web-apps/automated-testing/playwright/yaml.md +++ b/docs/web-apps/automated-testing/playwright/yaml.md @@ -620,7 +620,7 @@ artifacts: Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can be downloaded automatically using the `download` configuration, via the -[REST API](https://docs.saucelabs.com/dev/api/jobs/#get-a-job-asset-file), or through the test details page. +[REST API](/dev/api/jobs/#get-a-job-asset-file), or through the test details page. ```yaml artifacts: diff --git a/docs/web-apps/automated-testing/testcafe/yaml.md b/docs/web-apps/automated-testing/testcafe/yaml.md index 3392e6fa64..fa6769d970 100644 --- a/docs/web-apps/automated-testing/testcafe/yaml.md +++ b/docs/web-apps/automated-testing/testcafe/yaml.md @@ -624,7 +624,7 @@ artifacts: Define directories to archive and retain as a test asset at the end of a test run. Archived test assets can be downloaded automatically using the `download` configuration, via the -[REST API](https://docs.saucelabs.com/dev/api/jobs/#get-a-job-asset-file), or through the test details page. +[REST API](/dev/api/jobs/#get-a-job-asset-file), or through the test details page. ```yaml artifacts: From 9b0eca870938dd268fe544d81dbe88c65ff943de Mon Sep 17 00:00:00 2001 From: Mike Han <56001373+mhan83@users.noreply.github.com> Date: Tue, 23 Apr 2024 10:43:13 -0600 Subject: [PATCH 5/7] Update docs/web-apps/automated-testing/_partials/_advanced.md Co-authored-by: Alex Plischke --- docs/web-apps/automated-testing/_partials/_advanced.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/web-apps/automated-testing/_partials/_advanced.md b/docs/web-apps/automated-testing/_partials/_advanced.md index 39266ae05a..553c2a5359 100644 --- a/docs/web-apps/automated-testing/_partials/_advanced.md +++ b/docs/web-apps/automated-testing/_partials/_advanced.md @@ -181,8 +181,7 @@ env: ``` When configured this way, the directory named `report-directory` will be archived as `archived-report.zip`. -To maintain backwards compatibility, some asset types (e.g. images, logs, etc.) in `report-directory` will be -automatically attached to the test results. +To maintain backwards compatibility with our UI, some asset types (e.g. images, logs, etc.) in `report-directory` will still be automatically copied over to `__assets__/` and attached to the test results. ## Setting up a Proxy From c512ba5e2dcd817495ee7eb4dd246dbd9b98658a Mon Sep 17 00:00:00 2001 From: Mike Han Date: Tue, 23 Apr 2024 10:50:42 -0600 Subject: [PATCH 6/7] note v2 method is only for cypress and playwright --- docs/web-apps/automated-testing/_partials/_advanced.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/web-apps/automated-testing/_partials/_advanced.md b/docs/web-apps/automated-testing/_partials/_advanced.md index 553c2a5359..0d08c2b80d 100644 --- a/docs/web-apps/automated-testing/_partials/_advanced.md +++ b/docs/web-apps/automated-testing/_partials/_advanced.md @@ -165,7 +165,7 @@ you can use the `retain` feature to define a directory to archive and store as a Nested assets are stored **flat** in Sauce Labs. A test asset like `__assets__/mylogs/log.txt` would therefore be stored and available for download as `log.txt`. Please keep that in mind when creating custom assets, as examples like `__assets__/mylogs/log.txt` and `__assets__/myotherlogs/log.txt` would eventually collide when persisted. -There are cases where you may want to override this default behaviour; e.g. your test framework generates +For cypress and playwright, there are cases where you may want to override this default behaviour; e.g. your test framework generates an HTML report and you want to preserve the entire report directory and don't want the individual files to be flattened and automatically attached as described above. In that case, you can set an environment variable in your saucectl config to opt out of the default behaviour. When set, the configured output directory From 99670d0a4c06360ec0868308664373701d40a32a Mon Sep 17 00:00:00 2001 From: Mike Han <56001373+mhan83@users.noreply.github.com> Date: Tue, 23 Apr 2024 11:58:27 -0600 Subject: [PATCH 7/7] Update docs/web-apps/automated-testing/_partials/_advanced.md Co-authored-by: Alex Plischke --- docs/web-apps/automated-testing/_partials/_advanced.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/web-apps/automated-testing/_partials/_advanced.md b/docs/web-apps/automated-testing/_partials/_advanced.md index 0d08c2b80d..3d96dec47d 100644 --- a/docs/web-apps/automated-testing/_partials/_advanced.md +++ b/docs/web-apps/automated-testing/_partials/_advanced.md @@ -165,7 +165,7 @@ you can use the `retain` feature to define a directory to archive and store as a Nested assets are stored **flat** in Sauce Labs. A test asset like `__assets__/mylogs/log.txt` would therefore be stored and available for download as `log.txt`. Please keep that in mind when creating custom assets, as examples like `__assets__/mylogs/log.txt` and `__assets__/myotherlogs/log.txt` would eventually collide when persisted. -For cypress and playwright, there are cases where you may want to override this default behaviour; e.g. your test framework generates +For Cypress and Playwright, there are cases where you may want to override this default behavior; e.g. your test framework generates an HTML report and you want to preserve the entire report directory and don't want the individual files to be flattened and automatically attached as described above. In that case, you can set an environment variable in your saucectl config to opt out of the default behaviour. When set, the configured output directory