Skip to content

Commit

Permalink
Merge pull request #2861 from saucelabs/DEVX-2970
Browse files Browse the repository at this point in the history
docs: Add dedicated limitations pages for JS frameworks
  • Loading branch information
tianfeng92 authored Jul 22, 2024
2 parents 97dae0f + bab40f4 commit 5c2527e
Show file tree
Hide file tree
Showing 7 changed files with 154 additions and 120 deletions.
34 changes: 1 addition & 33 deletions docs/web-apps/automated-testing/cypress.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,36 +169,4 @@ If you would prefer to stay in Cypress, try the new [Cypress Sauce Labs Plugin](

## Limitations

### Special Characters in Test Names

We recommend that you avoid the use of special characters when naming your tests. If your test name contains any special characters, your test may not run or its artifacts may not be visible in our platform.

### Cypress 11+

A bug impacting Cypress 11+ is causing the Cypress UI to display a browser that is not the one actually used.
Edge is shown on Windows, and Webkit is shown on macOS.

The browser selected in your configuration file is used, and the console log of Cypress is still correct.

The issue has been reported to Cypress.
See: https://github.com/cypress-io/cypress/issues/25755 for more information.

### Firefox 101 on Windows

Cypress does not currently work with Firefox 101 on Windows.

### Firefox 105

Cypress does not currently work with Firefox 105.
See https://github.com/cypress-io/cypress/issues/23897 for more information.

### Microsoft Edge 120+ on Windows

Cypress does not currently work with Microsoft Edge 120+ on Windows.

### Webkit

- Cypress only supports launching Webkit with a fixed resolution of 1280x720.
- Cypress 12.6.0 does not work on Windows with Webkit browser.
- Cypress 12+ does not work on macOS 11 with Webkit browser.
- Cypress 13.10.0+ does not work on macOS 12 with Webkit browser.
Please check the [Limitations Page](cypress/limitations.md).
43 changes: 43 additions & 0 deletions docs/web-apps/automated-testing/cypress/limitations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
id: limitations
title: Limitations for Cypress
sidebar_label: Limitations
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import useBaseUrl from '@docusaurus/useBaseUrl';

### Special Characters in Test Names

We recommend that you avoid the use of special characters when naming your tests. If your test name contains any special characters, your test may not run or its artifacts may not be visible in our platform.

### Cypress 11+

A bug impacting Cypress 11+ is causing the Cypress UI to display a browser that is not the one actually used.
Edge is shown on Windows, and Webkit is shown on macOS.

The browser selected in your configuration file is used, and the console log of Cypress is still correct.

The issue has been reported to Cypress.
See: https://github.com/cypress-io/cypress/issues/25755 for more information.

### Firefox 101 on Windows

Cypress does not currently work with Firefox 101 on Windows.

### Firefox 105

Cypress does not currently work with Firefox 105.
See https://github.com/cypress-io/cypress/issues/23897 for more information.

### Microsoft Edge 120+ on Windows

Cypress does not currently work with Microsoft Edge 120+ on Windows.

### Webkit

- Cypress only supports launching Webkit with a fixed resolution of 1280x720.
- Cypress 12.6.0 does not work on Windows with Webkit browser.
- Cypress 12+ does not work on macOS 11 with Webkit browser.
- Cypress 13.10.0+ does not work on macOS 12 with Webkit browser.
39 changes: 1 addition & 38 deletions docs/web-apps/automated-testing/playwright.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,41 +170,4 @@ If you prefer to stay in Playwright, try the [Playwright Sauce Labs Reporter](ht

## Limitations

### Special Characters in Test Names

We recommend that you avoid the use of special characters when naming your tests. If your test name contains any special characters, your test may not run or its artifacts may not be visible in our platform.

### Playwright 1.31.1 + Webkit + Sauce-Connect

When using Sauce-Connect, Webkit browser is unable to load any website that is hosted on the Internet.
Local websites can still be loaded.

### macOS 12 + Playwright 1.44+ + Webkit

Webkit is not working on macOS 12 since Playwright 1.44.0.

### macOS 11 + Playwright 1.29+ + Webkit

Webkit is not working on macOS 11 since Playwright 1.29.0.

### Playwright HTML Reporter + Trace Viewer

Because of a limitation in how assets are stored on our platform, configuring your tests to use the built-in [HTML reporter](https://playwright.dev/docs/test-reporters#html-reporter) and Playwright's [Trace Viewer](https://playwright.dev/docs/trace-viewer-intro) feature will not work as expected.

### Chrome launchOptions

If you use project specific `launchOptions` along with setting Chrome as the browser, you'll encounter the following error:

```
browserType.launch: Chromium distribution 'chrome' is not found at C:\Users\sauce\AppData\Local\Google\Chrome\Application\chrome.exe
```

This is because playwright's project specific settings override the global configuration that Sauce Labs provides during runtime. In this case, the location of the browser binary. Make sure that `launchOptions` are not set when running on a Sauce VM.

### Webkit + Windows

There is a [known issue](https://github.com/microsoft/playwright/issues/24512) in Playwright that prevents some sites from loading in Webkit on Windows with the error `SSL peer certificate or SSH remote key was not OK`. We recommend running your Playwright + Webkit tests on our Mac VMs by setting the [platformName](https://docs.saucelabs.com/web-apps/automated-testing/playwright/yaml/#platformname) to either `macOS 13` or `macOS 12`.

:::note
Tests running through Sauce Tunnels are unaffected.
:::
Please check the [Limitations Page](playwright/limitations.md).
49 changes: 49 additions & 0 deletions docs/web-apps/automated-testing/playwright/limitations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
id: limitations
title: Limitations for Playwright
sidebar_label: Limitations
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import useBaseUrl from '@docusaurus/useBaseUrl';


### Special Characters in Test Names

We recommend that you avoid the use of special characters when naming your tests. If your test name contains any special characters, your test may not run or its artifacts may not be visible in our platform.

### Playwright 1.31.1 + Webkit + Sauce-Connect

When using Sauce-Connect, Webkit browser is unable to load any website that is hosted on the Internet.
Local websites can still be loaded.

### macOS 12 + Playwright 1.44+ + Webkit

Webkit is not working on macOS 12 since Playwright 1.44.0.

### macOS 11 + Playwright 1.29+ + Webkit

Webkit is not working on macOS 11 since Playwright 1.29.0.

### Playwright HTML Reporter + Trace Viewer

Because of a limitation in how assets are stored on our platform, configuring your tests to use the built-in [HTML reporter](https://playwright.dev/docs/test-reporters#html-reporter) and Playwright's [Trace Viewer](https://playwright.dev/docs/trace-viewer-intro) feature will not work as expected.

### Chrome launchOptions

If you use project specific `launchOptions` along with setting Chrome as the browser, you'll encounter the following error:

```
browserType.launch: Chromium distribution 'chrome' is not found at C:\Users\sauce\AppData\Local\Google\Chrome\Application\chrome.exe
```

This is because playwright's project specific settings override the global configuration that Sauce Labs provides during runtime. In this case, the location of the browser binary. Make sure that `launchOptions` are not set when running on a Sauce VM.

### Webkit + Windows

There is a [known issue](https://github.com/microsoft/playwright/issues/24512) in Playwright that prevents some sites from loading in Webkit on Windows with the error `SSL peer certificate or SSH remote key was not OK`. We recommend running your Playwright + Webkit tests on our Mac VMs by setting the [platformName](https://docs.saucelabs.com/web-apps/automated-testing/playwright/yaml/#platformname) to either `macOS 13` or `macOS 12`.

:::note
Tests running through Sauce Tunnels are unaffected.
:::
50 changes: 1 addition & 49 deletions docs/web-apps/automated-testing/testcafe.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,52 +168,4 @@ If all you want is to publish your TestCafe test results to Sauce Labs (but not

## Limitations

### Special Characters in Test Names

We recommend that you avoid using special characters when naming your tests. If your test name contains any special characters, your test may not run, or its artifacts may not be visible on our platform.

### TestCafe 3.0.1 + Chrome/Edge + Sauce Connect

When using Sauce-Connect, Chrome, and Edge browsers cannot load any website through the tunnel. It will behave as if there is no tunnel defined.

### TestCafe Native Automation + Chrome + Sauce Connect

If your tests are issuing [TestCafe HTTP requests](https://testcafe.io/documentation/403971/guides/intermediate-guides/api-testing#proxy-settings)
and require a Sauce Connect tunnel, you will need to either [set the proxy manually](https://testcafe.io/documentation/403971/guides/intermediate-guides/api-testing#proxy-settings) or disable TestCafe's Native Automation.

#### Disable Native Automation

This is the recommended approach, which poses less of a hassle.
Disable Native Automation in your `.testcaferc.js`:
```javascript
module.exports = {
disableNativeAutomation: true,
};
```

and then ensure that our runner picks up the TestCafe config file by also
specifying it in the saucectl yaml config:
```yaml
testcafe:
version: 3.4.0
configFile: .testcaferc.js
```
#### Manually Setting the Request Proxy:
Alternatively, you can apply the proxy settings as you make requests:
```javascript
// HTTP_PROXY is pre-populated when using Sauce Connect
const items = process.env.HTTP_PROXY.split(':');
const host = items[1].replaceAll('/', '');
const port = items[2];
const response = await t.request({
url: `http://some-internal-resource.example.com/`,
method: 'get',
proxy: {
protocol: 'http',
host,
port,
}
});
```
Please check the [Limitations Page](testcafe/limitations.md).
56 changes: 56 additions & 0 deletions docs/web-apps/automated-testing/testcafe/limitations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
id: limitations
title: Limitations for TestCafe
sidebar_label: Limitations
---


### Special Characters in Test Names

We recommend that you avoid using special characters when naming your tests. If your test name contains any special characters, your test may not run, or its artifacts may not be visible on our platform.

### TestCafe 3.0.1 + Chrome/Edge + Sauce Connect

When using Sauce-Connect, Chrome, and Edge browsers cannot load any website through the tunnel. It will behave as if there is no tunnel defined.

### TestCafe Native Automation + Chrome + Sauce Connect

If your tests are issuing [TestCafe HTTP requests](https://testcafe.io/documentation/403971/guides/intermediate-guides/api-testing#proxy-settings)
and require a Sauce Connect tunnel, you will need to either [set the proxy manually](https://testcafe.io/documentation/403971/guides/intermediate-guides/api-testing#proxy-settings) or disable TestCafe's Native Automation.

#### Disable Native Automation

This is the recommended approach, which poses less of a hassle.
Disable Native Automation in your `.testcaferc.js`:
```javascript
module.exports = {
disableNativeAutomation: true,
};
```

and then ensure that our runner picks up the TestCafe config file by also
specifying it in the saucectl yaml config:
```yaml
testcafe:
version: 3.4.0
configFile: .testcaferc.js
```
#### Manually Setting the Request Proxy:
Alternatively, you can apply the proxy settings as you make requests:
```javascript
// HTTP_PROXY is pre-populated when using Sauce Connect
const items = process.env.HTTP_PROXY.split(':');
const host = items[1].replaceAll('/', '');
const port = items[2];
const response = await t.request({
url: `http://some-internal-resource.example.com/`,
method: 'get',
proxy: {
protocol: 'http',
host,
port,
}
});
```
3 changes: 3 additions & 0 deletions sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -1484,6 +1484,7 @@ module.exports = {
],
},
'web-apps/automated-testing/cypress/advanced',
'web-apps/automated-testing/cypress/limitations',
],
},
{
Expand All @@ -1496,6 +1497,7 @@ module.exports = {
'web-apps/automated-testing/playwright/selenium-grid',
'web-apps/automated-testing/playwright/yaml',
'web-apps/automated-testing/playwright/advanced',
'web-apps/automated-testing/playwright/limitations',
],
},
{
Expand All @@ -1518,6 +1520,7 @@ module.exports = {
'web-apps/automated-testing/testcafe/quickstart',
'web-apps/automated-testing/testcafe/yaml',
'web-apps/automated-testing/testcafe/advanced',
'web-apps/automated-testing/testcafe/limitations',
],
},
{
Expand Down

0 comments on commit 5c2527e

Please sign in to comment.