Skip to content

Commit

Permalink
add baseline override docs
Browse files Browse the repository at this point in the history
  • Loading branch information
Logan Graham authored and omacranger committed Nov 27, 2024
1 parent 557aaec commit 50530e4
Show file tree
Hide file tree
Showing 7 changed files with 163 additions and 4 deletions.
27 changes: 27 additions & 0 deletions docs/visual-testing/_partials/_baseline-overrides-csharp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
At the global / client level

```csharp
VisualClient = await VisualClient.Create(Driver, Region.UsWest1);
VisualClient.BaselineOverride = new BaselineOverride()
{
Browser = Browser.Chrome,
Device = "Desktop (1024x627)",
OperatingSystem = OperatingSystem.Windows,
OperatingSystemVersion = "10",
};
```

Or at the snapshot level

```csharp
await VisualClient.VisualCheck("Login Page", new VisualCheckOptions()
{
BaselineOverride = new BaselineOverride()
{
Browser = Browser.Chrome,
Device = "Desktop (1024x627)",
OperatingSystem = OperatingSystem.Windows,
OperatingSystemVersion = "10",
}
});
```
18 changes: 18 additions & 0 deletions docs/visual-testing/_partials/_baseline-overrides-python-robot.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
At the global / client level

```robot
*** Keywords ***
# Typically you'd set this up in your __init__.robot file at the top-level, right after creating your visual build.
Setup
${override} = Visual BaselineOverride browser=CHROME device=Desktop (1024x627) operating_system=WINDOWS operating_system_version=10
Visual Set Global BaselineOverride ${override}
```

Or at the snapshot level

```robot
*** Test Cases ***
Valid Login
${override} = Visual BaselineOverride browser=CHROME device=Desktop (1024x627) operating_system=WINDOWS operating_system_version=10
Visual Snapshot Login Page baseline_override=${override}
```
33 changes: 33 additions & 0 deletions docs/visual-testing/_partials/_baseline-overrides-python.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
At the global / client level

```python
from saucelabs_visual.client import SauceLabsVisual
from saucelabs_visual.typing import BaselineOverride, Browser, OperatingSystem

visual_client = SauceLabsVisual()
visual_client.baseline_override = BaselineOverride(
browser=Browser.CHROME,
device="Desktop (1024x627)",
operatingSystem=OperatingSystem.WINDOWS,
operatingSystemVersion="10",
)
```

Or at the snapshot level

```python
from saucelabs_visual.client import SauceLabsVisual
from saucelabs_visual.typing import BaselineOverride, Browser, OperatingSystem
# ...
visual_client = SauceLabsVisual()
visual_client.create_snapshot_from_webdriver(
"Login Page",
driver=driver,
baseline_override=BaselineOverride(
browser=Browser.CHROME,
device="Desktop (1024x627)",
operatingSystem=OperatingSystem.WINDOWS,
operatingSystemVersion="10",
)
)
```
38 changes: 38 additions & 0 deletions docs/visual-testing/_partials/_baseline-overrides-wdio.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
At the global / service level

```js
import { Browser, OperatingSystem } from '@saucelabs/visual';
export const config = {
// ...
services: [
'sauce',
[
'@saucelabs/wdio-sauce-visual-service',
{
baselineOverride: {
browser: Browser.Chrome,
device: "Desktop (1024x627)",
operatingSystem: OperatingSystem.Windows,
operatingSystemVersion: '10',
},
},
],
],
}
```

Or at the snapshot level

```js
import { Browser, OperatingSystem } from '@saucelabs/visual';
// ...
// Passing on a per-snapshot level
await browser.sauceVisualCheck('Inventory Page', {
baselineOverride: {
browser: Browser.Chrome,
device: "Desktop (1024x627)",
operatingSystem: OperatingSystem.Windows,
operatingSystemVersion: '10',
}
});
```
5 changes: 5 additions & 0 deletions docs/visual-testing/integrations/webdriverio.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ sidebar_label: WebdriverIO

import FullPageJS from '../_partials/_fullpage-js.md';
import ClippingDescription from '../_partials/_clipping-description.md';
import BaselineOverridesWDIO from '../_partials/_baseline-overrides-wdio.md';
import EnvironmentVariables from '../_partials/_environment-variables.md';
import SelectiveDiffing from '../_partials/_selective-diffing.md';
import SelectiveDiffingGlobal from '../_partials/_selective-diffing-global.md';
Expand Down Expand Up @@ -280,6 +281,10 @@ await browser.sauceVisualCheck('Visible Sale Banner', {
})
```

### Using Baseline Overrides

<BaselineOverridesWDIO />

## Example

An example project is available [here](https://github.com/saucelabs/visual-examples/tree/main/wdio).
46 changes: 42 additions & 4 deletions docs/visual-testing/workflows/cross-browser-os.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ sidebar_label: Cross Browser / OS
import useBaseUrl from '@docusaurus/useBaseUrl';
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import BaselineOverridesWDIO from '../_partials/_baseline-overrides-wdio.md';
import BaselineOverridesCSharp from '../_partials/_baseline-overrides-csharp.md';
import BaselineOverridesPython from '../_partials/_baseline-overrides-python.md';
import BaselineOverridesPythonRobot from '../_partials/_baseline-overrides-python-robot.md';

# Cross Browser / OS Visual Testing

Expand Down Expand Up @@ -60,7 +64,6 @@ You may want to target a browser height of ~1200px. We'll be using 1920x1200 as
return [window.outerWidth - window.innerWidth + arguments[0],
window.outerHeight - window.innerHeight + arguments[1]];
`, [1920, 1200]);
//
await browser.setWindowRect(0, 0, width, height);
},
}
Expand Down Expand Up @@ -109,14 +112,49 @@ With your new configuration use a binding & test framework of your choice create

### 3. Configure baseline overrides

With the baselines of your target browser saved gather the keys of the baseline run you wish to compare against. For cross browser / OS this will likely be the following:
With the baselines of your target browser saved gather the keys of the baseline run you wish to compare against. You can do this in the UI by navigating into a build, selecting a diff, viewing the snapshot meta, and clicking 'Copy as JSON.'

<video style={{width: '100%', height: 'auto'}} autoPlay loop title="Copy baseline hash keys">
<source src={useBaseUrl('/img/sauce-visual/snapshot-metadata.webm')} />
</video>

For cross browser / OS testing you will likely want the following:

- `operatingSystem`
- `operatingSystemVersion`
- `browser`
- `device`

Use the baseline override feature of your binding (see the documentation for your specific binding on the sidebar). A few examples are included below for reference:

<Tabs
defaultValue="wdio"
values={[
{ label: 'WDIO', value: 'wdio' },
{ label: 'C#', value: 'csharp' },
{ label: 'Python', value: 'python' },
{ label: 'Python (Robot)', value: 'robot' },
]}
>
<TabItem value="wdio">
<BaselineOverridesWDIO />
</TabItem>
<TabItem value="csharp">
<BaselineOverridesCSharp />
</TabItem>
<TabItem value="python">
<BaselineOverridesPython />
</TabItem>
<TabItem value="robot">
<BaselineOverridesPythonRobot />
</TabItem>
</Tabs>
### 4. Run your build

Run a new build for each browser you wish to compare with and optionally approve the snapshots for each browser to allow them to take over their own baselines.

## Limitations

- Scrollbars rendered in browser on Windows
- Browser dimensions need to be the same for the best results
- Scrollbars on Windows generally take up space which can cause invalid / incorrect viewport dimensions for snapshots. You can use workarounds such as [setting the scrollbar width](https://developer.mozilla.org/docs/Web/CSS/scrollbar-width) to disable them temporarily. We'll do our best to do this automatically while using a Sauce Visual runner.
- Browser dimensions need to be the same for the best results. While having a uniform width can cover most cases, having a uniform height will ensure that pages which center content (and do not have enough content to scroll) will match enough to be visually diffed.
Binary file added static/img/sauce-visual/snapshot-metadata.webm
Binary file not shown.

0 comments on commit 50530e4

Please sign in to comment.