diff --git a/docs/basics/acct-team-mgmt/adding-deleting-teams.md b/docs/basics/acct-team-mgmt/adding-deleting-teams.md index 827de93572..ef034a9549 100644 --- a/docs/basics/acct-team-mgmt/adding-deleting-teams.md +++ b/docs/basics/acct-team-mgmt/adding-deleting-teams.md @@ -24,7 +24,7 @@ If you are an organization admin, you can create and delete teams and assign con Create new team 4. In the **TO LINE OF BUSINESS** dropdown, select the line of business the team will be associated with. -5. Under **TEAM VM CONCURRENCY LIMITS**, use the up and down arrows to set the number of concurrent virtual machines that the team can access. For more information about concurrency, see [Concurrency Limits and Team Accounts](/basics/acct-team-mgmt/concurrency-limits). +5. Under **TEAM VM CONCURRENCY LIMITS**, use the up and down arrows to set the number of concurrent virtual machines that the team can access. For more information about concurrency, see [Managing Concurrency](/basics/acct-team-mgmt/concurrency/managing-concurrency). 6. Click **ADD TEAM**. ## Deleting Teams diff --git a/docs/basics/acct-team-mgmt/concurrency-limits.md b/docs/basics/acct-team-mgmt/concurrency-limits.md deleted file mode 100644 index e0707f1e5f..0000000000 --- a/docs/basics/acct-team-mgmt/concurrency-limits.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -id: concurrency-limits -title: Concurrency Limits and Team Accounts -sidebar_label: Concurrency Limits and Allocations for Teams ---- - -import useBaseUrl from '@docusaurus/useBaseUrl'; - -

Enterprise Plans only

- -If your organization has multiple teams sharing a Sauce Labs account, you can use Team Limits to ensure that your organization's concurrency is distributed among different teams. - -If you are an org admin, you can view and control your organization's concurrency limits: - -1. In Sauce Labs, click **ACCOUNT** and then click **Organization Management**. - -Organization management navigation - -2. On the **Organization Management** page, under the ORGANIZATION NAME box, concurrency limits will be displayed for the selected data center. - -3. To view the limits for another data center, in the upper-right corner of the window, click the **DATA CENTER** dropdown and select the relevant data center. - -Data Center dropdown - -4. On the **Organization Management** page, click the **TEAMS** tab. **Team VM Concurrency** and **Peak VM Concurrency** will be displayed for each team in the organization. - -Team concurrency limits - -For more usage information, see [Viewing and Exporting Usage Data](/basics/acct-team-mgmt/viewing-exporting-usage-data/). - -5. To allocate concurrency to teams, click on the team name to open the Teams tab. - -6. Under the Teams tab, Enter **Team VM Concurrency** and click on Update. - - -## How Concurrency Allocation Works - -Through Sauce Labs, organization admins set the total concurrency of their organization's account. When setting the concurrency limit for a team, be mindful that team members share the concurrency limit of the team. If a team exceeds its concurrency limit, any additional tests will be queued. Be informed that queuing will be disabled from May 1st, 2024 (upon contract renewal). - -#### Example - Org Limits - -An organization has a total concurrency of 500 VMs. Team A has a concurrency limit of 250, and Team B has a concurrency limit of 350. - -When Team A is using all 250 concurrency slots, that only leaves 250 of the organization's slots open. Even though Team B has a concurrency limit of 350, it will only be able to use the 250 remaining slots until Team A's tests finish. - -#### Example - Team Limits - -An organization admin allocates 100 VMs to Team A, which has five members. Since the members of a team share the team's VM limit, the number of concurrent tests being run by the five team members cannot exceed 100. However, concurrency limits cannot be set at the user level, so User A could be running 20 tests, while user B is running 50 tests, and users C, D, and E are running 10 tests each. Or User A could be running 100 tests while Users B, C, D, and E are not running any tests. - -## Checking Concurrency - -You can use the [Get User Concurrency](/dev/api/accounts/#get-user-concurrency) API endpoint to retrieve a specific user's concurrency usage compared with their organization and team concurrency allowances. - -```jsx title="Sample Concurrency Request" -curl -u "$SAUCE_USERNAME:$SAUCE_ACCESS_KEY" --location \ ---request GET 'https://api.us-west-1.saucelabs.com/rest/v1.2/users//concurrency' \ ---header 'Content-Type: application/json' \ | json_pp -``` - -:::caution - -Queuing will be disabled on May 1st, 2024 (implemented upon contract renewal). To better manage your usage, check the [Usage Analytics](/insights/usage-report) that Sauce Labs provides for you - -::: - -## Queueing Tests - -As tests are complete, queued tests are allocated to concurrency slots in the order they were queued. - -If you run more tests at once than your concurrency allows (e.g., you have a concurrency of 100 but send 200 tests at once), the extra tests will be queued and run as the first tests finish. A test is only allowed to remain in the queue for 10 minutes. If it has not been moved out of the queue before 10 minutes elapse, or if too many tests are already queued, the test is removed from the queue and not run at all. - -Queuing is intended to deal with small mistakes in concurrency management. Sauce Labs does not recommend queuing tests intentionally. diff --git a/docs/basics/acct-team-mgmt/concurrency/managing-concurrency.md b/docs/basics/acct-team-mgmt/concurrency/managing-concurrency.md new file mode 100644 index 0000000000..732d4e7e02 --- /dev/null +++ b/docs/basics/acct-team-mgmt/concurrency/managing-concurrency.md @@ -0,0 +1,68 @@ +--- +id: managing-concurrency +title: Managing Concurrency +sidebar_label: Managing Concurrency +--- + +import useBaseUrl from '@docusaurus/useBaseUrl'; + +Managing your concurrency throughout your organization is an important aspect of administering your Sauce Labs account. We provide multiple options to make it easy for you to optimize your usage and enable your releases to go out smoothly. + +#### Setting Parallel Limits in Common Frameworks +The first and foremost way to manage how many concurrent tests to run on Sauce Labs is to use the out of the box mechanisms available in your test framework. For convenience we have included some of the more populare frameworks below but please consult the documentation for your specific framework to learn more. + + + + + + + + + + +#### Usage Reporting & Notifications +Sauce Labs provides you robust mechanisms to inspect your organization's usage. This includes seeing your organization wide usage over time and a team breakdown. To learn more see [Usage Report](/insights/usage-report). + +We also provide notification mechanisms to alert you when your organization is reaching its subscription amount. To learn more about these see [Usage Notifications](/insights/usage-report). + +#### Team Management + +

Enterprise Plans only

+ +If your organization has multiple teams sharing a Sauce Labs account, you can use Team Limits to ensure that your organization's concurrency is distributed among different teams. + +If you are an org admin, you can view and control your organization's concurrency limits: + +1. In Sauce Labs, click **ACCOUNT** and then click **Organization Management**. + +Organization management navigation + +2. On the **Organization Management** page, under the ORGANIZATION NAME box, concurrency limits will be displayed for the selected data center. + +3. To view the limits for another data center, in the upper-right corner of the window, click the **DATA CENTER** dropdown and select the relevant data center. + +Data Center dropdown + +4. On the **Organization Management** page, click the **TEAMS** tab. **Team VM Concurrency** and **Peak VM Concurrency** will be displayed for each team in the organization. + +Team concurrency limits + +For more usage information, see [Viewing and Exporting Usage Data](/basics/acct-team-mgmt/viewing-exporting-usage-data/). + +5. To allocate concurrency to teams, click on the team name to open the Teams tab. + +6. Under the Teams tab, Enter **Team VM Concurrency** and click on Update. + +For an example, let's say the organization admin allocates 100 VMs to Team A, which has five members. Since the members of a team share the team's VM limit, the number of concurrent tests being run by the five team members cannot exceed 100. However, concurrency limits cannot be set at the user level, so User A could be running 20 tests, while user B is running 50 tests, and users C, D, and E are running 10 tests each. Or User A could be running 100 tests while Users B, C, D, and E are not running any tests. + +#### Concurrency API + +In some cases you might want to configure your test framework to dynamically pull real time information about your organization and teams concurrency usage. + +You can use the [Get User Concurrency](/dev/api/accounts/#get-user-concurrency) API endpoint to retrieve a specific user's concurrency usage compared with their organization and team concurrency allowances. + +```jsx title="Sample Concurrency Request" +curl -u "$SAUCE_USERNAME:$SAUCE_ACCESS_KEY" --location \ +--request GET 'https://api.us-west-1.saucelabs.com/rest/v1.2/users//concurrency' \ +--header 'Content-Type: application/json' \ | json_pp +``` \ No newline at end of file diff --git a/docs/basics/acct-team-mgmt/concurrency/understanding-concurrency.md b/docs/basics/acct-team-mgmt/concurrency/understanding-concurrency.md new file mode 100644 index 0000000000..705be43855 --- /dev/null +++ b/docs/basics/acct-team-mgmt/concurrency/understanding-concurrency.md @@ -0,0 +1,70 @@ +--- +id: understanding-concurrency +title: Understanding Concurrency +sidebar_label: Understanding Concurrency +--- + +import useBaseUrl from '@docusaurus/useBaseUrl'; + +Your subscription to the Sauce Labs Virtual Cloud or Real Device Cloud entitles you to run a certain amount of concurrent tests as many times as you like within a given region. In essence, this means you pay for the ability to run at your peak needs and you can run at that as frequently as you like. Your concurrency can be used for both automated and live testing. + +## Virtual Cloud Concurrency + +Concurrency for the Virtual Cloud is defined at two levels, the organization and the team level. + +To view your organizations concurrency limit: + +1. In Sauce Labs, click **ACCOUNT** and then click **Organization Management**. + +Organization management navigation + +2. On the **Organization Management** page, under the ORGANIZATION NAME box, concurrency limits will be displayed for the selected data center. + +3. To view the limits for another data center, in the upper-right corner of the window, click the **DATA CENTER** dropdown and select the relevant data center. + +Data Center dropdown + +To view the concurrency for a given team + +4. On the **Organization Management** page, click the **TEAMS** tab. **Team VM Concurrency** and **Peak VM Concurrency** will be displayed for each team in the organization. + +### Exceeding Virtual Cloud Concurrency + +The Virtual Cloud is used to run business critical pipelines across large enterprise organizations. This means there will be times when it is difficult to predict exactly how much concurrency you will need at all times. For these situations we allow your organization to exceed it's concurrency limit. + +The following table describes the maximum amount you are allowed to exceed your concurrency limit. + +| Subscription Amount | % You Can Exceed| Example | +|---------------------------------------|---------| +|1-100| 200%| 50 -> 150| +|101-500|100%|200 -> 400| +|501-2000|50%|1000 -> 1500| +|2001+|10%|2500 -> 2750| + +Once your organization attempts to run a test past the maximum amount you can exceed your tests will receive a [You've Exceeding Your Concurrency Limit Error](/dev/error-messages/#youve-exceeded-your-sauce-labs-concurrency-limit) + +To learn more about how to manage your concurrency, including setting team limits and receiving usage notifications, please read [Managing Concurrency](/basics/acct-team-mgmt/concurrency/managing-concurrency/) + + +## Real Device Cloud Concurrency + +If your subscription includes access to our Public Real Device Cloud, then your subscription quantity equals the amount of concurrent tests that can be run on Sauce Labs real devices. + +To view your organizations concurrency limit: + +1. In Sauce Labs, click **ACCOUNT** and then click **Organization Management**. + +Organization management navigation + +2. On the **Organization Management** page, under the ORGANIZATION NAME box, concurrency limits will be displayed for the selected data center. + +3. To view the limits for another data center, in the upper-right corner of the window, click the **DATA CENTER** dropdown and select the relevant data center. + +Data Center dropdown + +### Exceeding Real Device Public Cloud Concurrency + +If your organization attempts to run more tests than your subscription permits then your tests will be queued until an available concurrency slot becomes available. Your request for a device will queue for up to 15 minutes by default and it can be configured up to a maximum of 30 minutes by adjusting your [Test Configuration Options] (/dev/test-configuration-options). + +For example, if you have a Public Real Device Cloud concurrency limit of 5 and you attempt to run 10 tests, 5 of the tests will run immediately and the other 5 will be queued until one of the other tests finishes. + diff --git a/docs/basics/acct-team-mgmt/private-device-mgmt.md b/docs/basics/acct-team-mgmt/private-device-mgmt.md index e33db638a2..ec1b3b209c 100644 --- a/docs/basics/acct-team-mgmt/private-device-mgmt.md +++ b/docs/basics/acct-team-mgmt/private-device-mgmt.md @@ -31,7 +31,7 @@ Private device management allows an organization admin to view and manage the pr For more information about managing teams, see [Adding and Deleting Teams](/basics/acct-team-mgmt/adding-deleting-teams). Private devices are not included in VM or RDC concurrency limits. -For more details, please see [Concurrency Limits and Team Accounts](/basics/acct-team-mgmt/concurrency-limits). +For more details, please see [Managing Concurrency](/basics/acct-team-mgmt/concurrency/managing-concurrency). ::: ## Viewing an Organization’s Private Devices diff --git a/docs/basics/acct-team-mgmt/viewing-exporting-usage-data.md b/docs/basics/acct-team-mgmt/viewing-exporting-usage-data.md index 886be283b2..74c8789ff9 100644 --- a/docs/basics/acct-team-mgmt/viewing-exporting-usage-data.md +++ b/docs/basics/acct-team-mgmt/viewing-exporting-usage-data.md @@ -63,7 +63,7 @@ The USAGE widget provides a quick look at the device usage, by test session, for ## Minutes Used vs. Concurrency -One way to tell if you're getting the most efficient use out of your Sauce Labs plan is to compare the minutes used with the number of concurrent tests run during the same period. If the ratio of minutes to concurrency is low, for example, 2:1 (100 minutes:50 concurrent tests), then you are using a lot of minutes to run very few tests. You should redesign your tests to take greater advantage of concurrency. See [Using Frameworks to Run Tests in Parallel](/web-apps/automated-testing/selenium#using-frameworks-to-run-tests-in-parallel) and [Concurrency Limits and Team Accounts](/basics/acct-team-mgmt/concurrency-limits/) for more information. +One way to tell if you're getting the most efficient use out of your Sauce Labs plan is to compare the minutes used with the number of concurrent tests run during the same period. If the ratio of minutes to concurrency is low, for example, 2:1 (100 minutes:50 concurrent tests), then you are using a lot of minutes to run very few tests. You should redesign your tests to take greater advantage of concurrency. See [Using Frameworks to Run Tests in Parallel](/web-apps/automated-testing/selenium#using-frameworks-to-run-tests-in-parallel) and [Understanding Concurrency](/basics/acct-team-mgmt/concurrency/understanding-concurrency) for more information. ## Exporting Usage Data diff --git a/docs/basics/sso/setting-up-sso.md b/docs/basics/sso/setting-up-sso.md index 758450af45..caaab98527 100644 --- a/docs/basics/sso/setting-up-sso.md +++ b/docs/basics/sso/setting-up-sso.md @@ -239,7 +239,7 @@ Users who are created via SSO are placed into the default team. Organization admins can [assign users to other teams](/basics/acct-team-mgmt/assigning-removing-users-teams). -Additionally, you have the option to limit concurrency, which controls the number of virtual machines available for running tests simultaneously, for newly provisioned accounts. This can be configured in the [default team settings](/basics/acct-team-mgmt/concurrency-limits/). +Additionally, you have the option to limit concurrency, which controls the number of virtual machines available for running tests simultaneously, for newly provisioned accounts. This can be configured in the [default team settings](/basics/acct-team-mgmt/concurrency/managing-concurrency). #### Usernames diff --git a/docs/dev/cli/saucectl/usage/use-cases.md b/docs/dev/cli/saucectl/usage/use-cases.md index 01564f8246..fc7a9f93a1 100644 --- a/docs/dev/cli/saucectl/usage/use-cases.md +++ b/docs/dev/cli/saucectl/usage/use-cases.md @@ -75,7 +75,7 @@ saucectl run --ccy 10 A setting of `10` runs up to 10 test suites at the same time. If the test has more suites than that, excess suites are queued and run in order as currently running suites complete and new slots are available. -When running on Sauce Cloud, the maximum concurrency that you can use is defined by your [account settings](/basics/acct-team-mgmt/concurrency-limits/). +When running on Sauce Cloud, the maximum concurrency that you can use is defined by your [account settings](/basics/acct-team-mgmt/concurrency/understanding-concurrency). ## Sauce Connect diff --git a/docs/dev/glossary.md b/docs/dev/glossary.md index 901975b675..8e0535eacb 100644 --- a/docs/dev/glossary.md +++ b/docs/dev/glossary.md @@ -123,7 +123,7 @@ See also: [http client](#http-client). ### Concurrency Limit -The maximum number of total Sauce Labs tests -- both automated and manual -- that you can run simultaneously across all user accounts within your organization. Concurrency limits vary according to pricing plan. Once you and/or your teammates have used all concurrency slots, additional tests will not launch until an existing test has finished. More information: [Understanding Concurrency Limits and Team Accounts](/basics/acct-team-mgmt/concurrency-limits). +The maximum number of total Sauce Labs tests -- both automated and manual -- that you can run simultaneously across all user accounts within your organization. Concurrency limits vary according to pricing plan. Once you and/or your teammates have used all concurrency slots, additional tests will not launch until an existing test has finished. More information: [Understanding Concurrency](/basics/acct-team-mgmt/concurrency/understanding-concurrency). ### Continuous Deployment (CD) diff --git a/docs/insights/scope.md b/docs/insights/scope.md index c896819a5f..d3c27d8837 100644 --- a/docs/insights/scope.md +++ b/docs/insights/scope.md @@ -23,7 +23,7 @@ Access the **Insights** menu from the left-side navigation menu of our web app. | Job History | Shows a visual snapshot of the results for a specific test over time. See the [Test History](/insights/history) page for specific views and capabilities descriptions. | | Trends | Shows graphical visualizations of all tests. Applying filters to this view makes comparing test outcomes for different variables, such as the device browser version, manageable. See the [Trends](/insights/trends) page for detailed documentation. | | Coverage | Org Admin Only Shows the breakdown of browsers and devices against which you've run tests, giving you an idea of how comprehensively you are testing across different platforms so you can align your test strategy with your own market usage data. | -| VM Concurrency | Shows how many VM instances are in use simultaneously at any given time. See [Team Concurrency](/basics/acct-team-mgmt/concurrency-limits) for information about how concurrency is allocated. | +| VM Concurrency | Shows how many VM instances are in use simultaneously at any given time. See [Team Concurrency](/basics/acct-team-mgmt/concurrency/understanding-concurrency) for information about how concurrency is allocated. | | Failure Analysis | Exposes the results of the Sauce Labs machine learning algorithms that comb through every command run in every test and each error thrown in those tests to determine emerging patterns. See the [Failure Analysis](/insights/failure-analysis) page for detailed documentation. | ## Using Filters to Adjust the Scope of Your Data diff --git a/docs/visual/component-testing/integrations/sauce-labs.md b/docs/visual/component-testing/integrations/sauce-labs.md index 56e32d8ba4..344d5b0df4 100644 --- a/docs/visual/component-testing/integrations/sauce-labs.md +++ b/docs/visual/component-testing/integrations/sauce-labs.md @@ -25,7 +25,7 @@ To learn more about Screener's Cross-Browser Testing, see [Visual Component Cros ## What You'll Need - A valid Sauce Labs account. -- Access to enough [concurrency](/basics/acct-team-mgmt/concurrency-limits) in your Sauce account to run Screener tests. Each browser/resolution combination will use one concurrent machine. +- Access to enough [concurrency](/basics/acct-team-mgmt/concurrency/understanding-concurrency) in your Sauce account to run Screener tests. Each browser/resolution combination will use one concurrent machine. ## Adding Sauce Labs Browsers diff --git a/sidebars.js b/sidebars.js index f0fc6339a5..f84f811ff8 100644 --- a/sidebars.js +++ b/sidebars.js @@ -895,13 +895,21 @@ module.exports = { label: 'Managing Users and Accounts', collapsed: true, items: [ - 'basics/acct-team-mgmt/concurrency-limits', 'basics/acct-team-mgmt/adding-deactivating-users', 'basics/acct-team-mgmt/managing-user-info', 'basics/acct-team-mgmt/viewing-exporting-usage-data', 'basics/acct-team-mgmt/switching-active-team', ], }, + { + type: 'category', + label: 'Managing Concurrency', + collapsed: true, + items: [ + 'basics/acct-team-mgmt/concurrency/understanding-concurrency', + 'basics/acct-team-mgmt/concurrency/managing-concurrency', + ], + }, { type: 'category', label: 'Managing Teams',