How do you run Cucumber features in Parallel? Add examples

[gkushang]

I am looking at running Cucumber features in parallel. Do you have any example?

[spnraju]

Hi @gkushang this is feature of cucumberjs itself. I remember reading somewhere that it is still on experimental stage. But on googling found this link. https://stackoverflow.com/questions/45037239/running-cucumberjs-feature-files-in-parallel. Ideally, there should not be of too much difference. We will think about adding a example but if you figure out and send us a PR, we will be glad to have it merged,

[mucsi96]

This is the official documentation related to parallel execution https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#parallel-experimental

[spnraju]

@gkushang Did it work?

[mucsi96]

Closing this due to no activity

[fedika]

hi @spnraju could you please add the example? 👍
I just upgraded to nightwatch-api

[tSte]

@mucsi96 @spnraju I'm pretty much on the same page. Cucumber creates slaves correctly, but it seems that Chromedriver session is already used by first slave...

Error: ChromeDriver process exited with code: 1
[1568972747.864][SEVERE]: bind() failed: Address already in use (98)                                                                                                                                                       
       at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)
[1568972747.864][SEVERE]: bind() failed: Address already in use (98)
 
VError: a BeforeAll hook errored, process exiting: cucumber.conf.js:18: ChromeDriver process exited with code: 1

We migrated to latest NW/Cucumber and removed Selenium...

"chromedriver": "^76.0.1",
"cucumber": "^5.1.0",
"cucumber-html-reporter": "^5.0.2",
"cucumber-pretty": "^1.5.2",
"nightwatch": "^1.2.3",
"nightwatch-api": "^2.3.0",

I can share nightwatch.conf.js and cucumber.conf.js if you wish...

[spnraju]

Hi @fergadipa , we will consider adding that but it will definitely take time as both of us are held up a little bit. Kindly note that we will be glad if someone can raise a Pull request for the same.

Hi @tSte Thank you for the details. configs will be useful in analyzing the issue more.

[tSte]

Okay then. Here's nightwatch.conf.js:

import { path as driverPath } from 'chromedriver';
import { launchUrl, isLocalhost, isHeadless } from './env.conf';

const globals = {
  retryAssertionTimeout: 5 * 1000,
  waitForConditionPollInterval: 100,
  waitForConditionTimeout: (isLocalhost ? 10 : 50) * 1000,
};

const defaultChromeOptions = ['window-size=1920,1080', 'disable-gpu'];
const headlessChromeOptions = [...defaultChromeOptions, 'no-sandbox', 'headless'];

const desiredCapabilities = {
  browserName: 'chrome',
  test_workers: true,
  javascriptEnabled: true,
  acceptSslCerts: true,
  webStorageEnabled: true,
  locationContextEnabled: true,
  applicationCacheEnabled: true,
  chromeOptions: {
    args: isHeadless ? headlessChromeOptions : defaultChromeOptions,
  },
  loggingPrefs: { browser: 'ALL' },
};

const screenshots = {
  enabled: true,
  on_error: false,
  on_failure: true,
  path: './reports/screenshots',
};

const webdriver = {
  start_process: true,
  server_path: driverPath,
  port: 4444,
  cli_args: ['--port=4444'],
};

const config = {
  src_folders: ['./steps'],
  output_folder: './reports',
  custom_commands_path: './commands',
  custom_assertions_path: './assertions',
  webdriver,
  test_settings: {
    default: { webdriver, screenshots, desiredCapabilities, globals, launch_url: launchUrl },
  },
};

export default config;

The cucumber.conf.js is pretty much the same as in docs:

import { setDefaultTimeout, AfterAll, BeforeAll } from 'cucumber';
import { createSession, closeSession, startWebDriver, stopWebDriver } from 'nightwatch-api';

setDefaultTimeout(60000);

BeforeAll(async () => {
  await startWebDriver();
  await createSession();
});

AfterAll(async () => {
  await closeSession();
  await stopWebDriver();
});

NPM script:
"test": "cucumber-js --require cucumber.conf.js --require steps --format node_modules/cucumber-pretty --require-module @babel/register --require-module @babel/polyfil --parallel=4"

[tSte]

@spnraju @mucsi96 guys, could you give us some insight? Is this a nightwatch-api's bug or you do have instances running tests (using this stack) in parallel? If so, could you pls share your configs so we can find out what's wrong? Many thanks!

[icloudphil]

@tSte the config looks ok to me. I have not run into the issue as you experience with the latest nightwatch-api, cucumber 6, (even with older version of cucumber 5 and nightwatch-api, it was fine) Are you trying to run this in the Selenium Grid or just on your local sandbox?on local, I would suggest try use the Selenium-grid-node docker. there is docker example https://github.com/expphchen/nightwatch-starter/blob/master/docker-compose.yml and the instrucation: https://github.com/expphchen/nightwatch-starter/blob/master/README.md

[spnraju]

Hello All, a script is added in selenium example to run feature files in parallel here. There seems to be some issue in doing the same when we are using a chrome driver because of some port issues. I have tried this with pure Nightwatch and it seems to be working. I will investigate further and let you know if I can find a way to do it.

[RazvanVuscan]

Hello @spnraju , have you had any success with finding a way to run in parallel using the '--parallel' parameter?

[tejasshekokar]

+1

[icloudphil]

@RazvanVuscan @tejasshekokar cucumber-js --require-module babel-core/register --require cucumber.conf.js --require features/ --format json:reports/cucumber.json --format node_modules/cucumber-pretty --parallel 2

We have success to use the --parallel , as well as we are able to run this in selenium-grid and local chrome browser, but the local chrome browser, we see sometime the issue starting the browser due to the port, so we use selenium-grid docker image, and it works quite well.

Here is the detail docker compose file, https://github.com/icloudphil/nightwatch-starter/blob/master/docker-compose.yml
and instruction on how to install docker: https://github.com/icloudphil/nightwatch-starter/readme.md (you can ignore the standalone night watch part)

[drabelo]

Still having issues with this. Using --parallel 2 causes the second process to fail binding.

How does the test runner know the port of the second process?

[humphreyn]

I use cucumber and had the same issue. I resolved it by setting a different port number for each process by using a function that returns the port number as follows, see below. In my Nighwatch config I just call that function.

Please note, this is only needed when using the "start_process: true" option

function getPort() {
let port = 4444;
if (process.env.CUCUMBER_PARALLEL) {
port = port + parseInt(process.env.CUCUMBER_SLAVE_ID, 10);
}
return port.toString();
}

[egarc12]

Hello everybody

Is there an update on this?

I am using nightwatch api with cucumber and I need to execute some features in parallel, the execution by scenarios does not work for me, I need it to be by features, I tried to use the cucumber-parallel module (https://www.npmjs.com/package/cucumber-parallel) but seems to be outdated with recent versions of cucumber

Any ideas? I greatly appreciate your help

[icloudphil]

Still having issues with this. Using --parallel 2 causes the second process to fail binding.

How does the test runner know the port of the second process?

Are you able to see it started 2 browsers on your local?

[icloudphil]

Hello everybody

Is there an update on this?

I am using nightwatch api with cucumber and I need to execute some features in parallel, the execution by scenarios does not work for me, I need it to be by features, I tried to use the cucumber-parallel module (https://www.npmjs.com/package/cucumber-parallel) but seems to be outdated with recent versions of cucumber

Any ideas? I greatly appreciate your help

Can you provide the example in a GitHub perhaps? You don’t need to use any package to make this work. The cucumber runner already support it. Please check out above comments.

[egarc12]

Can you provide the example in a GitHub perhaps? You don’t need to use any package to make this work. The cucumber runner already support it. Please check out above comments.

Thanks for your response

I am using the following script in my package.json

"test:parallel": "cucumber-js --require-module @babel/register --require cucumber.conf.js --require step-definitions --parallel 2 --format node_modules/cucumber-pretty"

The problem is that this runs the scenarios in parallel and I need to run the features in parallel because some scenarios depend on other scenarios and if I run them in parallel my tests will fail.

[bankimzededa]

@egarc12 did you find any solution to run individual feature in parallel ?

[egarc12]

@egarc12 did you find any solution to run individual feature in parallel ?

No :(