Need help in understanding the output stats.

[amit-saini-tx]

Hey,

I'm running artillery test with playwright Javascript and finding it difficult to understand the output. Can anyone please help here?

Question: Lets say I have a step for clicking on a Submit button and in output I'm getting min, max and mean times for the same like in below sample. So, my question is, the min/max time is the response time after that click or min/max time it took to click on that button(ignoring whatever time its took to response)? how should I read the output.

browser.step.Step 2: click_submit_button:
min: ......................................................................... 168
max: ......................................................................... 1232
mean: ........................................................................ 675
median: ...................................................................... 478.3
p95: ......................................................................... 1130.2
p99: ......................................................................... 1130.2

Please let me know if more clarification is required.

[bernardobridge]

Hello @amit-saini-tx 👋

You seem to be using the Step feature we provide. Those metrics are not calculating "response time". They represent the time it took to execute all the logic inside that step. For example, let's say you had this step:

  await step('search_record', async () => {
    await page.getByPlaceholder('enter id').fill(`${recordid}`);
    await page.getByRole('button', { name: 'Go' }).click();
    await page.locator('css=button.avatar-button').click();
  });

Artillery would record the time it takes to enter the id and click the two buttons in the example, and save it under the browser.step.search_record metric.

You should think of these as frontend-facing metrics that you model to represent user actions that you're interested in finding out how long they take.

[amit-saini-tx]

Thanks for the quick response @bernardobridge, it clarified my doubt. Would you mind answering a few more questions to better understand the report?

1. I can see report also capture some network codes but every time the count is different as shown in below sample of 3 runs. Could you please explain a little bit about this behavior? and also if you see in the second run code 304 is not captured.

`browser.page.codes.200: ........................................................ 134
browser.page.codes.204: ........................................................ 7
browser.page.codes.302: ........................................................ 7
browser.page.codes.304: ........................................................ 1
																				 
browser.page.codes.200: ........................................................ 111
browser.page.codes.204: ........................................................ 8
browser.page.codes.302: ........................................................ 7
													
browser.page.codes.200: ........................................................ 129
browser.page.codes.204: ........................................................ 8
browser.page.codes.302: ........................................................ 7
browser.page.codes.304: ........................................................ 1`

2. Is that possible to assert the min and max time in order to fail the script if the time is out of expected range so that we can focus on the failed runs only to debug.

[hassy]

The response codes are reported for every included asset in each web page by the browser (CSS, images, JS etc). The difference could be explained by many different factors, e.g. on some runs some of the assets may be cached so a 304 response is returned.

Assertions may be set with the ensure plugin: https://www.artillery.io/docs/reference/extensions/ensure