Machine readable output

[zainab-ali]

This issue was copied over from: disneystreaming/weaver-test#716
It was opened by: caenrique

---

Would it be possible to have the tests results written as json? This would make tooling integration much simpler

[zainab-ali]

This comment was copied over from: disneystreaming/weaver-test#716 (comment)
It was written by: Baccata

---

Can you elaborate on the context of the question ? Weaver doesn't come with a CLI, but integrates with Scala build tools (sbt, mill, scala-cli) via the sbt test-interface https://github.com/sbt/test-interface.

How are you intending to run the tests, such that results written in json would be relevant ?

[zainab-ali]

This comment was copied over from: disneystreaming/weaver-test#716 (comment)
It was written by: caenrique

---

Yes, sorry, I'll give more context

I'm using Sbt and I'm trying to integrate test results with Neovim. What this means for me is being able to filter failing tests, go to test location and show the output of the failed test.

To do this I need test result, test name, file, line of the test, output.

I've found a way of doing this to some extent by using an Sbt plugin which extract some of that information using sbt.TestListener which depends on the Sbt test-interface as you mention.

an example of what I'm getting back:

{
  "timestamp": "2023-11-12T21:17:28",
  "status": "SUCCESS",
  "durationWithSetup": 0.015,
  "duration": 0.087,
  "suite": "com.package.SuiteName",
  "test": "this test does something",
  "test-timestamp": "2023-11-12T21:17:38",
  "errorMessage": "",
  "stackTrace": ""
}

Ultimately this data is being extracted from a sbt.testing.Event

There is an important bit of information missing though, which is the test output (e.g. why the expectations failed, diff outputs, etc) which I believe is produced using a logger.

Now, I'm not familiar with the implementation here, so this is me mostly trying to find a good way to implement this. The way I was thinking to work around this is by parsing the test output, but it's a bit tricky and would be much easier if we could change the logger format to be json by using a config value or something like that.

Sorry for the rant and if you think there is a simpler way of achieving this, I would greatly appreciate the feedback :)

[zainab-ali]

This comment was copied over from: disneystreaming/weaver-test#716 (comment)
It was written by: Baccata

---

Sorry for getting back to you only now. Could you provide an example json payload you get for a failed test ?

[zainab-ali]

This comment was copied over from: disneystreaming/weaver-test#716 (comment)
It was written by: kubukoz

---

uninformed question, isn't this sort of thing covered by BSP?