1

I just upgraded to version 1.0.1 of Karate from 0.9.6. Earlier all karate cucumber format .json reports were generated in a single folder: target/surefire-reports.

After the upgrade I can see that for each test runner which uses Runner.path builder to run a test feature file with outputCucumberJson(true) is creating the json file in folder's that have a number suffixed. eg, I have folders in target like:

karate_reports
karate_reports_162988888
karate_reports_162988777

Because of this the jenkins cucumber plugin that was working previously does not find the .json files. I have tried clearing the field JSON Reports Path in jenkins cucumber plugin so it scans the whole directory for json files but it does not work.

Any ideas what is going on? Can we still get all json files created in single folder when tests run via maven?

Also I tried setting the reportDir in the Runner.path builder but I still get multiple folders with suffixed numbers.

Help appreciated.

rochitsen
  • 402
  • 6
  • 18

1 Answers1

0

Yes there is a configuration setting called: backupReportDir() which you can set to false.

And next time, please try the RC releases earlier so that you can avoid these kinds of surprises and also alert us if we un-intentionally make breaking changes.

Finally, we do not recommend multiple test-runners in CI, you should have only one, and within that you can "select" different folder hierarchies or individual feature files to compose into a "suite".

There are good reasons for this:

  • the start and end time will be calculated correctly
  • the report aggregation of tags will work correctly
  • parallel execution of your entire suite will be optimized

Yes some of the above points assume you use the Karate HTML reports instead of the CI, JUnit or 3rd-party Cucumber reports. But this is the best practice, and the Karate reports are what we will focus on in the long term.

Peter Thomas
  • 54,465
  • 21
  • 84
  • 248
  • 1
    thanks @peter-thomas that worked. The reason we have separate runners is because some tests require their own test data to be populated and the Junit runners do that. Might not be ideal, but been working fine for us. – rochitsen Aug 18 '21 at 04:02
  • @peter-thomas: we have a case were we don't really have a choice of having multiple runners: a big set of features can be run in parallel, and so we give a thread count very high, but another set of features have to be run 1 after the other, so they are started in a different test with only 1 single thread. Both are run one after the other, because we couldn't have 1 test where the number of threads can be tweaked per set of tags... – Jiehong Aug 24 '23 at 15:10
  • @Jiehong Karate is open-source. perhaps you can contribute code to make this happen, not many teams seem to need it. – Peter Thomas Aug 25 '23 at 03:23