Optimize reference tests run and report #1402
Conversation
| @@ -75,7 +75,6 @@ jobs: | |||
| timeout-minutes: 180 | |||
| continue-on-error: true | |||
| env: | |||
| REFERENCE_TESTS_PARALLELISM: 20 | |||
There was a problem hiding this comment.
Some tests consume a lot of heap space so it is not feasible to have tests run in parallel as the sum of heap usage of all concurrent tests will exceed the max heap size.
| Collection<Object[]> testParameters = FAILED_MODULE.isEmpty() | ||
| ? generateTestParametersForConfig(TEST_CONFIG_FILE_DIR_PATH) | ||
| : generateTestParametersForConfig(TEST_CONFIG_FILE_DIR_PATH, FAILED_MODULE, FAILED_CONSTRAINT); | ||
| Collection<Object[]> filteredTestParameters = testParameters.stream().filter(params -> (Boolean) params[2]).toList(); |
There was a problem hiding this comment.
This ensures that the test configs that were going to be skipped will be filtered here itself resulting in a cleaner test report without 10s of thousands of skipped tests. I still need to ensure if all test configs are filtered there is atleast one dummy test config returned. This is because junit expects atleast 1 set of parameters for parameterized test.
| Arguments.of(params[0], params[1], params[2]) | ||
| ); | ||
| Collection<Object[]> testParameters = generateTestParametersForConfig(TEST_CONFIG_FILE_DIR_PATH) | ||
| .stream().filter(params -> (Boolean) params[2]).toList(); |
There was a problem hiding this comment.
This ensures that the test configs that were going to be skipped will be filtered here itself resulting in a cleaner test report without 10s of thousands of skipped tests. I still need to ensure if all test configs are filtered there is atleast one dummy test config returned. This is because junit expects atleast 1 set of parameters for parameterized test.
No description provided.