Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always run all V2 tests #78

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

Always run all V2 tests #78

wants to merge 12 commits into from

Conversation

yamadapc
Copy link
Contributor

@yamadapc yamadapc commented Sep 17, 2024

This commit modifies the integration testing rig so that we run all V2 tests on the V3 integration suite, even if they are currently marked as failures.

The purpose of doing this is to:

  • Quickly understand what failures are happening from CI build logs
  • Be able to build tooling that automatically tries to diagnose current gaps
  • Identify tests that are not failing but are currently skipped

This diff adds a few other changes outside of test-utils:

  • The rust 'RPC' resolver code, which is meant for plugin support is changed so that it will not panic and crash a thread, since that will otherwise stall the test-suite
  • A few tests that should be passing on V3 are re-enabled
  • I reckon we should force the asset graph to be built always on the V3 implementation for now. So this adds force: true on the runs that are using the rust back-end. This is temporary until we have cache invalidation

This changes the v2/v3 test setup so that tests that are skipped
for V3 run. A subsequent change will improve our logging to
automatically diff the outputs of both versions in a single
process.

The idea here is to easily identify tests that are green but
skipped, as well as get a grasp of current failures.
This allows us to run integration tests that fail without crashing
the process.
Since we do not have cache invalidation properly set-up with the rust
implementation. It's better to force it for now. This should, if our
implementation is correct fix certain tests that depend on cache
invalidation.
Since we do not have cache invalidation properly set-up with the rust
implementation. It's better to force it for now. This should, if our
implementation is correct fix certain tests that depend on cache
invalidation.
@yamadapc yamadapc requested a review from a team September 17, 2024 07:53
@yamadapc yamadapc enabled auto-merge (squash) September 17, 2024 07:57
Copy link
Contributor

@mattcompiles mattcompiles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Haven't checked why CI is failing though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants