Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(DRAFT): reintroduce adder tests #32

Open
wants to merge 36 commits into
base: main
Choose a base branch
from
Open

Conversation

manuel3108
Copy link
Member

@manuel3108 manuel3108 commented Sep 21, 2024

This reintroduces the adder tests previously in svelte-add, now with significant enhancements:

  • "End to End" tests, as they were known, are those we had before. They install all dependencies, run the development server, and perform checks using playwright.
  • "Snapshot" tests run the adders across various project types, collect all the changed files, and create snapshots, but only from the changed files (to be committed to git).

Notes:

  • Snapshots are excluded from this PR for clarity. They should be re-added before or after merging, typically found in packages\adder-tests\_snapshots and community-adder-template\_snapshot, mirroring the structure of the test output directories.
  • The peculiar passing around of vitest functions (refer to: https://github.com/sveltejs/cli/blob/1532153aa16f164bb0d67ef735b2a099e8424bb6/packages/adder-tests/tests/official/end2end.ts) is intentional. The goal is to keep @svelte-cli/adder-testing-library as a standalone package for npm, aiding community adders. Thus, incorporating our test execution within @svelte-cli/adder-testing-library isn't viable. Moreover, vitest doesn't recognize tests within the current package if they're in node_modules.
  • New beforeAll and beforeEach test hooks have been introduced, along with their counterparts for after. This allows, for example, the drizzle adder to manage docker containers independently.

Remaining tasks:

  • Resolve an issue with eslint, which is producing an unclear error message.
  • Address vitest concerns, particularly regarding snapshots. (they probably arise because the snapshot files are not present. They are created locally on the fly, but probably not in CI)

Future enhancements (beyond the scope of this update):

  • Introduce a new project test type called "demo," which applies the project to our demo template via the CLI. This could be beneficial for non-clean projects.
  • Implement sanity tests for community adders: Check if the npm package exists, if the repository exists, and if it's compatible with the latest major version of sv, among other checks.
  • For adders that do not offer custom tests, we should at least apply the adder and execute the build command to ensure it hasn't caused any disruptions.
  • Explore methods to expedite the application of the storybook adder. Currently, applying the storybook adder to five different projects takes approximately 1.5 minutes, accounting for about 25% of the total test time.

@manuel3108 manuel3108 changed the title feat: reintroduce adder tests draft: feat: reintroduce adder tests Sep 21, 2024
Copy link

pkg-pr-new bot commented Sep 21, 2024

Open in Stackblitz

pnpm add https://pkg.pr.new/sveltejs/cli/@svelte-cli/adder-testing-library@32
pnpm add https://pkg.pr.new/sveltejs/cli/@svelte-cli/ast-tooling@32
pnpm add https://pkg.pr.new/sveltejs/cli/sv@32
pnpm add https://pkg.pr.new/sveltejs/cli/@svelte-cli/core@32

commit: a722aea

Comment on lines +86 to +90
// the containers take some time to startup and be ready
// to accept any connections. As there is no standard / easy
// way of doing this for different containers (mysql / postgres)
// we are waiting for them to startup
await new Promise((x) => setTimeout(x, 15000));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a wild one, if you have an idea on how to improve this, please let me know.

We did not had these problems in svelte-add, though this was sheer luck. In svelte-add starting the docker containers was the first thing we did, even before preparing the test environment. During this time the containers had the time to boot up properly.

I do think moving it here overweight's the drawback this brings

@manuel3108
Copy link
Member Author

@AdrianGonz97 Update the PR description, you should now be able to start a review here if you like. And if you have any idea about the linting issue, this would be highly appreciated.

@manuel3108 manuel3108 changed the title draft: feat: reintroduce adder tests feat: reintroduce adder tests Oct 5, 2024
@manuel3108 manuel3108 mentioned this pull request Oct 6, 2024
@manuel3108
Copy link
Member Author

Note the change in #76 may require some additional checks here

@manuel3108 manuel3108 changed the title feat: reintroduce adder tests draft: feat: reintroduce adder tests Oct 11, 2024
@manuel3108
Copy link
Member Author

This will require rather dramatic rebasing some time after summit

@manuel3108 manuel3108 changed the title draft: feat: reintroduce adder tests feat(DRAFT): reintroduce adder tests Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant