Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWSSDK.Extensions.NETCore.Setup 3.7.2 causing tests to be extremely slow #2000

Closed
dannystommen opened this issue Mar 30, 2022 · 13 comments
Closed
Labels
bug This issue is a bug. module/sdk-core needs-investigation p2 This is a standard priority issue perf queued

Comments

@dannystommen
Copy link

dannystommen commented Mar 30, 2022

Description

we're using AWSSDK.Extensions.NETCore.Setup and since updating it to 3.7.2, running tests became extremely slow in our pipeline. On my local (windows) machine, everything still works as expected. In our pipeline, total time of tests increased from 5 seconds to 130 sec.

Find attached the test results

When looking at all tests, multiple tests take 16 seconds suddenly, like something is being locked. Some tests even take 32 sec. Nothing in between. So a lots of tests complete in milliseconds, a few tests take like 16 sec, and 1 or 2 tests take 32 sec.

Environment

  • SDK Version: .NET 6
  • Package Version: AWSSDK.Extensions.NETCore.Setup 3.7.2
  • OS Info: Gitlab runner executing Docker image mcr.microsoft.com/dotnet/sdk:6.0
  • Build Environment dotnet test --configuration Release --filter Category=Unit --test-adapter-path:. --logger:"junit;LogFilePath=..\artifacts\{assembly}-unit-test-result.xml;MethodFormat=Class;FailureBodyFormat=Verbose"

This is a 🐛 bug-report
test-results.zip

@dannystommen dannystommen added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Mar 30, 2022
@ashishdhingra
Copy link
Contributor

Hi @dannystommen,

Good afternoon.

Would it be possible to share a sample code solution containing tests that target 3.7.1 and 3.7.2 respectively, in order for us to investigate the issue?

Thanks,
Ashish

@ashishdhingra ashishdhingra added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed needs-triage This issue or PR still needs to be triaged. labels Apr 1, 2022
@Scorpio59
Copy link

may be link to #1996
we experience the same bug, it work well on local with SSO profile but it take longer with normal profile

@ashishdhingra
Copy link
Contributor

may be link to #1996

we experience the same bug, it work well on local with SSO profile but it take longer with normal profile

@Scorpio59 The other issue doesn't appear to be related since user was getting different execution times due to way credentials were configured.

@ashishdhingra ashishdhingra added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. labels Apr 5, 2022
@dannystommen
Copy link
Author

@ashishdhingra I managed to create a sample. Can I sent you this via a DM/email instead of sharing the sample here? I stripped down one of our production projects, which I can not share here publically.

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Apr 7, 2022
@ashovlin ashovlin added the perf label Apr 8, 2022
@ashishdhingra
Copy link
Contributor

@ashishdhingra I managed to create a sample. Can I sent you this via a DM/email instead of sharing the sample here? I stripped down one of our production projects, which I can not share here publically.

@dannystommen We need to come up with the minimal reproducible sample to investigate the issue (reference https://stackoverflow.com/help/minimal-reproducible-example). May be just a test case having issue!

@ashishdhingra ashishdhingra added the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Apr 8, 2022
@dannystommen
Copy link
Author

@ashishdhingra I managed to make it minimal, find attached the zip file, with 1 single test in it.

I also managed to find the "cause". Removing line 11-15 in the Configuration.cs file makes the test fast again.

test-aws-sdk-extensions-setup.zip

Let me know if this is suffiecient, or need anything als from my end.

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Apr 12, 2022
@normj
Copy link
Member

normj commented Apr 14, 2022

@dannystommen Can you describe your pipeline setup? Is it AWS CodeBuild or something else and how are credentials expected to be retrieved for the service clients?

@dannystommen
Copy link
Author

@dannystommen Can you describe your pipeline setup? Is it AWS CodeBuild or something else and how are credentials expected to be retrieved for the service clients?

It's running in gitlab. If you commit this code to a gitlab repo, it will automatically run the pipeline (gitlab-ci.yml is included).

Not sure what you mean with "how are credentials expected to be retrieved".

@ashishdhingra ashishdhingra added investigating This issue is being investigated and/or work is in progress to resolve the issue. B labels Apr 15, 2022
@dannystommen
Copy link
Author

@ashishdhingra any updates on this?

@kellyrbourg
Copy link

Just ran into this issue on a brand new project. 30s to create a client through options.CreateServiceClient in 3.7.2 vs 125ms in 3.7.0. We do have SSO set up as well but we use a tool like yawsso to sync creds.

@ashishdhingra ashishdhingra added p2 This is a standard priority issue queued and removed B labels Nov 2, 2022
@zippo227
Copy link

I was able to fix it what that solution. Add this to your .json, of course merge it within a current AWS block where applicable without including the additional block.

"AWS": {
"DefaultsMode": "Standard"
},

@dscpinheiro
Copy link
Contributor

I'm going to resolve this as it seems to be the same issue as #2022. I haven't been able to reproduce using the latest version of the extensions package, but we'll take a look if the problem still exists.

FYI: I tried running your test-aws-sdk-extensions-setup.zip example in CodeBuild and did not see longer tests either.

Copy link

github-actions bot commented Oct 3, 2024

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. module/sdk-core needs-investigation p2 This is a standard priority issue perf queued
Projects
None yet
Development

No branches or pull requests

8 participants