Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pytest code coverage does not show up #24241

Open
loewexy opened this issue Oct 4, 2024 · 18 comments · Fixed by #24249
Open

Pytest code coverage does not show up #24241

loewexy opened this issue Oct 4, 2024 · 18 comments · Fixed by #24249
Assignees
Labels
info-needed Issue requires more information from poster triage-needed Needs assignment to the proper sub-team

Comments

@loewexy
Copy link

loewexy commented Oct 4, 2024

Type: Bug

Behaviour

Clicking on the Run Tests with Coverage button does not have any effect. No coverage data is generated.

As you can see in the output no option enabling coverage is also passed to pytest.

I use VScode with WSL. I tested the same on a native Linux Computer and it worked as expected.

Steps to reproduce:

  1. Setup any simple project with a pytest test.
  2. Click on Run test with Coverage button.

Diagnostic data

Output for Python in the Output panel (ViewOutput, change the drop-down the upper-right of the Output panel to Python)

2024-10-04 10:27:29.384 [info] Default formatter is set to null for workspace /home/user/dev/test
2024-10-04 10:27:30.088 [info] Python interpreter path: ./.venv/bin/python
2024-10-04 10:27:30.533 [info] > ~/.pyenv/bin/pyenv which python
2024-10-04 10:27:30.533 [info] cwd: .
2024-10-04 10:27:32.070 [info] Discover tests for workspace name: test - uri: /home/user/dev/test
2024-10-04 10:27:32.105 [info] > pixi --version
2024-10-04 10:27:32.121 [info] > . ./.venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode-server/extensions/ms-python.python-2024.16.0/python_files/printEnvVariables.py
2024-10-04 10:27:32.121 [info] shell: bash
2024-10-04 10:27:32.159 [info] > ./.venv/bin/python ~/.vscode-server/extensions/ms-python.python-2024.16.0/python_files/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear tests
2024-10-04 10:27:32.159 [info] cwd: .
2024-10-04 10:27:33.744 [info] > ./.venv/bin/python -m pytest --rootdir . --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-7959CmUhSOd99eL3.xml
2024-10-04 10:27:33.744 [info] cwd: .
2024-10-04 10:27:41.632 [info] Starting Pylance language server.

Extension version: 2024.16.0
VS Code version: Code 1.94.0 (d78a74bcdfad14d5d3b1b782f87255d802b57511, 2024-10-02T13:08:12.626Z)
OS version: Windows_NT x64 10.0.19045
Modes:
Remote OS version: Linux x64 5.15.153.1-microsoft-standard-WSL2

  • Python version (& distribution if applicable, e.g. Anaconda): 3.12.2
  • Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Poetry
  • Value of the python.languageServer setting: Default
User Settings


languageServer: "Pylance"

testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true

Installed Extensions
Extension Name Extension Id Version
autoDocstring - Python Docstring Generator njpwerner.autodocstring 0.6.1
autopep8 ms-python.autopep8 2024.0.0
Better Jinja samuelcolvin.jinjahtml 0.20.0
Even Better TOML tamasfe.even-better-toml 0.19.2
Flake8 ms-python.flake8 2023.10.0
Git Blame waderyan.gitblame 11.1.0
Git Graph mhutchie.git-graph 1.30.0
GitLab Workflow GitLab.gitlab-workflow 5.14.0
isort ms-python.isort 2023.10.1
JavaScript Debugger ms-vscode.js-debug 1.94.0
Mypy Type Checker ms-python.mypy-type-checker 2024.0.0
Pylance ms-python.vscode-pylance 2024.9.2
Python ms-python.python 2024.16.0
Python Debugger ms-python.debugpy 2024.10.0
REST Client humao.rest-client 0.25.1
Table Visualizer for JavaScript Profiles ms-vscode.vscode-js-profile-table 1.0.10
YAML redhat.vscode-yaml 1.15.0
System Info
Item Value
CPUs 11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz (8 x 1498)
GPU Status 2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled
webnn: disabled_off
Load (avg) undefined
Memory (System) 15.73GB (4.64GB free)
Process Argv --crash-reporter-id 4f92d2c0-8402-4937-a1b9-ca00fe63248a
Screen Reader no
VM 40%
Item Value
Remote WSL: ubuntu-22-04
OS Linux x64 5.15.153.1-microsoft-standard-WSL2
CPUs 11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz (8 x 0)
Memory (System) 7.63GB (6.02GB free)
VM 0%
A/B Experiments
vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492:30256859
vscod805:30301674
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
962ge761:30959799
pythongtdpath:30769146
welcomedialog:30910333
pythonnoceb:30805159
asynctok:30898717
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
accentitlementst:30995554
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
jg8ic977:31013176
a69g1124:31058053
dvdeprecation:31068756
dwnewjupyter:31046869
2f103344:31071589
impr_priority:31102340
nativerepl2:31139839
refactort:31108082
pythonrstrctxt:31112756
flightc:31134773
wkspc-onlycs-t:31132770
nativeloc2:31134642
wkspc-ranged-t:31151552
cf971741:31144450
autoexpandse:31146404
iacca2:31150323
notype1cf:31151524
cc771715:31146322

@github-actions github-actions bot added the triage-needed Needs assignment to the proper sub-team label Oct 4, 2024
@loewexy
Copy link
Author

loewexy commented Oct 4, 2024

It seems like the extensions does not see that pytest-cov is installed in that case. But I do not know how I can check what the reason for this behaviour is.

Running pytest with the --cov=. option directly on the cli works without problems.

@fbuessen
Copy link

fbuessen commented Oct 4, 2024

I can confirm this behavior (tests run, but no coverage is generated). I'm also seeing the same behavior when running VSCode+Python natively on Windows.
@eleanorjboyd

@falkben
Copy link

falkben commented Oct 4, 2024

Also seeing this behavior on Mac. Using conda for virtual environment

@eleanorjboyd
Copy link
Member

Hi! From your logs it looks like you aren't on the testing rewrite which is required. Could you update to our rewrite and try again? You can do so adding this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"]. Then you can confirm its enabled by finding Experiment 'pythonTestAdapter' is active in your python logs. If you are unable to see the log for the experiment change your log level to trace and it should show up.
To set to trace use theDeveloper: set log level command in the command palette. Thanks

@eleanorjboyd eleanorjboyd self-assigned this Oct 4, 2024
@github-actions github-actions bot added the info-needed Issue requires more information from poster label Oct 4, 2024
@karthiknadig karthiknadig changed the title Pytest code coverage does not work with WSL Pytest code coverage does not show up Oct 4, 2024
@fbuessen
Copy link

fbuessen commented Oct 4, 2024

Hi! From your logs it looks like you aren't on the testing rewrite which is required. Could you update to our rewrite and try again? You can do so adding this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"].

Ah, the pythonTestAdapter experiment fixes the issue. Thanks!

@eleanorjboyd
Copy link
Member

For clarity I have created a PR so the button is only surfaced on the rewrite where coverage is supported: #24249.

@loewexy
Copy link
Author

loewexy commented Oct 4, 2024

I will check tomorrow if this also fixes the issue for me.

Did I miss this information about the required experiment in the Docs? The release notes and also the Documentation do not mention that the setting is required. And why did it work on my Linux machine out of the box?

Hiding the button is one way to handle it, if this is clearly documented I do not see an issue. The other option would be to present the user with a hint how the feature can be enabled. For me that would have been the best options, otherwise I would have opened an issue that the button is missing.

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label Oct 4, 2024
@eleanorjboyd
Copy link
Member

yes we will add a doc update to clarify

cc @cwebster-99

@loewexy
Copy link
Author

loewexy commented Oct 5, 2024

I can confirm that the experiment also fixes the issue for me.

I can now also explain why it worked on my linux machine. The experiment has been active there as part of an A/B test. If I disable the A/B tests for the python extensions it also does not work here. Therefore discussions about the platform have not been based on the platform but rather random on the enabled A/B test.

The issue is therefore resolved for me, but I think for closing this issue it is at least required to add this to the documentation, otherwise other users will also be confused.

@ivan-toriya
Copy link

@eleanorjboyd just curious, is there a timeline when "pythonTestAdapter" will be enabled by default?

@eleanorjboyd
Copy link
Member

Hi @ivan-toriya, yes we want to get it to default (as it also makes it more maintainable) we are waiting until one final issue with named pipe communication is finished as this causes the new experience to break for some users.

@alspadoni
Copy link

alspadoni commented Oct 9, 2024

Even after adding the setting to my json config file I can't see the experiment active in the logs (I changed the log level) nor does the coverage show up. I'm on mac, any ideas?

@eleanorjboyd
Copy link
Member

Hi @alspadoni can you try running the command "Python: clear cache and reload window"? (Sorry about the delay as well)

@eleanorjboyd eleanorjboyd reopened this Oct 29, 2024
@github-actions github-actions bot added the info-needed Issue requires more information from poster label Oct 29, 2024
@FancyBanana
Copy link

I've tried adding the optIn option to my settings but I still don't see the coverage button...
Here's my settings:

{
    "python.testing.pytestArgs": [
        "core"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true,
    "python.experiments.optInto": [
        "pythonTestAdapter",
    ],
}

@eleanorjboyd
Copy link
Member

Can you put the experiment opt into setting in your user settings? Then you can confirm its enabled by finding Experiment 'pythonTestAdapter' is active in your python logs. If you are unable to see the log for the experiment change your log level to trace and it should show up. To set to trace use theDeveloper: set log level command in the command palette. Thanks

@FancyBanana
Copy link

@eleanorjboyd It is present in my user settings also. There is nothing present in logs, images attached below.
A thing that I've forgot to mention is that I connect into a VM through ssh for this project.
Image
Image

@FancyBanana
Copy link

Here is my entire python log history:

2024-10-31 16:30:23.815 [warning] Dir "/var/www/html/backend-project/project_project/.pixi/envs" is not watchable (directory does not exist)
2024-10-31 16:30:23.815 [info] Default formatter is set to null for workspace /var/www/html/backend-project/project_project
2024-10-31 16:30:23.815 [info] Starting Environment refresh
2024-10-31 16:30:23.815 [info] Searching for interpreters in posix paths locator
2024-10-31 16:30:23.815 [info] Searching for pyenv environments
2024-10-31 16:30:23.815 [info] Searching for conda environments
2024-10-31 16:30:23.815 [info] Searching for global virtual environments
2024-10-31 16:30:23.815 [info] Searching for custom virtual environments
2024-10-31 16:30:23.815 [info] pyenv is not installed
2024-10-31 16:30:23.815 [info] Finished searching for pyenv environments: 44 milliseconds
2024-10-31 16:30:23.815 [info] Finished searching for custom virtual envs: 42 milliseconds
2024-10-31 16:30:23.815 [info] > conda info --json
2024-10-31 16:30:23.815 [info] > pixi --version
2024-10-31 16:30:23.815 [info] > hatch env show --json
2024-10-31 16:30:23.815 [info] cwd: .
2024-10-31 16:30:23.815 [info] Finished searching for global virtual envs: 81 milliseconds
2024-10-31 16:30:23.815 [info] > /usr/share/anaconda/bin/conda info --json
2024-10-31 16:30:23.815 [info] Send text to terminal: printenv > "/tmp/tmp-10424bo46Vn0R6lPI.txt"
2024-10-31 16:30:23.815 [info] > pyenv which python
2024-10-31 16:30:23.815 [info] cwd: .
2024-10-31 16:30:23.885 [info] Found: /bin/python --> /bin/python3.9
2024-10-31 16:30:24.013 [info] Found: /bin/python3 --> /bin/python3.9
2024-10-31 16:30:24.019 [info] Found: /bin/python3.9 --> /bin/python3.9
2024-10-31 16:30:24.148 [info] > /usr/local/bin/python3.12 -I ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/interpreterInfo.py
2024-10-31 16:30:24.154 [info] Found: /usr/bin/python --> /usr/bin/python3.9
2024-10-31 16:30:24.188 [info] Found: /usr/bin/python3 --> /usr/bin/python3.9
2024-10-31 16:30:24.198 [info] Found: /usr/bin/python3.9 --> /usr/bin/python3.9
2024-10-31 16:30:24.204 [info] > /var/www/html/backend-project/env/bin/python -I ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/interpreterInfo.py
2024-10-31 16:30:24.216 [info] Found: /usr/local/bin/python3.12 --> /usr/local/bin/python3.12
2024-10-31 16:30:24.223 [info] > /bin/python -I ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/interpreterInfo.py
2024-10-31 16:30:24.232 [info] Finished searching for interpreters in posix paths locator: 1639 milliseconds
2024-10-31 16:30:24.232 [info] Environments refresh paths discovered (event): 1639 milliseconds
2024-10-31 16:30:24.232 [info] Environments refresh paths discovered: 1639 milliseconds
2024-10-31 16:30:24.257 [info] > /usr/bin/python -I ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/interpreterInfo.py
2024-10-31 16:30:24.261 [info] Python interpreter path: /var/www/html/backend-project/env/bin/python
2024-10-31 16:30:24.325 [info] Environments refresh finished (event): 1732 milliseconds
2024-10-31 16:30:24.341 [info] Environment refresh took 1749 milliseconds
2024-10-31 16:30:24.413 [error] Environment /var/www/html/backend-project/env/bin/python is missing from native locator
2024-10-31 16:30:25.056 [info] Starting Pylance language server.
2024-10-31 16:30:38.342 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:30:38.343 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:30:54.555 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:30:59.000 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:31:07.066 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:31:07.448 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:31:11.076 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:31:11.480 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:31:16.639 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:31:17.185 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.612 [debug] Testing: Refreshing all test data
2024-10-31 16:33:04.613 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.613 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.615 [info] Discover tests for workspace name: project_project - uri: /var/www/html/backend-project/project_project
2024-10-31 16:33:04.615 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.616 [debug] could not find a pixi interpreter for the interpreter at /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.665 [debug] Activation Commands received source /var/www/html/backend-project/env/bin/activate for shell bash, resource /var/www/html/backend-project/project_project and interpreter /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.666 [debug] Activating Environment to capture Environment variables, . /var/www/html/backend-project/env/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python /home/sow/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/printEnvVariables.py
2024-10-31 16:33:04.666 [info] > . /var/www/html/backend-project/env/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/printEnvVariables.py
2024-10-31 16:33:04.666 [info] shell: bash
2024-10-31 16:33:04.696 [debug] could not find a pixi interpreter for the interpreter at /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.699 [info] > /var/www/html/backend-project/env/bin/python ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear core
2024-10-31 16:33:04.699 [info] cwd: .
2024-10-31 16:33:05.977 [debug] Testing: Resolving item /var/www/html/backend-project/project_project
2024-10-31 16:33:05.978 [debug] Testing: Resolving item /var/www/html/backend-project/project_project/core
2024-10-31 16:33:05.978 [debug] Testing: Resolving item /var/www/html/backend-project/project_project/core/tests
2024-10-31 16:33:07.232 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:24.883 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:29.635 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:37.702 [debug] Found cached env for /var/www/html/backend-project/env/bin/python

@eleanorjboyd
Copy link
Member

yes so this shows you are not on the rewrite 2024-10-31 16:33:04.699 [info] > /var/www/html/backend-project/env/bin/python ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear core. Can you check your VM settings? Check this for the ordering, maybe some setting is override something else https://code.visualstudio.com/docs/getstarted/settings#_settings-precedence

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
info-needed Issue requires more information from poster triage-needed Needs assignment to the proper sub-team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants