Test coverage wrong when importing large library

[MattTheCuber]

Type: Bug

1. Create a simple project that imports MONAI with a test
2. Open the Testing tab and run tests with coverage
3. Module-level lines (e.g. imports, class definitions, and function definitions) will show as missed in the coverage report

Extension version: 2024.16.0
VS Code version: Code 1.94.0 (d78a74bcdfad14d5d3b1b782f87255d802b57511, 2024-10-02T13:08:12.626Z)
OS version: Windows_NT x64 10.0.19045
Modes:
Remote OS version: Linux x64 4.18.0-348.23.1.el8_5.x86_64
Remote OS version: Linux x64 4.18.0-348.23.1.el8_5.x86_64
Remote OS version: Linux x64 4.18.0-348.23.1.el8_5.x86_64
Remote OS version: Linux x64 4.18.0-348.23.1.el8_5.x86_64

System Info

Item	Value
CPUs	AMD Ryzen 7 5825U with Radeon Graphics (16 x 1996)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	undefined
Memory (System)	38.83GB (21.36GB free)
Process Argv	--file-uri file:///c%3A/Users/mvine/Documents/Programming/goliath-fragct.code-workspace
Screen Reader	no
VM	0%

Item	Value
Remote	SSH: 10.10.10.137
OS	Linux x64 4.18.0-348.23.1.el8_5.x86_64
CPUs	AMD Ryzen 9 5950X 16-Core Processor (32 x 3650)
Memory (System)	125.82GB (89.95GB free)
VM	0%

Item	Value
Remote	SSH: 10.10.10.137
OS	Linux x64 4.18.0-348.23.1.el8_5.x86_64
CPUs	AMD Ryzen 9 5950X 16-Core Processor (32 x 3650)
Memory (System)	125.82GB (89.95GB free)
VM	0%

Item	Value
Remote	SSH: 10.10.10.137
OS	Linux x64 4.18.0-348.23.1.el8_5.x86_64
CPUs	AMD Ryzen 9 5950X 16-Core Processor (32 x 3650)
Memory (System)	125.82GB (89.95GB free)
VM	0%

Item	Value
Remote	SSH: 10.10.10.137
OS	Linux x64 4.18.0-348.23.1.el8_5.x86_64
CPUs	AMD Ryzen 9 5950X 16-Core Processor (32 x 3589)
Memory (System)	125.82GB (89.95GB free)
VM	0%

[eleanorjboyd]

thank you for the bug report! Could you include what is shown if you run coverage from the terminal as comparison?

[MattTheCuber]

thank you for the bug report! Could you include what is shown if you run coverage from the terminal as comparison?

Sure thing:

---------- coverage: platform linux, python 3.11.6-final-0 -----------
Name                     Stmts   Miss  Cover   Missing
------------------------------------------------------
app/__init__.py              0      0   100%
app/main.py                  1      0   100%
app/tests/__init__.py        0      0   100%
app/tests/test_main.py       3      0   100%
------------------------------------------------------
TOTAL                        4      0   100%

[MattTheCuber]

@eleanorjboyd any update on this? Anything more I can provide to help debug?

[eleanorjboyd]

What is your root dir when you run it from the terminal? You can see what one is selected for running tests in vscode and so im trying to figure out what setting could be different between what you are putting in the terminal and the extension might be passing to run coverage in the UI.

[MattTheCuber]

My root directory is test_coverage.

I also ran the full command exactly as it shows in the terminal excluding the -p argument: pytest --cov=. --cov-report term-missing --rootdir=/home/mvine/Documents/test_coverage /home/mvine/Documents/test_coverage/app/tests/test_main.py::test and got the same results. My suspicions are that either the plugin is doing something or the --cov=. is not working because it's using a relative path. However, I tested running the previous command from /home/mvine/Documents/test_coverage/app/tests, /home/mvine/Documents/test_coverage/app, /home/mvine/Documents/test_coverage, and /home/mvine/Documents without replicating the problem.

[eleanorjboyd]

I wondering if it has to do with the plugins you have enabled because I am now seeing the error "unable to read testIds from temp fileNo...". Also I see you added --cov-report term-missing to your run in terminal, are you expecting this to be run in the extension because I don't see that in the logs.