Update breaks global handling of cli packages with the same name #10861
Comments
DanielHabenicht
added
S: needs triage
|
This looks like a bug in papermill to me. In my environment (created by following your instructions), pip correctly identifies the papermill version as 2.3.4: $ pip install papermill ipykernel papermill-nb-runner
Collecting papermill
Downloading papermill-2.3.4-py3-none-any.whl (37 kB)
[...]
Successfully installed [...] papermill-2.3.4 [...]
$ pip show papermill
Name: papermill
Version: 2.3.4
Summary: Parametrize and run Jupyter and nteract Notebooks
Home-page: https://github.com/nteract/papermill
Author: nteract contributors
Author-email: nteract@googlegroups.com
License: BSD
Location: c:\users\uranusjr\documents\play\pip-10861\lib\site-packages
Requires: ansiwrap, click, entrypoints, nbclient, nbformat, pyyaml, requests, tenacity, tqdm
Required-by:
$ papermill --version
1.1.16 from C:\Users\uranusjr\Documents\play\pip-10861\lib\site-packages\papermill\cli.py (3.10.0)This leads me to believe this is a problem in whatever method papermill is using to identify its version, not pip. |
uranusjr
added
resolution: wrong project
|
I don't think so, but I am not that familiar with python and pip. But what you showed is even more of a problem. But both report back as papermill 2.3.4? |
What does this mean? papermill and papermill-nb-runner are different packages; pip 22.0 installs papermill 2.3.4, and after installation it reports the installed papermill version as 2.3.4, which is correct. Since papermill-nb-runner is a different pacakge from pip’s perspective, it having version 1.1.16 has nothing to do with papermill’s version. From pip’s perspective, everything is correct and consistent. If this version scheme is incorrect from papermill’s perspective, papermill needs to implement additional logic for it, not pip. |
|
I would suggest perhaps bringing the issue up to papermill maintainers (or papermill-nb-runner? I am not familiar with the scosystem around this area) first and see how they can provide better insights on the situation. |
Well, both are available via the cli after installation. If I install > papermill --version
1.1.16 from /opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/site-packages/papermill/cli.py (3.10.0)If I do the same with pip version 21.3.1 it returns: > papermill --version
2.3.4 from /opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/site-packages/papermill/cli.py (3.10.0)Which is both to be expected because I installed two packages which have the same cli name and they are conflicting. But changing which of them gets to rule over the other seems inconsistent. |
|
So the problem is you are intentionally installing two packages that overwrite each other, and you want to control which overwrites which? I would say please don’t do that, but if you insist, you can do |
|
Having two packages (papermill and papermill-nb-runner) both trying to install "the same thing" (I'm being vague here because like @uranusjr I don't know much about papermill or the ecosystem around it) is not really supported by anything in the packaging ecosystem (at best, you'll get errors that one package is trying to overwrite something installed by another, at worst you'll get weird interactions like you describe).
There is no guarantee about what order pip installs stuff, and it can change arbitrarily. Unless 2 packages conflict with each other, that will make no difference. That's what I mean by saying that packages installing the same thing isn't supported...
Pip makes assumptions that these packages violate. Those assumptions might not be written down anywhere (I honestly don't know) but they are very fundamental to the packaging ecosystem - at about the same level as "two packages can't have the same name". |
|
Ok, nice of you to take the time to explain. Thanks!
No, I wasn't aware that I still had the old dependency and it only started crashing with pip version 22.0.
Would it be good to disallow packages from using the same cli-name? Or at least issue a warning on install if they overwrite each other? I haven't found an issue regarding that topic. |
I think there’s a general issue floating around somewhere about doing something when anything clashes during installation (including cli files, Python modules, etc.) between two packages. But feel free to open a new issue if you can’t find one, we can always close one of them when we spot duplicates. |
|
Yeah I think I found it: #4625 |
|
Well noticing the hardly made progress of the other issue I opened one just for the CLI issue. |
Description
When installing two competing cli packages (same cli name) the latest version of pip seems to break old functionality.
I am using
papermillfor running python notebooks and happend to still havepapermill-nb-runnerinstalled which is a fork ofpapermilland has not been updated for years.In previous version of pip the new version was used. With the new version pip uses the old version.
Expected behavior
It should either fail early or restore the previous functionality.
pip version
22.0 and above
Python version
3.10.0
OS
Windows (and Github Actions ubuntu-latest)
How to Reproduce
See this repo for the reproduction
https://github.com/DanielHabenicht/pip-bug
python -m pip install pip==22.0.2pip install papermill ipykernel papermill-nb-runnerpapermill --versionFor Version
22.0the output is1.1.16 from /opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/site-packages/papermill/cli.py (3.10.0)For Version
21.3.1the output is2.3.4 from /opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/site-packages/papermill/cli.py (3.10.0)Output
Code of Conduct
The text was updated successfully, but these errors were encountered: