Tight Version Constraints on thinc in spaCy 3.8.2 Causing Dependency Resolution Issues for AutoGluon Users

[tonyhoo]

How to reproduce the behaviour

When attempting to install autogluon==1.1.1, the installation fails due to dependency resolution issues involving spaCy<4.0 and its tight version constraints on thinc. This leads pip to select incompatible versions of packages like onnx, resulting in installation failures.

Steps to reproduce:

1. Create a new virtual environment:

   python -m venv test_env
   source test_env/bin/activate  # On Windows, use: test_env\Scripts\activate

2. Upgrade pip to the latest version:

   pip install --upgrade pip

3. Attempt to install autogluon==1.1.1:

   pip install autogluon==1.1.1

4. Observe the installation failure, particularly with the onnx package.

Error Message:

Collecting onnx
  Using cached onnx-1.10.0.tar.gz (10.0 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [11 lines of output]
      /tmp/pip-install-xxxxxx/onnx/setup.py:36: DeprecationWarning: Use shutil.which instead of find_executable
        CMAKE = find_executable('cmake3') or find_executable('cmake')
      /tmp/pip-install-xxxxxx/onnx/setup.py:37: DeprecationWarning: Use shutil.which instead of find_executable
        MAKE = find_executable('make')
      fatal: not a git repository (or any of the parent directories): .git
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-xxxxxx/onnx/setup.py", line 318, in <module>
          raise FileNotFoundError("Unable to find " + requirements_file)
      FileNotFoundError: Unable to find requirements.txt
      [end of output]

Additional Information:

• The issue started occurring after the release of spaCy==3.8.2 on 2024-10-01.
• Downgrading spaCy to version 3.7.5 resolves the installation issue.
• The tight version constraints on thinc specified by spaCy<4.0 (e.g., thinc<8.4.0,>=8.3.0) appear to be causing dependency resolution conflicts.
• This affects users who are installing packages that depend on spaCy<4.0, such as autogluon.

Explanation:

• spaCy<4.0 specifies strict version constraints on thinc, requiring versions between >=8.3.0 and <8.4.0.
• This tight constraint causes pip to spend a long time resolving dependencies and ultimately selects older, incompatible versions of other packages (like onnx), which fail to install.
• Relaxing the version constraints on thinc could allow pip to find a compatible set of package versions, preventing the installation failure.

Your Environment

• Operating System: Linux (Ubuntu 20.04)
• Python Version Used: 3.10
• spaCy Version Used: spaCy==3.8.2
• Environment Information:
  • pip Version: 24.2
  • Other Relevant Packages:
    • autogluon==1.1.1
    • thinc versions being considered: 8.3.0, 8.3.1, 8.3.2

---

Our Request:

We kindly request the spaCy team to consider adjusting the version constraints on thinc in spaCy<4.0 to be less restrictive, or to release a patch version that relaxes these constraints. This adjustment could help prevent dependency resolution issues for users installing packages that depend on spaCy<4.0.

Is it feasible to modify these constraints without impacting spaCy's functionality? Any guidance or assistance on this matter would be greatly appreciated.

Thank you for your time and support.

---

Note: If this issue is better suited for a discussion, please let me know, and I'll be happy to open a discussion thread instead.

[tonyhoo]

kind reminder

[Goldziher]

Also happening in 3.12.