Non-default identifiers?

[henryiii]

Description

Currently, we have added a single opt-in identifier (per platform): the free threaded build requires a custom setting, free-threaded-support. But there is an incoming build that we might also want to make opt-in (#1538, GraalPy), and PyPy3.7-3.9 should also be opt-in. Mayne even past-EoL Pythons could be opt in if requires-python was not found. Pyodide (and iOS/Android in the future) are platforms, so not an issue. Maybe we should come up with a way to specify a non-default builds more generally?

As a quick thought to get design ideas going, we could have an opt-in list:

[tool.cibuildwheel]
opt-in = ["free-threaded", "graalpy", "old-pypy", "old-python"]

CIBW_OPT_IN: "free-threaded;graalpy;old-pypy;old-python"

Build log

No response

CI config

No response

[joerick]

Yep. All good points. I'd like a system such as this too. On the design, here's whats top of mind for me:

• Do we want to reuse the syntax of build identifiers? e.g. CIBW_BUILD_INCLUDE=graalpy* or CIBW_BUILD_INCLUDE=cp3??t-*

  Probably not... it gets pretty confusing when compared with the CIBW_BUILD option - this ends up being just a more powerful CIBW_BUILD

• Do we want to change the default CIBW_BUILD to do this?

  Given the amount of wildcards in existing configurations, it's probably not a good idea, as they'd end up opting-in users unexpectedly (e.g. *-manylinux_x86_64 would select all pythons, whereas before it just meant CPython and PyPy)

• Add more options to the --platform option?

  can't think of a way to do this while keeping the name 'platform', so doesn't feel worth it

I do think it's a shame that we keep adding more build selector options. We already have platform, arch, build/skip, requires-python, free-threaded-support, prerelease-pythons. But, actually, we could also roll prerelease-pythons into this too, which would help clean things up a bit.

So, yeah, I like the design above. On syntax/naming, I'd be inclined to be something like

[tool.cibuildwheel]
enable = ["cpython-free-threaded", "graalpy", "pypy-eol", "cpython-eol", "cpython-prerelease"]

CIBW_ENABLE: "cpython-free-threaded graalpy pypy-eol cpython-eol cpython-prerelease"

[henryiii]

I do think it's a shame that we keep adding more build selector options

This is supposed to help with that, instead of adding a new one for graalpy. :) The others have their own reasons for existing.

Naming seems fine to me.

[henryiii]

Should we make it an error to set enable and free-threaded-support or prerelease-pythons? We can make it a warning to use those old settings, maybe after one release, then we could remove them eventually. Also, should we make setting this from an environment variable additive? That logically seems better, as you don't want to disable an enable, you just want to add a new enable. Also, adding prerelease-pythons means you'd be able to set this from the pyproject.toml now, but I think that's fine for consistently. Though I'd recommend people use the CIBW_ENABLE: "cpython-prerelease" and make it additive with the TOML setting.

[henryiii]

And what about musl? Should that require an enable too?

[mayeut]

And what about musl? Should that require an enable too?

I don't think so, there was really some demand for those wheels (i.e. should be enable by default) and those not wishing to provide them already have the proper skip in place.

Maybe even past-EoL Pythons could be opt in if requires-python was not found.

Sounds reasonable.
Should we warn when it's not found or is this a potential nightmare w.r.t. false positives (cibuildwheel can't resolve it but metadata gets correctly populated with python-requires) ?

Though I'd recommend people use the CIBW_ENABLE: "cpython-prerelease" and make it additive with the TOML setting.

Must have IMHO.
We probably want to recommend not setting this one in pyproject.toml or only if abi3 (useful to set it with abi3 to get the tests to run as early as possible with new python versions) and no free-threading.

What about PyPy itself ? The same rational as musl could be used to say nothing more than managing eol could be done but if going this route with GraalPy, maybe we ought to do the same with PyPy ?

[henryiii]

musl

One other way to argue this would be to note that all the other groups we've proposed are rows on the identifier table, this would be the only column-based one. Okay, sounds good. I think this is very much just trying to get the 'default' build avoid things most projects won't need unless there's a special need.

nightmare w.r.t. false positives

The only case affected is projects trying to build EoL Python version with python requires set via setup.py will suddenly not get those old wheels, and will have to add this to opt back in. I think we could have a warning if python-requires is not set.

What about PyPy itself?

It would help to have a good idea of the long-term plans of PyPy. I've heard conflicting things about it's future. If PyPy is going away, we could put it behind a flag. If it's not, having just the supported PyPy on by default is an improvement, and people already have the skips / builds in place for avoiding it if they don't like it.