feat: use the gateway to fetch repodata

[baszalmstra]

This branch replaces repodata fetching with the repodata gateway from conda/rattler#560 . This yields considerable time and memory savings!

[ruben-arts]

I would be more than glad to merge this feature but there are a few questions I have.

• Can users opt-in to this feature?
• Can users disable it if it gives them issues?
• What is the fallback if the website goes down?
• How can someone request the real latest version if the shards get out of date due to server issues?

[baszalmstra]

Hey @ruben-arts !

This feature is not yet done. The biggest missing piece is proper feedback of repodata fetching. I will add this in the next few days.

There are two features here:

1. The gateway itself which replaces the old way of fetching repodata. This still uses the "old" method underneath.

2. The use of the sharded index. To use the sharded index you currently have to use a special channel: e.g. https://fast.prefix.dev/conda-forge . If and when the CEP gets merged and the sharded index will become more widespread the gateway will have a detection mechanism similar to JLAP. This should similarly then then also be disableable through config.

[ruben-arts]

Great, thanks for the info!