Python SDK incompatible with old GLIBC

[emilk]

In short

The Rerun Python SDK is incompatible with GLIBC 2.17, and therefore CentOS 7.

I don't know what is our minimum required GLIBC version.

Problem

A user reported trying to use the Rerun Python SDK in a university cluster running CentOS 7, which uses GLIBC 2.17 (released 2012-12-25). The result:

ERROR: Could not find a version that satisfies the requirement rerun-sdk (from versions: none)

The user further reports:

I see that the wheel file on PyPy specifies the platform tag manylinux_2_31 which is incompatible with my GLIBC version (2.17)

Solution

I think the solution is building the wheels on a machine with an old glibc version.

There are some clues here:

• https://kobzol.github.io/rust/ci/2021/05/07/building-rust-binaries-in-ci-that-work-with-older-glibc.html
• Fix CI for manylinux build of python package explosion/tokenizations#7

We should at least try, and we should also figure out what is the exact minimum GLIBC version required by the Rerun Python SDK.

[nchodosh]

Hi,

I'm the user who originally created this issue.

My university cluster was updated to CentOS 8 finally but our GLIBC is still only 2.28. I'd really like to try out your tool so I'm wondering if there's any progress on this?

[jleibs]

@nchodosh when we tried to do this before we ran into some issues with build environments that were old enough to include the older glibc not having access to a recent enough libgtk3.

However, I just found out that the packages we build for conda-forge are built using an older glibc that should be compatible with centos8.

Is there any possibility you could use the Conda package instead of the pypi wheel?

[Parskatt]

Ubuntu 18 is still stuck at glibc 2.28 (2.27?), can we get this on pip? Don't like conda installs.

[ducha-aiki]

My university cluster was updated to CentOS 8 finally but our GLIBC is still only 2.28. I

Exactly my situation as well.

[alexanderswerdlow]

Same here; our cluster is still on CentOS 7.

I'd imagine that most users on very old glibc versions (maybe not Ubuntu 18, but likely CentOS 7) care less about display support, especially given the excellent support for streaming w/re-run, if that makes things any easier!

[jleibs]

We actually made some good progress on this recently. The viewer is no longer linked into the wheel .so. Instead we bundle a separately built rerun executable which we launch via subprocess. This means these deps shouldn't be necessary for building the wheel anymore.

I think this makes it possible to try moving the wheel-building environment to a manylinux container with an older glibc. I'll try taking a look at this again sometime soon.

[abey79]

I noticed that our latest 0.17.0 wheels (linux x86) affectively use 2.29 max. We should consider advertising that version instead of 2.31 as we currently do.

~/Downloads/rerun_sdk-0.17.0-cp38-abi3-manylinux_2_31_x86_64.whl
❯ greadelf -W -s rerun_sdk/rerun_cli/rerun  |  rg -o "@GLIBC_2.[0-9]{2,}[.0-9]*" | sort | uniq
@GLIBC_2.10
@GLIBC_2.12
@GLIBC_2.14
@GLIBC_2.15
@GLIBC_2.17
@GLIBC_2.18
@GLIBC_2.25
@GLIBC_2.27
@GLIBC_2.28
@GLIBC_2.29

~/Downloads/rerun_sdk-0.17.0-cp38-abi3-manylinux_2_31_x86_64.whl
❯ greadelf -W -s rerun_bindings/rerun_bindings.abi3.so |  rg -o "@GLIBC_2.[0-9]{2,}[.0-9]*" | sort | uniq
@GLIBC_2.10
@GLIBC_2.12
@GLIBC_2.14
@GLIBC_2.15
@GLIBC_2.17
@GLIBC_2.18
@GLIBC_2.25
@GLIBC_2.28
@GLIBC_2.29

[teh-cmc]

Related: https://github.com/corsix/polyfill-glibc

[emilk]

We should be compatible with any glibc 2.17+, see:

• https://blog.rust-lang.org/2022/08/01/Increasing-glibc-kernel-requirements.html
• New C++ linker issue in 0.19 #7788
• Add explicit sysroot_linux to ensure glibc>=2.17 since this is needed by rustc cpp-example-vrs#15