Undefined symbol setHighMemDynamic

[dsp]

When compiling hsthrift lib:thrift-server with GHC 8.8.4 (due to the requirement of base < 4.15), it fails with

Configuring library 'if-glean-hs' for glean-0.1.0.0..
Preprocessing library for thrift-server-0.1.0.0..
Building library for thrift-server-0.1.0.0..
[3 of 3] Compiling Thrift.Server.CppServer ( Thrift/Server/CppServer.hs, /home/dsp/src/glean/dist-newstyle/build/x86_64-linux/ghc-8.8.4/thrift-server-0.1.0.0/build/Thrift/Server/CppServer.o )
Warning: 'hs-source-dirs: glean/if/search/gen-hs2' directory does not exist.
Warning: 'hs-source-dirs: glean/schema/hs' directory does not exist.
Warning: 'hs-source-dirs: glean/schema/thrift/gen-hs2' directory does not exist.
Warning: 'hs-source-dirs: glean/schema/thrift/query/gen-hs2' directory does not exist.
Warning: 'hs-source-dirs: glean/if/search/gen-hs2' directory does not exist.
Warning: 'hs-source-dirs: glean/schema/hs' directory does not exist.
Warning: 'hs-source-dirs: glean/schema/thrift/gen-hs2' directory does not exist.
Warning: 'hs-source-dirs: glean/schema/thrift/query/gen-hs2' directory does not exist.
<command line>: /home/dsp/src/glean/dist-newstyle/build/x86_64-linux/ghc-8.8.4/fb-util-0.1.0.0/build/libHSfb-util-0.1.0.0-inplace-ghc8.8.4.so: undefined symbol: setHighMemDynamic
Preprocessing library 'if-glean-hs' for glean-0.1.0.0..
Building library 'if-glean-hs' for glean-0.1.0.0..
Preprocessing library 'if-glean-cpp' for glean-0.1.0.0..
Building library 'if-glean-cpp' for glean-0.1.0.0..
[1 of 3] Compiling Glean.Types      ( glean/if/gen-hs2/Glean/Types.hs, /home/dsp/src/glean/dist-newstyle/build/x86_64-linux/ghc-8.8.4/glean-0.1.0.0/l/if-glean-hs/build/if-glean-hs/Glean/Types.o )
[2 of 3] Compiling Glean.GleanService.Service ( glean/if/gen-hs2/Glean/GleanService/Service.hs, /home/dsp/src/glean/dist-newstyle/build/x86_64-linux/ghc-8.8.4/glean-0.1.0.0/l/if-glean-hs/build/if-glean-hs/Glean/GleanService/Service.o )
[3 of 3] Compiling Glean.GleanService.Client ( glean/if/gen-hs2/Glean/GleanService/Client.hs, /home/dsp/src/glean/dist-newstyle/build/x86_64-linux/ghc-8.8.4/glean-0.1.0.0/l/if-glean-hs/build/if-glean-hs/Glean/GleanService/Client.o )

Please note:

<command line>: /home/dsp/src/glean/dist-newstyle/build/x86_64-linux/ghc-8.8.4/fb-util-0.1.0.0/build/libHSfb-util-0.1.0.0-inplace-ghc8.8.4.so: undefined symbol: setHighMemDynamic

due to missing setHighMemDynamic, which seems to be defined in GHC RTS https://gitlab.haskell.org/ghc/ghc/-/commit/7a65f9e140906087273ce95f062775f18f6a708d. Looking at gitlab the function seems to first appear in GHC 9.2.X, released a year ago and not available in 8.8.4 RTS.

It seems to me that either hsthrift needs to be bumped to require 9.2.X or setHighMemDynamic needs to be removed. Maybe i am missunderstanding how i am supposed to build this, so please advice.

[dsp]

FYI A similar problem seems to occur with unloadNativeObj.

[simonmar]

This is mysterious - you're right, those symbols aren't in 8.8.4, and yet our CI on github is passing for 8.8.4 (and indeed versions back to 8.4.4). Perhaps there's some lazy symbol resolution going on, that's somehow disabled on your OS?

@alpmestan @donsbot any ideas?

[dsp]

I don't know the details and so i am just guessing here, but i assume this has to do with me building on NixOS where build steps usually required patching the elf records in order to insert the absolute paths of the libraries as NixOS doesn't have dynamic loading (as libs aren't installed into /usr/lib or /usr/local/lib)

On that note I just removed Util.Dll and Util.Executor from the compilation completely as it doesn't seem to be required by Glean which I assume is the main downstream repo for hsthrift. I found a few more instances of symbols not correctly compiled or available and removing those two just fixed it, in case we are looking for a quick & dirty hack.

[simonmar]

Why Util.Executor?

[alpmestan]

Hello,

@alpmestan @donsbot any ideas?

Hard to tell without inspecting the -v3 output of cabal or something along those lines, comparing to what we see in e.g CI's env. It does look like we've been fairly lucky, somehow not entering a codepath that calls the function...? And indeed for some reason there's a stricter linking policy on @dsp's system that triggers the build error.

Also, @dsp - what does your nix env look like, could you share your shell.nix and build process? I can try to reproduce the problem (I'm on NixOS too, but build Glean from the same docker image that CI uses).

[dsp]

@alpmestan here is my hacked together flake file : https://gist.github.com/dsp/05dd9e3402bf4454f10e5ff7684c8b0b . Note that i must build folly and a few other things myself due to the haswell flag required for AVX2 support. Note also that i cannot yet build hsthrift and glean as a package because of the weird ghc8 + cabal3.6 requirement, that i just can't get to work.

The workflow of nix develop + cabal buildshould work (I use it out of the glean directory and hsthrift is a subdirectory).

Note #90 explains why my builds fail. @exi was so kind to help me debug this.

[dsp]

@simonmar Executor fails due to a missing symbol. I see it's actually later required by Glean, so I fixed the missing symbol.

I send a PR that allows me to build Glean on NixOS (see #91)

[dsp]

Closed by #91