Help installing NaMaster on M2 macOS

[irene-left]

Hello everyone. I have been trying to install NaMaster on my M2 Mac for a while now but unfortunately I've had no luck completing the installation.
I have been trying to do it manually so I have cloned the NaMaster repository to my home directory.
I have already installed cfitsio and it is located in the following directory /Users/el/cfitsio
When running python setup.py install , after activating a separate virtual environment (called tensorflow2) I get the following error output:

running install
/Users/el/miniconda3/envs/tensorflow2/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
/Users/el/miniconda3/envs/tensorflow2/lib/python3.10/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running bdist_egg
running egg_info
writing pymaster.egg-info/PKG-INFO
writing dependency_links to pymaster.egg-info/dependency_links.txt
writing top-level names to pymaster.egg-info/top_level.txt
reading manifest file 'pymaster.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'ar-lib'
warning: no files found matching 'compile'
warning: no files found matching 'config.guess'
warning: no files found matching 'config.h.in'
warning: no files found matching 'config.sub'
warning: no files found matching 'configure'
warning: no files found matching 'depcomp'
warning: no files found matching 'install-sh'
warning: no files found matching 'Makefile.in'
warning: no files found matching 'missing'
adding license file 'LICENSE'
writing manifest file 'pymaster.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-11.0-arm64/egg
running install_lib
running build_py
Found automake.
Makefile.am:3: error: Libtool library used but 'LIBTOOL' is undefined
Makefile.am:3: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
Makefile.am:3: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
Makefile.am:3: If 'LT_INIT' is in 'configure.ac', make sure
Makefile.am:3: its definition is in aclocal's search path.
autoreconf: error: automake failed with exit status: 1
Using -march=native. Binary will not be portable.
configure: WARNING: unrecognized options: --enable-shared, --with-pic
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... yes
checking whether make supports the include directive... yes (GNU style)
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in /Users/el/NaMaster/_deps/libsharp2': configure: error: C compiler cannot create executables See config.log' for more details
make: *** No targets specified and no makefile found. Stop.
make: *** No rule to make target `install'. Stop.
cp: directory ../include/libsharp2 does not exist
Darwin
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking build system type... arm-apple-darwin21.6.0
checking host system type... arm-apple-darwin21.6.0
checking how to print strings... printf
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /Library/Developer/CommandLineTools/usr/bin/ld
checking if the linker (/Library/Developer/CommandLineTools/usr/bin/ld) is GNU ld... no
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 786432
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert arm-apple-darwin21.6.0 file names to arm-apple-darwin21.6.0 format... func_convert_file_noop
checking how to convert arm-apple-darwin21.6.0 file names to toolchain format... func_convert_file_noop
checking for /Library/Developer/CommandLineTools/usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @file support... no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for mt... no
checking if : is a manifest tool... no
checking for dsymutil... dsymutil
checking for nmedit... nmedit
checking for lipo... lipo
checking for otool... otool
checking for otool64... no
checking for -single_module linker flag... yes
checking for -exported_symbols_list linker flag... yes
checking for -force_load linker flag... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... yes
checking for gcc option to produce PIC... -fno-common -DPIC
checking if gcc PIC flag -fno-common -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/Library/Developer/CommandLineTools/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin21.6.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking for gcc option to accept ISO C99... none needed
checking for pkg-config... no
checking for CFITSIO... no
checking for ffgnrwll in -lcfitsio... no
configure: error: could not find the cfitsio library
ERROR: failed to configure HEALPix. Check CFITSIO is installed and reachable.
error: Failed to install libchealpix.

I have manually updated the path to my cfitsio installation in the install_libchealpix.sh file but the error seems to occur on line 16, which is before that.
I saw from a previous post that running libtoolize in the NaMaster directory might help but I get the following output:


libtoolize: You should add the contents of the following files to 'aclocal.m4':
libtoolize: '/usr/local/share/aclocal/libtool.m4'
libtoolize: '/usr/local/share/aclocal/ltoptions.m4'
libtoolize: '/usr/local/share/aclocal/ltsugar.m4'
libtoolize: '/usr/local/share/aclocal/ltversion.m4'
libtoolize: '/usr/local/share/aclocal/lt~obsolete.m4'
libtoolize: Consider adding 'AC_CONFIG_MACRO_DIRS([m4])' to configure.ac,
libtoolize: and rerunning libtoolize and aclocal.
libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am.

As I am relatively new to manually building libraries on a macOS, any help will be appreciated.

[Cameron-Van-Eck]

Possibly too late to be helpful (except for others having the same problem): I also fought a long fight to get it installed on my Apple Silicon Mac (with an M1 CPU in my case).

The solution I found in the end was to uninstall all my Python stuff and reinstall it using x86_64 binaries. Since macOS comes with a tool for automatically translating x86 binaries to work on ARM CPUs, things can still run. It's a LOT easier to install things assuming an x86 architecture, since most packages come with pre-compiled binaries. I was able to install NaMaster and all dependencies with x86 binaries, and they run fine on my Macbook.

[defjaf]

I have managed to get NaMaster installed from source on both an M2 and M1 Mac, compiled for arm (x86 should only be used as a last resort!).

• I used homebrew to install gsl, fftw, cfitsio, openblas, libomp, and gcc (the following is for the current version 13 of gcc).

• Python is installed from https://www.python.org/downloads/, but I think this would all work equally with homebrew or even anaconda python. I use a virtual environment, so python3 points to that version of python.

• I used pip to install setuptools, numpy, scipy, healpy.

• Then, the install seems to work with

CC=gcc-13 LDFLAGS=-L/opt/homebrew/lib CPPFLAGS=-I/opt/homebrew/include python3 setup.py install

There are a few warning along the way (e.g., mismatches between versions of libraries and headers) but everything seems to work: both make check and python -m pytest -v pymaster pass (with a couple of "skipped" tests).

---

For what it's worth, I can't seem to get a compilation with clang to work. Looking at config.log, this seems to be at least in part due to using the option -fopenmp rather than -Xclang openmp -lomp which is needed under clang. Is it possible to change this by hand? (Just changing the obvious lines in setup.py doesn't seem to help...) There also might be problems with the compiler flags propagating down to the other installed components.

The following has been superseded by the next comment.
With either compiler, and even with the above environment variables set, if I run various flavours of pip install, I get ModuleNotFoundError: No module named 'numpy'. I suspect that this is related to https://github.com/LSSTDESC/NaMaster/pulls and #138 (but nb. adding install_requires=['scipy', 'numpy', 'healpy'] does not change this behaviour).

We should probably figure out a way to get pip to run successfully for this case, as I understand that using setuptools in this way is being deprecated.

[defjaf]

Following up, I think we can use pip by adding a file pyproject.toml containing the following:

[build-system]
requires = ["setuptools", "numpy"]

and then running

CC=gcc-13 LDFLAGS=-L/opt/homebrew/lib CPPFLAGS=-I/opt/homebrew/include pip install .

(You can also use ... pip wheel . to just build the wheel which can be installed separately, or moved to another machine.)

(See #143 and https://stackoverflow.com/a/54138355/12266)

[brandonshensley]

A note that @defjaf's recipe below was super helpful for me getting up and running. I would add to use homebrew python and not the Mac default. My final install command was

CC=/opt/homebrew/opt/gcc/bin/gcc-13 LDFLAGS=-L/opt/homebrew/lib CPPFLAGS=-I/opt/homebrew/include pip3 install pymaster

I have managed to get NaMaster installed from source on both an M2 and M1 Mac, compiled for arm (x86 should only be used as a last resort!).

• I used homebrew to install gsl, fftw, cfitsio, openblas, libomp, and gcc (the following is for the current version 13 of gcc).
• Python is installed from https://www.python.org/downloads/, but I think this would all work equally with homebrew or even anaconda python. I use a virtual environment, so python3 points to that version of python.
• I used pip to install setuptools, numpy, scipy, healpy.
• Then, the install seems to work with

CC=gcc-13 LDFLAGS=-L/opt/homebrew/lib CPPFLAGS=-I/opt/homebrew/include python3 setup.py install

[ChristopherAJDuncan]

Hi, I'm trying to install PyMaster on an M3 Mac. It's taken a few days so far, so I'm interested to follow the above. I started using macports, but moved to brew, partly as I was struggling to install pyfftw (linking libraries) and was hoping that by getting that in hand this would also fall in to place.).

I followed the above exactly, using CC=gcc-13 LDFLAGS=-L/opt/homebrew/lib CPPFLAGS=-I/opt/homebrew/include python3 setup.py install and I get the following:

running build
running build_py
Darwin
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 404 0 404 0 0 714 0 --:--:-- --:--:-- --:--:-- 713
100 420 0 420 0 0 447 0 --:--:-- --:--:-- --:--:-- 447
100 659 0 659 0 0 555 0 --:--:-- 0:00:01 --:--:-- 555
100 397 100 397 0 0 191 0 0:00:02 0:00:02 --:--:-- 757
100 138 100 138 0 0 53 0 0:00:02 0:00:02 --:--:-- 1662
100 405 100 405 0 0 101 0 0:00:04 0:00:03 0:00:01 968
100 302k 100 302k 0 0 67118 0 0:00:04 0:00:04 --:--:-- 535k
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking build system type... arm-apple-darwin23.3.0
checking host system type... arm-apple-darwin23.3.0
checking how to print strings... printf
checking for style of include used by make... GNU
checking for gcc... gcc-13
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc-13 accepts -g... yes
checking for gcc-13 option to accept ISO C89... none needed
checking dependency style of gcc-13... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc-13... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... no
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 786432
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert arm-apple-darwin23.3.0 file names to arm-apple-darwin23.3.0 format... func_convert_file_noop
checking how to convert arm-apple-darwin23.3.0 file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @file support... no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc-13 object... ok
checking for sysroot... no
checking for mt... no
checking if : is a manifest tool... no
checking for dsymutil... dsymutil
checking for nmedit... nmedit
checking for lipo... lipo
checking for otool... otool
checking for otool64... no
checking for -single_module linker flag... no
checking for -exported_symbols_list linker flag... yes
checking for -force_load linker flag... yes
checking how to run the C preprocessor... gcc-13 -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc-13 supports -fno-rtti -fno-exceptions... no
checking for gcc-13 option to produce PIC... -fno-common -DPIC
checking if gcc-13 PIC flag -fno-common -DPIC works... yes
checking if gcc-13 static flag -static works... no
checking if gcc-13 supports -c -o file.o... yes
checking if gcc-13 supports -c -o file.o... (cached) yes
checking whether the gcc-13 linker (/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin23.3.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking for gcc-13 option to accept ISO C99... none needed
checking for pkg-config... no
checking for CFITSIO... no
checking for ffgnrwll in -lcfitsio... yes
checking fitsio.h usability... yes
checking fitsio.h presence... yes
checking for fitsio.h... yes
checking our pkgconfig libname... chealpix
checking our pkgconfig version... 3.11.4
checking our pkgconfig_libdir... ${libdir}/pkgconfig
expanded our pkgconfig_libdir... /Users/user/Code/libraries/NaMaster/_deps/lib/pkgconfig
checking our pkgconfig_libfile... chealpix.pc
checking our package / suffix... chealpix /
checking our pkgconfig description... chealpix Library
checking our pkgconfig requires...
checking our pkgconfig ext libs... -lchealpix
checking our pkgconfig cppflags...
checking our pkgconfig ldflags...
noninstalled pkgconfig -L /Users/user/Code/libraries/NaMaster/_deps/chealpix-3.11.4/.libs
noninstalled pkgconfig -I /Users/user/Code/libraries/NaMaster/_deps/chealpix-3.11.4/.
configure: creating ./config.status
config.status: creating Makefile
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing chealpix.pc commands
config.status: creating chealpix.pc.in
config.status: creating chealpix.pc
config.status: creating chealpix-uninstalled.pc
config.status: creating chealpix-uninstalled.sh
Successful configure.
CC chealpix.lo
CCLD libchealpix.la
Successful make.
test -z "/Users/user/Code/libraries/NaMaster/_deps/lib" || ./install-sh -c -d "/Users/user/Code/libraries/NaMaster/_deps/lib"
/bin/sh ./libtool --mode=install /usr/bin/install -c libchealpix.la '/Users/user/Code/libraries/NaMaster/_deps/lib'
libtool: install: /usr/bin/install -c .libs/libchealpix.lai /Users/user/Code/libraries/NaMaster/_deps/lib/libchealpix.la
libtool: install: /usr/bin/install -c .libs/libchealpix.a /Users/user/Code/libraries/NaMaster/_deps/lib/libchealpix.a
libtool: install: chmod 644 /Users/user/Code/libraries/NaMaster/_deps/lib/libchealpix.a
libtool: install: ranlib /Users/user/Code/libraries/NaMaster/_deps/lib/libchealpix.a
test -z "/Users/user/Code/libraries/NaMaster/_deps/include" || ./install-sh -c -d "/Users/user/Code/libraries/NaMaster/_deps/include"
/usr/bin/install -c -m 644 chealpix.h '/Users/user/Code/libraries/NaMaster/_deps/include'
test -z "/Users/user/Code/libraries/NaMaster/_deps/lib/pkgconfig" || ./install-sh -c -d "/Users/user/Code/libraries/NaMaster/_deps/lib/pkgconfig"
/usr/bin/install -c -m 644 chealpix.pc '/Users/user/Code/libraries/NaMaster/_deps/lib/pkgconfig'
./scripts/install_libnmt.sh: line 8: autoreconf: command not found
./scripts/install_libnmt.sh: line 10: ./configure: No such file or directory
ERROR: failed to configure libnmt. Check all dependencies are installed
Dependencies:
- GSL
- FFTW
- CFITSIO
- HEALPix
error: Failed to compile C library.

I notice that it states checking for CFITSIO... no, although it does find the fits libraries. If I list the brew installed ports:

== Formulae

ca-certificates gcc healpix libidn2 libunistring open-mpi pmix tree-sitter
cfitsio gettext hwloc libmpc lz4 openblas python@3.11 unbound
cmake gmp isl libnghttp2 mpdecimal openssl@3 readline xz
emacs gnutls jansson libomp mpfr p11-kit sqlite zstd
fftw gsl libevent libtasn1 nettle pcre2 swig

so CFITSIO, GSL, HEALPix and FFTW are present.

This is on branch ducc_migration, on master I get the same error.

Note I'm working within a venv linked to python3.11 sourced from brew. If I try instead to use pip install . I get an error connecting the numpy (see below). This was an error I also got installing pyCCL, and that was only fixed by using setup.py directly. If there is a solution to that problem that anyone knows of, I'm happy to try pip again.

Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [20 lines of output]
Traceback (most recent call last):
File "/Users/Shared/env/py311brew/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in
main()
File "/Users/Shared/env/py311brew/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/Shared/env/py311brew/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/rf/0gffwms124s2pyftslxm1nlc0000gp/T/pip-build-env-2wrcxmuh/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/rf/0gffwms124s2pyftslxm1nlc0000gp/T/pip-build-env-2wrcxmuh/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
self.run_setup()
File "/private/var/folders/rf/0gffwms124s2pyftslxm1nlc0000gp/T/pip-build-env-2wrcxmuh/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 480, in run_setup
super().run_setup(setup_script=setup_script)
File "/private/var/folders/rf/0gffwms124s2pyftslxm1nlc0000gp/T/pip-build-env-2wrcxmuh/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
exec(code, locals())
File "", line 12, in
ModuleNotFoundError: No module named 'numpy'
[end of output]

Any help massively appreciated: I've run out of expertise :)

[damonge]

@ChristopherAJDuncan this line tells me you may be missing autotools

 ./scripts/install_libnmt.sh: line 8: autoreconf: command not found

Can you try installing that (not sure if that's through homebrew on Mac)? Let me know if that doesn't work.

[ChristopherAJDuncan]

@ChristopherAJDuncan this line tells me you may be missing autotools

 ./scripts/install_libnmt.sh: line 8: autoreconf: command not found

Can you try installing that (not sure if that's through homebrew on Mac)? Let me know if that doesn't work.

Thanks as always @damonge. I think autotools are present. Certainly I can run autoreconf , automake etc on command line

[damonge]

ok, strange... Can you run autoreconf -ivf from the source directory of namaster and try installing it again?

[ChristopherAJDuncan]

Hmm, ok some progress. I did brew install autoconf automake libtool to make sure those tools were present, ran autoreconf -ivf and that line had disappeared. But I still got linking errors on master

I switched to ducc_migration and it seemed to install ok. However, if I try to import pymaster, I get:

Warning: cannot import _healpy_pixel_lib module

ImportError Traceback (most recent call last)
Cell In[1], line 1
----> 1 import pymaster

File /Users/Shared/env/py311brew/lib/python3.11/site-packages/pymaster-1.6.1-py3.11-macosx-14-arm64.egg/pymaster/init.py:52
50 from pymaster import nmtlib as lib # noqa
51 import numpy as np # noqa
---> 52 from pymaster.utils import ( # noqa
53 nmt_params,
54 set_sht_calculator,
55 set_n_iter_default,
56 set_tol_pinv_default,
57 get_default_params,
58 NmtMapInfo,
59 NmtAlmInfo,
60 mask_apodization,
61 mask_apodization_flat,
62 synfast_spherical,
63 synfast_flat,
64 moore_penrose_pinvh,
65 map2alm, alm2map,
66 )
67 from pymaster.field import NmtField, NmtFieldFlat # noqa
68 from pymaster.bins import NmtBin, NmtBinFlat # noqa

File /Users/Shared/env/py311brew/lib/python3.11/site-packages/pymaster-1.6.1-py3.11-macosx-14-arm64.egg/pymaster/utils.py:4
2 import numpy as np
3 import os
----> 4 import healpy as hp
7 def _setenv(name, value, keep=False):
8 """Set the named environment variable to the given value. If keep==False
9 (the default), existing values are overwritten. If the value is None, then
10 it's deleted from the environment. If keep==True, then this function does
11 nothing if the variable already has a value."""

File /Users/Shared/env/py311brew/lib/python3.11/site-packages/healpy/init.py:67
26 from .version import version
28 from .pixelfunc import (
29 ma,
30 mask_good,
(...)
64 get_map_size,
65 )
---> 67 from .sphtfunc import (
68 anafast,
69 map2alm_lsq,
70 map2alm,
71 alm2map,
72 Alm,
73 synalm,
74 synfast,
75 smoothing,
76 smoothalm,
77 almxfl,
78 alm2cl,
79 pixwin,
80 alm2map_der1,
81 gauss_beam,
82 bl2beam,
83 beam2bl,
84 check_max_nside,
85 map2alm_spin,
86 alm2map_spin,
87 blm_gauss,
88 resize_alm
89 )
91 from ._query_disc import query_disc, query_strip, query_polygon, boundaries
92 from ._pixelfunc import ringinfo, pix2ring

File /Users/Shared/env/py311brew/lib/python3.11/site-packages/healpy/sphtfunc.py:33
30 DATAURL = "https://healpy.github.io/healpy-data/"
31 DATAURL_MIRROR = "https://github.com/healpy/healpy-data/releases/download/"
---> 33 from . import _healpy_sph_transform_lib as sphtlib
34 from . import _sphtools as _sphtools
35 from . import cookbook as cb

ImportError: dlopen(/Users/Shared/env/py311brew/lib/python3.11/site-packages/healpy/_healpy_sph_transform_lib.cpython-311-darwin.so, 0x0002): Library not loaded: /opt/local/lib/libz.1.dylib
Referenced from: /Users/Shared/env/py311brew/lib/python3.11/site-packages/healpy/_healpy_sph_transform_lib.cpython-311-darwin.so
Reason: tried: '/opt/local/lib/libz.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/local/lib/libz.1.dylib' (no such file), '/opt/local/lib/libz.1.dylib' (no such file)

/opt/local/lib is the directory for macports which I've deprecated, so I wonder If there was an issue due to it not being a clean install. Are there any steps required to ensure a clean reinstall? I deleted everything in _deps for example.

EDIT: it's a healpy issue, I can't import haply itself. I'll try and work on this and get back to you. Thanks

[damonge]

ok, the issue seems to be healpy in this case, right? Have you tried importing healpy and tracking where the issue may come from there? Seems to be related to something called libz...

[ChristopherAJDuncan]

Thanks @damonge, now have a working version (at least import-able, but that's enough for tonight). For reference, I did

brew install autoconf automake libtool
CC=gcc-13 LDFLAGS=-L/opt/homebrew/lib CPPFLAGS=-I/opt/homebrew/include python3 setup.py install

as well as the steps mentioned by @defjaf above. I also completely deleted macports (not sure if that was really necessary but no harm in it), and since I found that I had an error on healpy I reinstalled that using pip uninstall healpy; pip install healpy --no-cache-dir (since the cached version linked to the now-deleted macports.)

This was on ducc_migration, did not check on master. I also ran autoreconf -ivf though not sure if that would have made any difference.

I owe you a beer etc next time I see you :)

[damonge]

OK, great. We merged ducc_migration into master yesterday, so this should work there too.

[gfabbian]

I went through installation of the code today for a brand new Apple M3 Pro today with clang compilers from Xcode and the latest OS version (14.3.1). Nothing else was installed on the laptop (e.g. macports etc).

Here's a summary of my adventure hoping it will be useful to others.

Problem

• I freshly installed python=3.12 from conda-forge.
• I then installed all the relevant major Namaster dependencies cfitsio,gsl,fftw etc.
• Installing Namaster from conda-forge or pip did not work
• Downloading the source from git and running python setup install even setting the environment variables by hand as suggested by @defjaf, @brandonshensley , @ChristopherAJDuncan resulted in errors such as

checking build system type... Invalid configuration `arm64-apple-darwin20.0.0': machine `arm64-apple' not recognized
      configure: error: /bin/sh ./config.sub arm64-apple-darwin20.0.0 failed
      ERROR: failed to configure HEALPix. Check CFITSIO is installed and reachable.
      error: Failed to install libchealpix.

• I tracked down the error to an error thrown by the configure script used to compile the chealpix library on the fly when installing Namaster. The error can be reproduced running `scripts/install_libchealpix.sh' in the Namaster main directory. I have in fact cfitsio already installed and available in my environment.

Solution / workaround

1. I downloaded Healpix 3.82 and installed the C library
2. Symlink (or copied) the static library in the _deps directory in the Namaster main dir.
3. Edited the scripts/install_libnmt.sh script adding the library and include path to my Conda environment and to the _deps directory and run it.
4. The script ran correctly and created a library file _deps/libnmt.a
5. At this point I ran pip install . and concluded the installation.
6. pytest -vv pymaster ran correctly.

I am not sure if all the problems are connected to the clang specific compiler OR by the new architecture OR by the specific python version. Any insight/suggestion appreciated but this was a relatively clean and simple work around.

[1cosmologist]

@gfabbian This workaround solution was very helpful.

My steps:

1. Install cfitsio, gsl, fftw through conda.
2. Installing chealpix directly with conda install doesn't work. As @gfabbian said, chealpix should be compiled as a static library with conda compilers, linked to the cfitsio installed in the conda evironment. (You will need to provide the path to your conda env lib and include.)
3. Create _deps/lib and _deps/include directory inside NaMaster. Copy libchealpix.a to _deps/lib and chealpix.h to _deps/include.
4. Change CFLAGS in scripts/install_libnmt.sh to add include flag and library-linking flag for your conda environment include and lib directories.
5. Run scripts/install_libnmt.sh. This should create libnmt.a inside _deps/lib.
6. Then run pip install . This successfully installed pymaster in my current conda environment.

pytest -vv pymaster runs with 94 passed, 1 skipped and 1 warning.

[defjaf]

Note, perhaps only to myself. To compile the most recent 2.0 version, on my M1 Mac Studio with homebrew gcc-14 and Python.org python 3.12.3, I encountered link errors like

      libtool: link: gcc-14 -fopenmp -O3 -o test/.libs/check_nmt test/check_nmt-nmt_test.o test/check_nmt-nmt_test_utils.o test/check_nmt-nmt_test_fsk.o test/check_nmt-nmt_test_hpex.o test/check_nmt-nmt_test_bins.o test/check_nmt-nmt_test_bins_flat.o test/check_nmt-nmt_test_field_flat.o test/check_nmt-nmt_test_master_flat.o test/check_nmt-nmt_test_covar_flat.o test/check_nmt-nmt_test_mask.o  -L./ /Users/jaffe/Developer/NaMaster/NaMaster-source/.libs/libnmt.dylib -L/opt/homebrew/lib -L/Users/jaffe/Developer/NaMaster/NaMaster-source/_deps/lib ./.libs/libnmt.dylib -lfftw3_omp -lm -lfftw3 -lchealpix -lcfitsio -lgsl -lgslcblas -fopenmp
      ld: warning: alignment (1) of atom '_ctest_nmt_ut_errors' from '/Users/jaffe/Developer/NaMaster/NaMaster-source/test/check_nmt-nmt_test_utils.o' is too small and may result in unaligned pointers
...
      ld: building fixups: pointer not aligned at _ctest_suite_test+0x0 from /Users/jaffe/Developer/NaMaster/NaMaster-source/test/check_nmt-nmt_test.o

This was fixed following some info buried here by setting MACOSX_DEPLOYMENT_TARGET=11:

MACOSX_DEPLOYMENT_TARGET=11 CC=gcc-14 LDFLAGS=-L/opt/homebrew/lib CPPFLAGS=-I/opt/homebrew/include pip3 install .

Is anyone else seeing this and/or have a better solution?

[damonge]

@defjaf and others: have you tried installing this latest version of namaster from conda (it was uploaded yesterday). It'd be really good to know if these issues are gone.

[tanveerkarim]

Another data point from today: initially I was also having issues by trying to install namaster from pip in my M1 Macbook. But using conda got around the issue. I will point out though that by default if do conda install -c conda-forge namaster then it installs the version 1.6. But when I specified the version number as conda install -c conda-forge namaster=2.2.1 then it worked fine.