Gensim install in Python 3.11 fails because of missing longintrepr.h file-CodePudding

Operating System: macOS Monterey 12.6 Chip: Apple M1 Python version: 3.11.1

I try:

pip3 install gensim

The install process starts well but fatally fails towards the end while running 'clang'. The error message is:

clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.11/include/python3.11 -I/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/numpy/core/include -c gensim/models/word2vec_inner.c -o build/temp.macosx-10.9-universal2-cpython-311/gensim/models/word2vec_inner.o
      gensim/models/word2vec_inner.c:217:12: fatal error: 'longintrepr.h' file not found
        #include "longintrepr.h"
                 ^~~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

This issue is raised in a couple of github postings and is attributed to some incompatibility between cython and python 3.11. However, no suggestion is forwarded as to what users should do until cython is updated. I may have misrepresented the details of the discussions on github but I think this is the gist of it.

Can anyone help me in installing gensim in the meantime?

Thanks.

I updated cython and aiohttp. The latter because I had seen a post where the aiohttp install failed for the same reason as mine (missing "longintrepr.h").

No improvement. "pip install gensim" still fails and fails with the same message as copied above.

CodePudding user response：

It seems your issue may be due to the specifics of a fairly-new Python, and lagging library support, on a somewhat new system (a MacOS M1 machine) which has its own somewhat-unique build toolchains.

Unless you absolutely need to use Python 3.11.1, I'd suggest using Gensim within a Python environment with a slightly-older Python interpreter, where the various pacakges you truly need may be a little more settled. For example, on many OS/architecture/Python combinations, a standard pip install will grab precompiled libraries – so build errors of the type you're seeing can't happen.

That your installation is falling back to a local compilation (which hits a problem without an easy off-the-shelf solution) is a hint that something about the full configuration is still somewhat undersupported by one or more of the involved libraries.

If you use the conda 3rd-party system for managing Python virtual environments, it also offers you the ability to explicitly choose which Python version will be used in each environment. That is, you're not stuck with the exact version, and installed libraries, that are default/global on your OS. You could easily try Python 3.10, or Python 3.9, which might work better.

And, keeping your development/project virtual-environment distinct from the system's Python is often considered a "best practice" for other purposes, too. There's no risk that you'll cause damage to the system Python and any tools reliant on it, or face an issue where several of your Python projects need conflicting library versions. (You just use a separate environment for eah project.) And, the exercise of rigorously specifying what needs to be in your project's environment helps keep its prerequisites/dependencies clear for any future relocations/installations elsewhere.

When using the conda tool for this purpose, I usually start with the miniconda version, so that I have explicit control over exactly what packages are installed, and can thus keep each environment minimally-specified for its purposes. (The larger anaconda approach pre-installs tons of popular packages instead.)

CodePudding user response：

Hey you can download the zip file from here

And then unzip the source tar.gz package and install it :

python setup.py install