When programming in Python, I frequently use virtual environments (using venv). This always worked fine, but recently some of my existing virtual environments seem to be "broken". When I try to activate and use them, I get the following error:

Failed to import the site module
Traceback (most recent call last):
  File "c:\temp\virtualenv\lib\site.py", line 703, in <module>
    main()
  File "c:\temp\virtualenv\lib\site.py", line 683, in main
    paths_in_sys = addsitepackages(paths_in_sys)
  File "c:\temp\virtualenv\lib\site.py", line 282, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "c:\temp\virtualenv\lib\site.py", line 204, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "c:\temp\virtualenv\lib\site.py", line 173, in addpackage
    exec(line)
  File "<string>", line 1, in <module>
  File "c:\temp\virtualenv\lib\importlib\util.py", line 13, in <module>
    from contextlib import contextmanager
ModuleNotFoundError: No module named 'contextlib'

This is strange, since contextlib is part of the standard Python library, and I don't understand why it cannot find it. Because of this error, I cannot even run python (except for python --version or other options that do not require loading any Python modules) or pip (even pip --version doesn't work).

This has been asked and answered before, like here and here, but neither of the answers explain why this error occurs, and both only provide a work-around, which is to remove and recreate the virtual environment. This is frustrating since I do not always know which packages I had previously installed, and I cannot do pip freeze at this point, so recreating the same virtual environment may take a lot of time.

I want to know why this error occurs, and if there's a solution that does not force me to recreate the virtual environment.

To rule out some other unrelated issues:

• My operating system is Windows 10.
• The virtual environments were created using venv and using Python 3.x (mostly 3.6 or later).
• I did not move (relocate) the virtual environment.
• The virtual environment is on a local directory (not a network drive).

CodePudding user response：

It's hard to tell what's happened for things to break.

If it's only contextlib that's missing for some reason, you can copy it back to the venv's library directory from your Python install directory. The same applies for other files (and in fact, you might be able to simply "reinitialize" a venv in the same directory using python -m venv THATDIRECTORY (or python -m virtualenv if you used virtualenv instead of venv) with the non-virtualenv interpreter to fix it).

Then, pip freeze the virtualenv's contents and recreate it properly.

If all this fails, you can still look at the various metadata files in the venv's library directory to figure out the package versions you have installed so you can get your recreated venv back on track. You should never consider a virtualenv to be eternal. Use e.g. pip-tools to lock your dependencies to a known set you can always reinstall.

CodePudding user response：

I've found the (likely) reason for the error in my case. Apparently, some of my virtual environments were made with virtualenv instead of venv.

The potential "breaking" of virtual environments created with virtualenv is documented:

Created python virtual environments are usually not self-contained. (...) This does mean that if you upgrade your system python your virtual environments might break, so watch out.

venv does not seem to have this problem, or at least it seems easier to fix it. venv generates a file venv.cfg which contains the location of the system interpreter:

home = C:\Python\3.6-64
include-system-site-packages = false
version = 3.6.8

so even if you're system-wide Python installation has changed, you could easily update the location in this file.

Note: it seems possible to "update" from virtualenv to venv by running the following command:

python -m venv <existing_virtual_env_directory>

I'm not sure if this is guaranteed to work. The documentation of venv states:

Changed in version 3.4: In earlier versions, if the target directory already existed, an error was raised, unless the --clear or --upgrade option was provided. Now, if an existing directory is specified, its contents are removed and the directory is processed as if it had been newly created.

However, in my case, no installed modules were removed and the result was a working virtual environment :-)