Running pip install twice to see changes ("developer mode") -- second install fails but fi-CodePudding

I am wondering how to use pip to develop a Python package which is going through many revisions rapidly. My work flow is to write C code, compile and install with pip install and test my code.

Then, I would like to change some underlying C code, recompile and reinstall with pip, test the new feature, change something else, go back etc. until my package is ready.

Can someone help me understand why pip install ./cmake_example worked well the first time but when making changes to the code, reinstalling with recompiling produced an error? I just re-ran the command pip install ./cmake_example.

I changed a single line of C code in an innocuous way (adding 1 in the 'add' function just to see if I can change code and recompile) and the code compiled fine in my IDE without pip.

My basic idea was to use pip following this method to avoid having to hackishly insert my shared object into some python directory each time I make a change.

I used the cmake_example from pybind here and followed the steps and did pip install ./cmake_example and it worked very well. I ran the example fine in a Python console.

Then, I changed some code (just added 1 to the adding function), so nothing substantial and wanted to re-install the package.

I then got this error:

Building wheels for collected packages: cmake-example
  Building wheel for cmake-example (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for cmake-example (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [65 lines of output]
      running bdist_wheel
      running build
      running build_ext
      CMake Error at CMakeLists.txt:2 (project):
        Running

         '/tmp/pip-build-env-q_8_pyjk/overlay/bin/ninja' '--version'

        failed with:

         No such file or directory

and

ERROR: Could not build wheels for cmake-example, which is required to
install pyproject.toml-based projects

I have tried pip uninstall cmake_example and then reinstall but to no avail. The upgrade function did not work either. Does pip change something in the project folder?

So many thanks!

PS: The makers of pybind11 also provide a scikit example here where removing the cache for repeated builds is not necessary. The cmake_example seems to be intended for legacy projects that do not use the cmake extension scikit. So if the structure of the cmake_example is essential to the project, removal of the cache is the only way to go. Thank you again for the answer!

CodePudding user response：

I found that deleting the build directory inside the cmake_example directory resolved the problem and pip install ./cmake_example worked again as it did the first time. You can combine the two commands:

rm -rf ./cmake_example/build && pip install ./cmake_example

Looking a little closer, (for me) it was sufficient to delete cmake_example/temp.linux-x86_64-3.8/CMakeCache.txt. I suspect the lines

//Program used to build from build.ninja files.
CMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-yqopewjn/overlay/bin/ninja

inside CMakeCache.txt mean that cmake caches the path to the ninja executable, but the second time around the temporary path is different, so it can no longer be found where cmake expects it.