In my C project I'm using a source generator to embed some resources into the binary.
I use CMake to build my project and my code works but had some issues. I am pretty sure that what I want to accomplish is possible but I didn't find any answer online.
The current problems I have are:
The generator runs every time, even if the input files did not change. This is not too big of a deal because it is really fast, but I hopped there was a better way to do it
While using
Ninja
the generator runs at every build (as described above) without rebuilding every time. I think that Ninja sees that the file has not changed and does not build it again, but when I make changes in the resources change it still uses the old version. It takes another build to "realize" that the generated file has changed and rebuild itWhile using
Make
the code rebuilds every time, even when the generated file does not change, resulting in wasted build time
In both cases (looking at the output) the generator runs before the compiler.
This situation is not unsustainable but I was wondering if a better solution was possible.
Here's a code snippet from my CMakeLists.txt
add_subdirectory(Generator)
file(GLOB RESOURCES Resources/*)
add_custom_command(
OUTPUT src/Resources/Generated.hpp src/Resources/Generated.cpp
COMMAND Generator ${RESOURCES}
DEPENDS ${RESOURCES}
DEPENDS Generator
)
add_custom_target(Generated DEPENDS src/Resources/Generated.hpp src/Resources/Generated.cpp)
add_dependencies(${PROJECT_NAME} Generated)
Here's a minimal reproducible example, sorry for not having it before.
EDIT: I implemented the functional solution from the correct answer in the fix
branch, on the same repo. It might be useful for future reference :)
CodePudding user response:
This one is interesting, because there are multiple errors and stylistic issues, which partially overlap each other.
First off:
file(GLOB_RECURSE SRC src/*.cpp src/*.hpp)
add_executable(${PROJECT_NAME} ${SRC})
While convenient in the beginning, globbing your sources is not a good idea. At some point you will have a testme.cpp
in there that should not be built with the rest, or a conditionally_compiled.cpp
that should only be compiled if a certain option is set. You end up compiling sources that you really did not intended to.
In this case, the file src/Generated.hpp
from your git repository. That file is supposed to be generated, not checked out from repo. How did it even get in there?
add_custom_command(
OUTPUT src/Generated.hpp
COMMAND ${PROJECT_SOURCE_DIR}/generator.sh
${PROJECT_SOURCE_DIR}/Resources/data.txt
> ${PROJECT_SOURCE_DIR}/src/Generated.hpp
DEPENDS Resources/data.txt
DEPENDS ${PROJECT_SOURCE_DIR}/generator.sh
)
Do you see the output redirection there? You wrote to ${PROJECT_SOURCE_DIR}
. That is not a good idea. Your source tree should never have anything compiled or generated in it. Because these things end up being committed with the rest... like it happened to you.
Next issue:
add_custom_target(Generated DEPENDS src/Generated.hpp)
This creates a make
target Generated
. Try it: make Generated
. You keep getting the following output:
[100%] Generating src/Generated.hpp
[100%] Built target Generated
Obviously it does not realize that Generated.hpp
is already up-to-date. Why not?
Let's look at your custom command again:
add_custom_command(
OUTPUT src/Generated.hpp
COMMAND ${PROJECT_SOURCE_DIR}/generator.sh
${PROJECT_SOURCE_DIR}/Resources/data.txt
> ${PROJECT_SOURCE_DIR}/src/Generated.hpp
DEPENDS Resources/data.txt
DEPENDS ${PROJECT_SOURCE_DIR}/generator.sh
)
What if I told you that your OUTPUT
is never actually generated?
Quoting from CMake docs on add_custom_command
, emphasis mine:
OUTPUT
Specify the output files the command is expected to produce. If an output name is a relative path it will be interpreted relative to the build tree directory corresponding to the current source directory.
So your output claims to be to the binary tree, but your command's redirection is to the source tree... no wonder the generator keeps getting re-run.
Having your header generated in the wrong location does not give you a compiler error, because that header from your source tree gets picked up by your GLOB_RECURSE
. As that one keeps getting re-generated, your executable keeps getting recompiled as well.
Try this from your build directory:
mkdir src && touch src/Generated.hpp && make Generated
Output:
[100%] Built target Generated
You see that make
has nothing to do for the Generated
target, because it now sees an OUTPUT
of your custom command that is newer than its dependencies. Of course, that touch
ed file isn't the generated one; we need to bring it all together.
Solution
Don't write to your source tree.
Since ${PROJECT_BINARY_DIR}/src
does not exist, you need to either create it, or live with the created files on the top dir. I did the latter for simplicity here. I also removed unnecessary ${PROJECT_SOURCE_DIR}
uses.
add_custom_command(
OUTPUT Generated.hpp
COMMAND ${PROJECT_SOURCE_DIR}/generator.sh \
${PROJECT_SOURCE_DIR}/Resources/data.txt \
> Generated.hpp
DEPENDS Resources/data.txt
DEPENDS generator.sh
)
Don't glob, keep control over what actually gets compiled:
add_executable( ${PROJECT_NAME} src/main.cpp )
Add the binary tree to your target's include path. After add_executable
:
target_include_directories( ${PROJECT_NAME} PRIVATE ${PROJECT_BINARY_DIR} )
That's it, things work as expected now.