I have a program which has grown in size over the period of a couple of years. As it has grown, the time to compile and link have grown. For a program of roughly 100k lines, it is now taking somewhere around 8 minutes for a complete rebuild and almost 3 minutes to relink if a single file has changed. The catch? Running the same build on Linux takes a mere 3 minutes with a practically non-existent linking time.
Some more information:
- Both compilations are performed on computers with relatively the same specifications (RAM, CPU speed, cores, etc...)
- MinGW must be used in order to maintain low-level compiler compatibility (such things as
__attribute__((packed))are used in order to ensure minimum packet sizes between programs - I am locked into using MinGW-810 or greater 32 and 64-bit Qt 5.15.2 due to several reasons
- Many small static libraries are used as part of the build process (the main code is around 60k lines with the remainder being packaged into many small statically compiled libraries).
- CMake is used for compilation
The problem isn't so much with the initial build, but the relinking time is simply too much. How can I reduce the linking and relinking time to something closer to what I experience on Linux without changing compilers or having to do something crazy like cross compile for Windows on Linux?
CodePudding user response：
Researching this issue led me down a lot of different paths, so I'd like to share my experience with others in the hopes of saving you days and/or weeks of trying at this. Let me start with the solution, and then I'll go back to explain the alternatives that I worked through:
Solution:
The solution is to download the LLVM Compiler Infrastructure and quite literally copy and paste the lld.exe from the installation path directly into your Qt toolchain. For example, if your Qt installation is located here: C:\Qt\ then copy lld.exe into C:\Qt\Tools\mingw810_32\bin (for 32-bit), rename it to ld.exe and rebuild your application. This reduced my relinking time from approximately 2 minutes 45 seconds down to 10 seconds, with Qt MOC.
Notes:
I'm still looking more closely at how Qt searched for the linker. I started renaming all ld.exe files and found that simply renaming the file didn't necessarily stop Qt from linking. That is, to discern which ld.exe precisely to copy over, I would rename the ld.exe in the 32-bit mingw directory to ld.exe.bak. The program, being compiled for 32-bits, would still link successfully. This may indicate that the CMake/Qt toolchains are capable of picking the 64-bit ld.exe, though I'm not yet certain of this.
Other Attempts:
-fuse-ld=lld: I'm not a Windows guy and while it turns out that this linker flag works for MinGW 11, even when adding lld.exe to Window's PATH, I was unable to get this linker to be recognized. My goal here was to alter the linker without having to play directly with the Qt installation.
PIMPL: I went through the entirety of the program, ensuring that every single struct, every single class, was properly forward declared where possible, reducing the number of headers included within headers. Additionally, I ensured that where abstraction could take place, it did. We had been practicing this all along, so the change was about as great as I had expected.
Static Library of Static Libraries: Elsewhere it was suggested that one may reduce linking times by making a large static library which contains other static libraries. This turned out to have no effect whatsoever.
Anti-Virus Scan/Windows Defender Exception: It was postulated in the comment section here that part of the issue could be related to anti-virus scans.
Some other helpful resources:
I hope this helps someone else who might be struggling with relinking times.