First of all, a little digression about build systems.
I’d like to clarify that I’m no lover of autotools. But I’m no lover of any other build system, neither, and autotools is used on several open source projects, including most of the ones I participate. It’s easily copied and pasted by new projects. It’s known by distro maintainers how to hack on it to work on different scenarios like cross-compilation, distribute build across machines, different compilers, rpath, etc etc. Moreover, from my experience, project maintainers usually dislike changing the build system because it causes several headaches not related to the raison d’être of the project. So in general I prefer to modernize the build system rather than radically change it to another one.
Enlightenment window manager is about to be released and I was really bored by the amount it was taking to compile. I use icecream to spread the build across the machines on the local network, so I usually do things like “make -j40”. No matter how many jobs I could put there to parallelize the build it was still painfully slow, particularly while compiling the modules. Taking a look in the Makefile.am files, my suspicion was true: it was using recursive makefiles and since each module has a few source files (circa 3 \~ 6), the build was parallelized only among the files in the same directory. This is because the build is serialized to build each directory. There are plenty of links on the web about why projects should use non-recursive automake. I will enlist some:
- http://www.cse.iitb.ac.in/soumen/teach/cs699a1999/make.html
- http://www.flameeyes.eu/autotools-mythbuster/automake/nonrecursive.html
- http://www.mega-nerd.com/erikd/Blog/CodeHacking/nonrecursive_automake.html
- http://www.eyrie.org/~eagle/notes/style/build.html
- http://www.murrayc.com/blog/permalink/2009/07/24/non-recursive-automake-is-the-best-alternative-to-automake/
- http://danielkitta.org/blog/2009/07/30/non-recursive-automake-performance/ (this contains some performance evaluation)
So I decided to convert E17’s automake files to non-recursive ones. At least the modules part. After hours of repetitive tasks for converting it, fixing bugs, building out-of-tree, in-tree, distcheck, etc, I committed it and the build time was improved like below:
[table ID=1 /]
So, after configuring it we can build E17 in roughly 1/4 of the previous time.
After the commit introducing the change there were several others to improve it even more, prettify the output, fix some other bugs. It also got reverted once due to causing problems to other developers, but in the end it was applied back. The worst bug I found was related to subdir-objects option to Automake and Gettext’s Automake macros. That option means that the objects built are kept in the same directory as the correspondent source file. This is needed, particularly in a non-recursive automake scenario, so the objects from different modules don’t conflict due to being put in the same directory. However, letting this option in configure.ac made “make distcheck” fail in some obscure ways and I later tracked it down to be gettext’s fault. A simple “fix” was to remove it from configure.ac and set it in the “AUTOMAKE_OPTIONS” variable of the modules’ Makefile.am. I really hope someone has the time and will to fix gettext macros - they are a horrible mess and I don’t want to play with them.