BerandaComputers and TechnologyRejuvenating Autoconf

Rejuvenating Autoconf

Welcome to LWN.net

The following subscription-only content has been made available to you
by an LWN subscriber. Thousands of subscribers depend on LWN for the
best news from the Linux and free software communities. If you enjoy this
article, please consider accepting the trial offer on the right. Thank you
for visiting LWN.net!

Free trial subscription

Try LWN for free for 1 month: no payment
or credit card required. Activate
your trial subscription now
and see why thousands of
readers subscribe to LWN.net.

October 23, 2020

This article was contributed by Sumana Harihareswara

GNU Autoconf, a
widely used build tool that shines at compatibility with a
variety of Unixes, has accumulated many improvements since its last release
in 2012 — and there are patches awaiting review. While many projects have switched to
other build systems, interest in Autoconf remains. Now, a small team
(disclaimer: including me) is rejuvenating it, working through some
deferred maintenance and code review. A testable
beta is now out
, a new stable release is due in early November, and
interested parties can build on this momentum to further refresh the rest
of the GNU
Build System
(also known as Autotools).

A widely used default

GNU Autoconf is a tool for producing configure scripts
for building, installing and packaging software on POSIX systems. It is a
core component of the
GNU Build System. When a user installs a software package on the command line by compiling
it from source, they are often instructed to run:

    $ ./configure; make; make install

Those steps do the following:

  • configure: test system features with attention to portability
    concerns, prepare and generate appropriate files (including a
    makefile) and directories, etc.
  • make: use the makefile as instructions to build the package,
    performing any necessary compilation steps
  • make install: place the built binary and any other needed
    files into the appropriate location

configure is a portable shell script that must run on many
platforms. Writing a configure script by hand can be tedious and
difficult, so Autoconf helps automate this process. A software developer
writes
a configure.ac file
specifying system features the software
will need (e.g. “is the X Window System installed, and if so,
where?”). Each test for a system feature is a macro written in the GNU M4 language. Autoconf
comes with many macros
that developers will likely need, and a
library of add-on macros (“autoconf-archive”)
(source)
provides dozens more.

Thus, in the base case, a programmer wanting to distribute code to be
built with the GNU
Build System needs to write only a bit of M4 in configure.ac, and
would likely only need to use one or two additional macros from
autoconf-archive. They do need to learn more M4 if they need
configure to detect a system feature for which there is not an
existing macro.

Autoconf has built-in support for various compiled languages: C, C++,
Objective C, Objective C++, Fortran, Erlang, and Go. More crucially, it
performs feature detection with knowledge of a wide variety of POSIX
platforms. If you are building new software that has few arcane
dependencies and your users are all on modern Linuxes plus FreeBSD, or if you
want to make Ninja build
files
, perhaps you’d be better served using alternatives
such as CMake, SCons, or Meson — and indeed many
projects have switched away from the GNU Build System over the years,
including GTK+
and KDE. Complaints that the
GNU Build System is slow,
complex, and hard to use
have been aired (including in LWN’s comment
threads) for years. However, if your customers need to be able to build a
shared library on Solaris, AIX, HP-UX, IRIX, and all the BSDs, then
Autoconf will come in handy.

From 1991 to the present

Autoconf’s founding in 1991 and immediate subsequent history is chronicled
in its manual
and in the
book The GNU Project Build Tools
. Its function in the 1990s and
early 2000s was to smooth over differences among the proliferating Unix
variants. Autoconf’s last big change was the version jump from 2.13
to 2.50
in 2001
, which broke many existing configure scripts and
required several follow-up point releases. Version 2.50 extensively
overhauled several components, including autoupdate
, and
changed cross-compilation defaults; it was such a disruptive release that
some users are still using 2.13 so as not to have to port their old
scripts.

However, in recent years, Autoconf’s star has faded. Linux’s ascendance
has made it easier for developers to get away with ignoring portability
among Unixes — and the GNU Build System’s balky Windows integration doesn’t
help those who need to deliver software to all three major desktop
operating systems. But older, more complex projects include legacy code that already
depends on Autoconf; converting it would be risky and expensive. In
addition,
competing build systems don’t cover all of the edge cases that Autoconf does.

The rise of languages that use their own package management (such
as Python, Perl, Go, and Rust) means that developers writing
single-language code bases can avoid system-level build tools entirely. On
the other hand, if you’re writing software that combines C++, Fortran, Python, Perl,
and Erlang, the GNU Build System can make those multiple languages play
well together. It is more language-independent than, say, setup.py,
and you can use the built-in macros plus the autoconf-archive
macros to say: “I need to be able to use the 2011 dialect of C++, and I
need this particular Python module installed”.

Users of the GNU Build System need stability, multi-language
compilation, and cross-language compatibility, so the incremental
improvements and bug fixes in post-2.50 versions of Autoconf have supported
those goals. Autoconf’s users have lived with version
2.69
since 2012; there have been no stable releases since
then. However, development has not stopped; commits to the Git
repository continued
. Users also submit patches using the
autoconf-patches mailing list
and Savannah; by our
estimation, as of mid-2020, there were hundreds of these patches awaiting
review. (There are fewer now, but we’ll get to that.) Maintainer Eric Blake
had
been aiming to make a release
but hasn’t had time; as he said
in 2016
: “The biggest time sink is digging through the mail archives to
ensure that all posted patches that are still relevant have been applied
“.

Fresh momentum and work in progress

My involvement in Autoconf started when Keith
Bostic emailed the autoconf mailing list in January
, asking: “is there
someone we could pay to do a new release of the autoconf toolset?
” Zack
Weinberg, an Autoconf contributor, forwarded the note to me.

Bostic was interested in Autoconf’s future because one of
his projects used it. He funded Weinberg and me to assess
the work remaining
; as we did that, we talked with Autoconf’s
maintainers (including Blake and Paul Eggert) and they agreed that
we could do further release work.
Then, starting a few months ago,
Bostic — along with Bloomberg and the GNU Toolchain Fund of the
FSF
— has further funded our work so that we can work toward a 2.70
release in early November.

Weinberg released
a testable beta version in July
(even though this is a beta version of
2.70, the beta is labeled 2.69b) and a
second beta, 2.69c
, in September. We are now partway through our goals
for this funded project:

  • Along with other users, we’ve started testing the upcoming release
    against real Autoconf scripts for complex projects, but haven’t yet put it
    through its paces with Emacs, GCC, and CPython.
  • Since Autoconf has no continuous integration (CI) at present, we’re going
    to set up proper CI system to find regressions, at least on Linux, probably at
    sourcehut.
  • We’ve gotten a fraction of the hundreds of disorganized patches and
    bug reports filed, so Autoconf contributors can prioritize and assess our
    backlog; unfortunately, we don’t have enough time to organize even half of
    them.
  • We’ve reviewed several high-priority patches that downstream
    redistributors (such as Arch Linux and the Yocto Project) already carry and
    merged them into the mainline repository.
  • We’ve started working with existing maintainers, contributors, and
    users to get the project on a more sustainable path.

These activities, fortunately, have gathered more momentum with testing
and review help from existing maintainers and contributors, plus new
volunteers. And the new scrutiny and testing have also led to fixes
in related tools, such as the portability library Gnulib.

Speedups, bugfixes, and stricter parsing

The 2.70 release
notes/NEWS file
, which is in progress at the time of this writing,
discusses speedups, several small new features, and many bug fixes that
will be in the new release. The bug fixes alone are an appealing reason
to upgrade. For instance, configure scripts generated by the new
Autoconf will work correctly with the current generation of C and C++
compilers, and their contents no longer depend on anything outside the
source tree (this is important for build reproducibility).

2.70 does, unfortunately, include a few places where backward
compatibility with 2.69 could not be preserved.
In particular, Autoconf is now more strict about M4 quotation syntax (a perennial
headache for Autoconf users) and some macros do not perform as many
checks as they used to, which speeds up the configuration process but
can break configure scripts that assumed that some shell variable was
set without actually calling the macro that sets it. In addition, more
configure scripts now require the helper scripts config.sub,
config.guess, and install-sh. (See the release notes
for the complete list.)

Maintainers of complex Autoconf scripts
will find it well worth their time to test the beta releases and report any
problems encountered to the Autoconf
mailing list
.

Beyond this release: future resilience

In October and early November, Weinberg and I will likely use up the
last of the funding we received. We intend to solicit more funding and
to get more corporate contributors to commit to helping with testing,
code review, and so on. After all, a big open question is: who will
commit to serving as release manager for Autoconf 2.71? It might make
sense to schedule that release for around 12-18 months from now. After
Autoconf 2.50, a steady stream of people reported problems that
contributors fixed in the next several releases. If we have someone
motivated to triage bugs and prioritize and review patches, it may make
sense to do that again, especially since, after 2.70, there will
almost certainly be new bug reports, including for bugs introduced by the
release but not found during beta testing.

There’s also an open question as to who will work to organize the
multiple backlogs of patches and bug reports, so that maintainers can
properly assess, prioritize, and delegate work. Even once we finish the
work that we’ve already received funds to perform, there will still
remain scores of patches languishing in the various mailing lists and/or
in patch sets currently carried by distributions (such as OpenEmbedded
and the BSDs) but not yet merged back into the mainline. Getting all of
those into Savannah, or the
new GNU forge when that shows up
, would help contributors, as would
proper CI on multiple operating systems and environments. Gathering all of
the submitted
patches into one forge will also help downstream distributors
cherry-pick specific fixes to carry in between Autoconf releases.

Autoconf has only been able to revive itself because of the
funding from our sponsors. Conversations in the coming months will
reveal whether and to what extent they and other enterprise users want to
invest to keep Autoconf on a stable footing.
This is certainly not the only piece of old software that free software
depends on as infrastructure and that has significant deferred maintenance
that needs doing; there are closely related projects that could also stand
to be revitalized. Automake is one example;
Libtool could be deprecated,
and have its features refactored into the faster and more integrated
functionality in Automake.

Regardless, in this case, it has been gratifying to help break a
bottleneck so that users of a widely used, even crucial part of the
open-source ecology can benefit from eight years’ worth of improvements —
and
get Autoconf in better shape to make future release cycles better too.

[I would like to thank Zack Weinberg for reviewing this article.]








Did you like this article? Please accept our
trial subscription offer to be
able to see more content like it and to participate in the discussion.


(Log in to post comments)

Read More

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular

Recent Comments