Mailing-List: contact cygwin-announce-help AT cygwin DOT com; run by ezmlm List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-announce-owner AT cygwin DOT com Delivered-To: mailing list cygwin-announce AT cygwin DOT com Delivered-To: moderator for cygwin-announce AT cygwin DOT com Date: Sun, 2 Dec 2001 14:35:37 +0100 From: Corinna Vinschen To: cygann Subject: Restructuring the automake and autoconf packages Message-ID: <20011202143537.A10634@cygbert.vinschen.de> Reply-To: cygwin AT cygwin DOT com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i This message is VERY important. PLEASE read the whole thing -- it will save everyone a lot of time and a lot of FAQs. Sections: What, Why, How, Who, Where, When, and Installation WHAT: ---------------------------------------- (1) automake has been replaced by three separate packages automake-stable automake-devel automake (*) (2) autoconf has also been replaced by three separate packages autoconf-stable autoconf-devel autoconf (*) (*) These are NOT the real autotools. These are wrapper scripts. Since the auto*-stable packages install into /usr/autotool/stable/, and the auto*-devel packages install into /usr/autotool/devel/, the wrappers sit in /usr/bin, and redirect to the appropriate version, based on the AC_PREREQ(version) statement in the target project's configure.[in|ac] WHY: ---------------------------------------- (1) There are incompatibilities between autoconf-2.13 and autoconf-2.5x automake-1.4 and automake-1.5 and most (cygwin-external) projects have been slow and/or resistant to migrate to the newer versions. (2) cygwin should not "push the envelope" and try to force non-cygwin developers and projects to adopt the new autotools (or try to coerce external projects to stay with the old autotools for our sake) [Thus spake cgf] In other words, cygwin's earlier move to autoconf-2.52 and automake-1.5, without providing some sort of backward compatibility, was ill-advised. (3) autoconf/automake/libtool must all share the same prefix (usr or usr/local or...) because they each access each other's m4 macros (underneath /share/). (4) the newest version of libtool (not even in CVS yet) that transparently supports dll-building on cygwin requires the NEW versions of autoconf/automake. If we want a cygwin-friendly libtool, we NEED the new versions. Currently, this libtool is only available here at the following link -- but rest assured, we want to get this into the official distribution soon. http://www.neuro.gatech.edu/users/cwilson/cygutils/robert-collins/contrib/ (The pre"compiled" version at that website won't work with these new auto* packages -- it is a vestige of earlier experiments. However, you should be able to download the "source" version from the same site and build it) (5) Since the new versions of autotools are NOT completely backward compatible with the old versions, we -- the cygwin community -- are left with four choices: a) provide both autoconf-2.13 and -2.52, and both automake-1.4p5 and -1.5 b) immediately fork every package we distribute, fix 'em so that they all work with the new autotools -- AND then convince their upstream maintainers to accept these changes. This contradicts point (2) above. c) do NOT support the new versions at all. Stay with the old versions -- but then we won't get the spiffy new transparent dll-supporting libtool. d) okay, developers: you can't work with the autostuff of packages that require the OTHER autoconf/make. Sorry. (Okay, you could use setup to uninstall/reinstall and switch between the two versions). Or cygwin-developer1 can promise to stay with the old autotools -- and he can be in charge of "stuck-in-the-past" packages, and cygwin-developer2 can track the new tools-- and she can be in charge of "progressive" packages... Option (b) is ruled out by point (2) above. Option (c) will probably work NOW with most packages -- but we'll be left behind when the upstream maintainers eventually DO update to the new versions of autotools (it WILL happen eventally). Plus, we lose the spiffy libtool (see point (3) above). Option (d) is just plain silly. Option (a) -- provide both new and old versions of autotools and somehow convince them to play nicely together -- is the only real choice we (the cygwin community) have. (6) Sociological issues. For MANY years, the autotools remained backward compatible with each new release, until 21 May 2001, the advent of autoconf-2.50. The autoconf developers decided that the benefits of breaking backward compatibility outweighed the harm. Actually, they had gone *as far as they could go* with the 2.13 structure. Since autoconf (and the whole autotools suite) is the backbone -- not counting gcc -- of most cross-platform open source packages, this incompatibility is going to cause a massive ripple effect. We (the open source community) are entering a phase where some packages will AC_PREREQ(2.50) -- but others will delay the transition and will remain at AC_PREREQ(2.13) or earlier. This means that Joe Developer can't update his local autotools to the new version, unless all of the packages he works on are ALSO updated -- but any given project can't update until most of its developers EACH update their own local autotool installation. This gives you a huge web of interlocking dependency, leading to statis: gridlock. The only way to fix this problem -- in an open source community global way -- is for distribution maintainers, such as Red Hat, Debian, (and cygwin), to make it easy and automatic to have BOTH a stable and a development version of the autotools installed. Debian has done some work toward that end, by providing wrapper scripts that autosense the target project's desired version, and then manipulating symbolic links to point to the correct version. However, this is not multi-user safe. The *right* answer here is for the autotools people to take charge -- but they've focused on making it easy to migrate a package from old to new (lots of work in 'autoupdate'), rather than making it easy for two versions to coexist. In other words, they have focused on the technical issues, rather than the sociological ones (and that's not a bad thing). For now (and the future) I think a wrapper-script solution will be necessary -- on all platforms. But let's make things work on cygwin, first -- we have a compelling need, in the new libtool stuff. So, to sum up: Why all this fuss? Because we have a need for both the "old" versions of autotools and the cutting edge versions to coexist. We need the old versions in order to work with existing packages that have not yet taken the plunge and converted to the newer versions. But, we need the newer versions because the SOME packages HAVE taken the plunge, AND the new super-special dll-supporting libtool won't work with the older autotools. HOW: ---------------------------------------- are the packages installed? autoconf-2.13 and automake-1.4-p5 are built and installed into /usr/autotool/stable/[bin,share,...]. These are packaged in cygwin-setup compatible form under the names: autoconf-stable-2.13-X.tar.bz2 autoconf-stable-2.13-X-src.tar.bz2 automake-stable-1.4p5-X.tar.bz2 automake-stable-1.4p5-X-src.tar.bz2 Of course, you also get /usr/doc/autoconf-stable-2.13/* /usr/doc/automake-stable-1.4p5/* /usr/doc/Cygwin/autoconf-stable-2.13.README /usr/doc/Cygwin/automake-stable-1.4p5.README As stated earlier, everything else goes under /usr/autotool/stable/* autoconf-2.52 and automake-1.5 are built and installed into /usr/autotool/devel/[bin,share,...]. These are packaged in cygwin-setup compatible form under the names: autoconf-devel-2.13-X.tar.bz2 autoconf-devel-2.13-X-src.tar.bz2 automake-devel-1.4p5-X.tar.bz2 automake-devel-1.4p5-X-src.tar.bz2 In addition to the "text" documentation, these development packages install their info files and man pages under the "real" /usr/info and /usr/man/ trees, so you needn't set your MANPATH or INFOPATH variables. So, under the main /usr/ tree: /usr/doc/autoconf-devel-2.52/* /usr/doc/automake-devel-1.5/* /usr/doc/Cygwin/autoconf-devel-2.52.README /usr/doc/Cygwin/automake-devel-1.5.README /usr/info/* /usr/man/* As stated earlier, everything else goes under /usr/autotool/devel/* So, we now have both stable and development versions of autotools installed -- but we can't use them, because /usr/autotool/stable/bin and /usr/autotool/devel/bin aren't likely to be in your PATH. That brings us to autoconf-scripts-2.52 and automake-scripts-1.5a, from http://www.neuro.gatech.edu/users/cwilson/auto-wrappers/. These are built and installed under /usr/[bin,doc,...] and are packaged in cygwin-setup compatible form as: autoconf-2.52-X.tar.bz2 autoconf-2.52-X-src.tar.bz2 automake-1.5a-X.tar.bz2 automake-1.5a-X-src.tar.bz2 And again, you also get /usr/doc/autoconf-2.52/* /usr/doc/automake-1.5a/* /usr/doc/Cygwin/autoconf-2.52.README /usr/doc/Cygwin/automake-1.5a.README Of course, the natural question is, "Why is this new 'autoconf-scripts' thing packaged as 'autoconf'? Ditto 'automake-scripts'? Isn't that really confusing?" The answer is "To help setup update everyone's system properly. With the current setup.exe, it seems to be difficult to "cleanly" obsolete a package -- witness the trauma when the 'xpm' package was removed and the 'xpm-nox' package was added. Besides, the /usr/doc/Cygwin/auto*.README files explain everything. " Final note: these packages all passed their own internal tests with flying colors, assuming the new 'mktemp' package is installed. autoconf-2.13 : expected passes 226 unexpected failures 33 (*) autoconf-2.52 : All 155 tests successfull automake-1.4p5 : All 195 tests passed automake-1.5 : All 323 tests passed (*) This is the same performance as the old autoconf-2.13 package from four months ago. HOW: ---------------------------------------- do the packages work? Well, the autotools themselves are just magic. But I can explain how the wrapper scripts work. There is a wrapper script that corresponds to (and is named the same as) every "real" autotools program. The wrapper accepts all of the same options that the real "stable" and "development" versions of the target program accept. The wrapper parses those objects, and will call the target program with the same argument list that it received. Each wrapper script looks in the directory from which it was called for a 'configure.in' or 'configure.ac' script, and parses that file looking for the AC_PREREQ(version) command. If version <= 2.13, then the wrapper will call the stable target. If version > 2.13, or if there is no AC_PREREQ() statement, then the development target is called. The wrappers use the autoconf version to determine which target to call -- even when the wrapper is for an automake program like 'aclocal' or 'automake'. The reason? It's easier that way -- and the compatibility issues between automake-1.4-p5 and automake-1.5 are much less severe than those between autoconf-2.13 and autoconf-2.52. In other words, it matters less if we call the "wrong" target for automake. However, the wrappers don't just exec the appropriate target. They also set the PATH variable and the M4PATH variables and export them. That way subsequent calls to other autotools will go DIRECTLY to the appropriate target, without running the wrappers. For instance, if you run 'autoreconf', then the autoreconf wrapper will determine (say) that you need the development version. So, it sets the PATH to /usr/autotool/devel/bin:${PATH} (and similarly for M4PATH). THEN it calls the "real" (devel) autoreconf. But the "real" (devel) autoreconf itself will call autoconf and aclocal and autoheader...but since the PATH has already been set, it will directly call the "real" (devel) versions, not the wrappers. Finally, there's one or two other little things. First, the scripts do not have '/usr/autotool/devel/' and '/usr/autotool/stable/' hardcoded into them -- although the scripts default to using those prefixes for the stable and development versions. However, you can install your own versions of the "real" autotools into /bob/stable/ and /cindy/devel/, if you like. Just set (and EXPORT!!!) the environment variables "AUTO_STABLE" and "AUTO_DEVEL": $ export AUTO_STABLE=/bob/stable $ export AUTO_DEVEL=/cindy/devel Second, many NON-autotools packages install m4 scripts to assist the autotools. Gettext is one such package: it puts the following files in /usr/share/aclocal/ (NOT into /usr/autotool/devel/share/aclocal nor into /usr/autotool/stable/share/aclocal): /usr/share/aclocal/codeset.m4 /usr/share/aclocal/gettext.m4 /usr/share/aclocal/glibc21.m4 /usr/share/aclocal/iconv.m4 /usr/share/aclocal/isc-posix.m4 /usr/share/aclocal/lcmessage.m4 /usr/share/aclocal/progtest.m4 BUT the autotools will NOT look there without help -- being preoccupied as it were with their own /usr/autotool/[stable|devel]/share/aclocal directory. So, the automake-stable cygwin package and the automake-devel cygwin package have been modified to look not only in /usr/autotool/stable/share/aclocal (for automake-stable) /usr/autotool/devel/share/aclocal (for automake-devel) but they ALSO parse the file /usr/autotool/stable/share/aclocal/dirlist (for automake-stable) /usr/autotool/devel/share/aclocal/dirlist (for automake-devel) for additional directories to search -- BEFORE their own aclocal. In the current releases, dirlist contains only /usr/share/aclocal Thus, 'aclocal' -- which is the only autotool affected by the "added .m4 file" problem -- will use the following search path: (automake-stable): /usr/share/aclocal:/usr/autotool/stable/share/aclocal (automake-devel): /usr/share/aclocal:/usr/autotool/devel/share/aclocal when looking for .m4 files. Lastly, if you want to FORCE the use of the stable or development version of the autotools, nothing could be simpler. Just $ export PATH=/usr/autotool/stable/bin:${PATH} $ export M4PATH=/usr/autotool/stable/share/autoconf for autoconf programs. Or $ export PATH=/usr/autotool/stable/bin:${PATH} $ export M4PATH=/usr/autotool/stable/share/aclocal:\ /usr/share/aclocal:\ /usr/autotool/stable/share/automake:\ /usr/autotool/stable/share/automake/am:\ /usr/autotool/stable/share/autoconf for automake programs. Then just call your autotool. WHO: ---------------------------------------- Charles Wilson wrote the autoconf-scripts (wrapper) package and the automake-scripts (wrapper) package. He also packaged the initial release of the the new autoconf-stable, autoconf-devel, automake-stable, automake-devel, autoconf (wrapper) and automake (wrapper) packages. However; Corinna Vinschen is the maintainer of the autotools packages. But ALL communication regarding these tools on cygwin, and the new wrappers and packaging, should go to cygwin AT cygwin DOT com Thanks also to Earnie Boyd, Chris Faylor, and Robert Collins for helpful ideas and feedback on the list. WHERE: ---------------------------------------- At your friendly neighborhood cygwin mirror. Note that downloads from sources.redhat.com (aka cygwin.com) aren't allowed due to bandwidth limitations. This means that you will need to find a mirror which has this update. In the US, ftp://mirrors.rcn.net/mirrors/sources.redhat.com/cygwin/ is a reliable high bandwidth connection. In Japan, ftp://ftp.u-aizu.ac.jp/pub/gnu/gnu-win32/ is already updated. In DK, http://mirrors.sunsite.dk/cygwin/ is usually up-to-date. If one of the above doesn't have the latest version of this package you can either wait for the site to be updated or find another mirror. WHEN: ---------------------------------------- As soon as the new versions propagate to the mirrors. (okay, this was just an excuse to get all six "reporter's" questions in here: who, what, why, when, where, how) INSTALLATION: ---------------------------------------- To update your installation, click on the "Install Cygwin now" link on the http://cygwin.com/ web page. This downloads setup.exe to your system. Save it and run setup, answer the questions and pick up 'autoconf' from the 'Devel' category. 'autoconf-stable' from the 'Devel' category. 'autoconf-devel' from the 'Devel' category. 'automake' from the 'Devel' category. 'automake-stable' from the 'Devel' category. 'automake-devel' from the 'Devel' category. You may need to click on the "Full" button if the new packages (*-devel, *-stable) do not appear. Final Note: You made it!!!! You read this whole message!!! Congratulations -- now there are just six more messages to go. :-) Actually, there will be six 'pro-forma' announcements, for each of the new and updated packages. But each will just refer back to this message for the real "meat". Thanks, Chuck Wilson Corinna Vinschen *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO *** To unsubscribe to the cygwin-announce mailing list, look at the "List-Unsubscribe: " tag in the email header of this message. Send email to the address specified there. It will be in the format: cygwin-announce-unsubscribe-YOU=YOURDOMAIN DOT COM AT cygwin DOT com