Mailing-List: contact cygwin-apps-help AT sourceware DOT cygnus DOT com; run by ezmlm Sender: cygwin-apps-owner AT sourceware DOT cygnus DOT com List-Subscribe: List-Archive: List-Post: List-Help: , Delivered-To: mailing list cygwin-apps AT sources DOT redhat DOT com Message-ID: <3BE4ABAE.1010102@ece.gatech.edu> Date: Sat, 03 Nov 2001 21:45:02 -0500 From: Charles Wilson User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 X-Accept-Language: en-us MIME-Version: 1.0 To: Robert Collins CC: cygwin-apps AT cygwin DOT com Subject: Re: patches to vendor source trees - discussion References: <1004752145 DOT 521 DOT 38 DOT camel AT lifelesswks> <20011102210311 DOT D31918 AT redhat DOT com> <1004753755 DOT 520 DOT 55 DOT camel AT lifelesswks> <20011102212558 DOT F31918 AT redhat DOT com> <1004755024 DOT 520 DOT 61 DOT camel AT lifelesswks> <20011102214324 DOT K31918 AT redhat DOT com> <3BE40C86 DOT 590FC8F4 AT yahoo DOT com> <1004834853 DOT 4708 DOT 52 DOT camel AT lifelesswks> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Robert Collins wrote: >>How about a script? Call it say pristine-src or something like that. >>You would give it a package-version for the parameter. Then the script >>would apply the patch and if successful remove the patch file. You >>could even give it some options to allow for a backup of the current src >>directory first. >> > > Sure. But this is all predicated on the patch being in the source dir. > (And someone needs to write the script). > > Chuck, I haven't heard from you on this bar the initial comments - and > given the number of packages you maintain.... > > Also, please note: I'm not suggesting that anyone has to repackage > existing stuff... only that new packages, and new releases of existing > packages should follow the guidelines we establish. I think anything > else would be madness. You've stated several times that you're looking only at "baby steps" -- but I think baby steps are counterproductive here. Imagine a sequence of 5 baby step changes in the src packaging. There are three (bad) choices: 1) after each baby step, force all maintainers to update their src packages to the new standard. This is, as you say, "madness". 2) Enforce that after each baby step, new packages and updated packages must obey whatever new standard is in place at that time. this will lead to a HUGE hodgepodge of src packages, each obeying different "standards" as the standard evolves over these baby steps. 3) don't force anybody to change. You end up with the worst of (2) coupled with no real "standard" at all. I'm not opposed to doing away with CYGWIN-PATCHES, or putting the patch file as a sibling to the package-src directory, or whatever -- I just think these changes, these baby-steps, will prove counterproductive in the long run. A data point: I wanted to update my 10 dll-ized library packages so that they would accept the -DALL_STATIC flag as opposed to requiring various incantations of -DZLIB_STATIC -DPNG_STATIC .... This took what? six months? The update cycle for dependent packages is VERY long -- if you want to minimize risk. Thus, the likelihood of (2) occuring is very high, IMO. If we're gonna restructure the src packaging, let's define the BULK of it now, and tweak later -- instead of tweak now and radically restructure later, or tweak now, tweak later, and continue tweaking forever. *Now* is when the new super-spiffy setup.exe is coming out. Big changes are in the wind, why not this, too? Of course, I'm not saying enforce immediate switchover of all packages NOW -- just define the new standard, however radical, and then switch each package over when it is updated in the normal course of events. As much as Chris doesn't like it, there ARE advantages to the RPM style. Here's my take (I'll use zlib as my example): /usr/src/cygwin/SRC *unpacked* sources go here. Packages that do not support VPATH or for some reason must be built within their src directory are also compiled here. /usr/src/cygwin/BUILD "normal" packages are compiled here. /usr/src/cygwin/ORIG .tgz's and primary sources (patch files and the like) go here. "ORIG" is not an ideal name, but it's shorter than "PRIMARY_SOURCES"... /usr/src/cygwin/SCRIPTS (think SPECS) /usr/src/cygwin/INST (I've never liked RPM's habit of using /var/tmp// for temporary installs} /usr/src/cygwin/PACKAGES This is where the newly-created .tar.bz2's and -src.tar.bz2's end up. /usr/src/cygwin/DOCS cygwin-specific READMEs and stuff. So, zlib-1.1.3-9-src.tar.bz2 contains: /usr/src/cygwin/ORIG/zlib/zlib-113.tgz (pristine tarball, in whatever format the primary developers provide it. No tarball renaming. /etc/setup/zlib-src.list.gz will take care of removing it at the proper time. BTW, *DOES* setup.exe worry about files installed from -src packages?) /usr/src/cygwin/ORIG/zlib/zlib-1.1.3-9.patch contains diffs and additional files to make zlib build properly. Does NOT create the cgywin-specific README (that goes in DOCS). Does NOT create the zlib-postinstall script. /usr/src/cygwin/ORIG/zlib/zlib-1.1.3-9.postinstall /usr/src/cygwin/ORIG/zlib/* other patches and semi-standard source files (like the lossless jpg patch) /usr/src/cygwin/DOCS/zlib/zlib-1.1.3.README Goes here, not in the .patch or inside the tarball. Why? because these cygwin-specific READMEs are not usually intended for submission back to the upstream maintainers. So, keep 'em out of the .patch; makes it easier for the cgywin maintainer to organize the patches for upstream submission... Also, this README no longer contains build instructions, (except "run /usr/src/cygwin/SCRIPTS/zlib-build. Here are the allowed options:") /usr/src/cygwin/SCRIPTS/zlib-build poor man's rpm+spec. A shell script that unpacks the source (into SRC), applies the appropriate patches, creates a build dir under BUILD, and creates an inst dir under INST. Configures, builds, installs (into INST//usr/* -- this includes copying ORIG/zlib*postinstall into INST//etc/postinstall/zlib.sh), and creates packages. Note that THIS file doesn't go into a "SCRIPTS/zlib" subdir -- while anything installed or created in the other CAPITAL directories does go into a zlib subdir. The reason? There will never be more than ONE build script for any given package. And that's (pretty much) it. Why do I want zlib-build to handle so much? Why do I want so much stuff in ORIG, and NOT created within the unpacked src dir by .patch? It's very difficult to add a file to a package, AND munge the Makefile's to install it. Okay, it's not *technically* difficult -- for non-autotool packages you just *do* it. For autotool packages, you have to add the new file to the Makefile.am, then re-run automake and autoconf -- which leads to a LOT of changes in a LOT of the files == BIG patch. But in both cases, these changes are NOT ultimately destined for upstream inclusion. They are cygwin-only, and will never make it into the "real" package. IMO, .patch should be limited to code changes that are upstream-compatible. Or at least to those changes that aren't frufru. Heck, in most of cgf's packages, the README (if it exists) is just hand-copied into his inst dir immediately before tar'ring -- and is often forgotten (binutils? ) Adding a file to a package The Right Way(tm) is just plain HARD. Why enforce extra work? (BTW, I now officially hate the "put patch file into CYGWIN-PATCHES" idea -- even though it was mine, originally. I'm always forgetting to remove it before creating the new patch, or forgetting to put it back in before creating the src tarball. Ugh.) Using a build script (zlib-build) allows us to have multiple patches, applied in the proper order. We don't need to "revert" to pristine source, 'cause that's what we're distributing. AND, this sort of structure is easily translated to dpkg or rpm style tools, if we ever go there. (Plus, scripts -- once they are properly written -- don't make mistakes, or forget to include the patch in the src tarball, etc) My proposal also minimizes the hurdle that new porters must overcome -- IF we (I?) provide a "sample" build script. Any other plan requires porters to do a LOT of work. For instance, I don't know how **I** would get readline or ncurses to build with a single patch and one "configure; make; make install" run. Currently, I do *two* make; make install runs -- and in the readline case, there's a secondary patch applied before the second (DLL-building) make; make install. Worse, I don't see this changing until libtool supports the new dll-building/auto-import stuff in binutils (or whatever gcc profile-based alternative Robert comes up with) The point? Simpler src packaging standards don't allow for these complex builds -- and in my experience, ports of new packages seem to always go thru a stage where building is complicated. Then later, as the port is refined (*) the build becomes simpler. (*)more stuff accepted by upstream maintainers, better autotool integration -- not to mention ld improvements (past) and libtool improvements (future). Do we want a simple src packaging standard -- that serves as a disincentive for new package ports, because it doesn't allow for complex builds? "No, I'm sorry -- you can't submit that package until you can get it to build using this src packaging standard." "But I can't -- I need to wait for this change to autoconf and that change to binutils -- THEN I can get it to build 'simply'" "Sorry, you must table your package. Go work on autoconf and binutils, then come back" too much delayed gratification. If I had had to WAIT until libtool was working nicely so that "./configure --build-shared --build-static" worked for ncurses on cygwin, we'd still be waiting. Closer to home: if we go with an extrememly simple src packaging standard, I won't be able to update readline for a LONG time (unless I get a "waiver") -- because I won't be able to support the current ugly build procedure of readline under a new -src standard if it isn't flexible/powerful enough. (Oh, yeah -- and then there's the multiple packages from a single source problem as in ncurses/libncurses: Can't be done without a well-written build script. In the simpler src packaging standards, where would IT go? Right now it's in CYGWIN-PATCHES, but only for lack of a better place) The danger is in making the "complex" -src standard too difficult. Well, since my proposal is "put these files in those places, and use a build script to do all the work" and "oh BTW, here's a sample build script, adjust to suit your project" I think it isn't too difficult. And it is quite similar to the structure used by rpm (dpkg) so it should facilitate moving to that at some point in the future, if we decide to do so. Then the answer to "How do I build cgywin from CVS" becomes cd /usr/src/cygwin/SRC cvs co winsup mv src cygwin-cvs cp cygwin-cvs/winsup/maint/cygwin-cvs-build /usr/src/cygwin/SCRIPTS/ cd /usr/src/cygwin/SCRIPTS ./cygwin-cvs-build doitall and poof! /usr/src/cygwin/PACKAGES/cgywin-.tar.bz2 and cygwin--src.tar.bz2. And cygwin--src.tar.bz2, naturally, follows the -src standard, and contains its own cygwin-build script (slightly different from the cgywin-cvs-build script, so this requires two similar build scripts in winsup/maint. Think about it, you'll realize why) --Chuck