Mailing-List: contact cygwin-apps-help AT sourceware DOT cygnus DOT com; run by ezmlm Sender: cygwin-apps-owner AT sourceware DOT cygnus DOT com List-Subscribe: List-Archive: List-Post: List-Help: , Delivered-To: mailing list cygwin-apps AT sources DOT redhat DOT com Message-ID: <3BF8B243.6090200@ece.gatech.edu> Date: Mon, 19 Nov 2001 02:18:27 -0500 From: Charles Wilson User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: Robert Collins CC: cygwin-apps AT cygwin DOT com Subject: Re: -src package standard: proposal #5 and #5a References: <3BF8385F DOT 8010606 AT ece DOT gatech DOT edu> <02a301c17084$9b178b50$0200a8c0 AT lifelesswks> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Robert Collins wrote: > > Ok, this sounds reasonable... how about this script though. ==== > #!/bin.sh > echo "unpack the source archive, apply the patch, and then optionally" > echo "build the package for you using the script found in " > echo "/CYGWIN-PATCHES/ to build this package." > echo "Call this script like \"cygbuild packagenameandversion\"" > echo "add --build to cause the package to be built" ... If for simple packages only, fine. It's really no different than I was saying, except that for "simple" packages, setup.exe doesn't do the unpack/patch thing; 'cygbuild pkgnameversion' does it. As before, for simple packages, the "real" work -- configure, build, pack, etc -- is done by /CYGWIN-PATCHES/. Very debian. But this .sh file is functionally unnecessary -- as was mine, for simple packages. I merely put it there to show *that it wasn't necessary*. And because the instant FAQ would be "I tried to build foo from -src, but where's the .sh file?" if ANY other package -- a more complicated package -- contained a real, non-window-dressing .sh file. However, if you intend the above script only for simple packages -- those that are easily built by a simple "unpack,patch,run CYGWIN-PATCHES/buildscript" sequence -- then your use of cygbuild is a mere replacement for the functionality I assumed was to be put into setup.exe. So, no meaningful differences there. However, you still don't address (e.g. agree or disagree or counterpropose) the issue of more complicated packages. Therefore, I must assume that you intend for that script above to be used both for simple AND complex packages, including using "cygbuild" to handle the initial unpacking and patching. If I'm wrong, then the rest of this message is probably of academic interest only. However, I don't think I am. I believe you want a window-dressing .sh file for ALL packages, instructing builders of ALL packages to use 'cygbuild pkgnameversion'. If you intend the above "script" for both simple and complex packages -- that is, tell users to use 'cygbuild pkgnameversion' for complex packages as well -- then there are problems (the same ones I've previously mentioned) -- unless you resort to shar and meta patches. However, EVEN for simple packages, you'll have problems with your "--build" option if the pristine source archive isn't named "correctly" or packed "correctly". These naming problems can't be fixed without either (a) repacking or renaming the pristine source archive, or (b) providing additional metadata somehow, or (c) add a lot more smarts to cygbuild, and keep the original -src package around (because cygbuild will have to parse its content listing "What was the name of the tarball inside "foo-VER-REL-src.tar.bz2? Oh! foosrc.vVER.tar.gz"). You assume that the only meta-data necessary is completely contained in "packagenameandversion". Not so. For instance, "jpeg-6b" comes in the following tarball: "jpegsrc.v6b.tar.gz" (no, I am not kidding) which unpacks into "jpeg-6b/" (but not jpegsrc.v6b/) A human can tell that those two are probably related, but I doubt even a human -- without a_priori knowledge of the naming rule -- will be able to derive the .tar.gz name from the unpacked dirname, or vice versa. A generic computer program (like cygbuild) can't either -- without some other source of metadata. Unless: you're gonna parse the -src tarball 'tar tvf' output to determine the pristine tarball's name, and then parse the pristine tarball 'tar tvf' output to determine the srcdir name...this implies that you keep the -src tarball around, but previously you indicated that the -src tarball was never stored on disk, and would be unpacked on-the-fly by setup.exe Or you can just explicitly include the meta data in the -src tarball! With rpm's, meta data is stored in the spec file. My proposal puts this meta data in an external build script -- if and only if necessary. It really seems like your scheme -- and debian -- are trying to jump thru a HELL of a lot of hoops, in the pursuit of an unnecessary goal: "I want two files and two files only: pristine source and one big patch". The "one big patch" leads you to (a) using shar to generate binary files (b) including meta-patches in your "one big patch" -- the big patch creates secondary patches and shar files... BLEAGH. Sure, I see WHY you want to do this -- so that all port-specific stuff is selfcontained, and can be completely separated from the upstream "pristine" source -- even downloaded separately. There's another reason why you'd want to do this -- but that can wait until I put my ranting hat on. I don't think that goal is worth the hoops and hassles. > > dirname = sed rule on $1 > packagever = sed rule on $1 to remove cygwin version Again, not gonna work with jpeg. > if [test -d $dirname ]; then > mv dirname dirname.old > fi hmm..good thought. I should but this in my version -- but you have to insure that dirname.old doesn't already exist... > tar xjf ${packagever}-src.tar.(bz2|gz) > cd dirname dirname is not reliable... > patch -p1 ../$1.patch > cd .. > if (--build was passed) > ${dirname}/CYGWIN-PATCHES/${1}.sh all > fi > === > > That could be put in /bin by a package - call it cyghelper - and > called something like cygbuild, and is global for all packages. And > setup.exe doesn't need to extract the inner source, it can just look > for cygbuild and if it's there invoke it appropriately (ie without > --build). Actually, with your script as is, you'd even have problems with jpeg just unpacking and patching the source (cd dirname breaks). That's easy to fix (do a patch -p0 from the parentdir instead of patch -p1 from inside srcdir) -- but accesing /CYGWIN-PATCHES/buildscript is not easy to fix without additional smarts or external metadata. Sure -- all of this is *possible* to do with the "one pristine src and one patch only" method -- provided you jump thru enough hoops: shar to create binaries (.jpg files, additional tarballs) "one big patch" creates secondary patches, along with the buildscript cygbuild parses the tarballs to extract dir and filenames rather than assuming that they follow a sane pattern (which jpeg *doesn't*) Or, you drive it all with one external package-specific script that KNOWS about the peculiarities of the package (contains the metadata itself). RPM+package.spec. bash+package.sh. package.spec/package.sh are *custom* to the package, and KNOW about its wackiness if any. As you've probably guessed, I've created a jpeg-6b-5 example at http://www.neuro.gatech.edu/users/cwilson/cygutils/packaging/ for you to take a crack at. (FYI: This will NOT be my real jpeg-6b-5 release; there are other changes I want to make to this package...like autoimport and stuff...before I make a "real" new release) jpeg is a 'fully ported' dllized library -- it doesn't require "after building the static lib, do a make clean, then reconfigure and apply and then build the dll" mumbo-jumbo. ncurses and readline DO -- which makes them harder. But they will be hard for ANY automated build system; the right thing to do, of course, is to complete the port so that ncurses builds both the statlib and dll all in one step, like jpeg here, and not waste time trying to cram the half-ported round peg into the automated build square hole. Completing the ncurses or readline port is a job for (the new) libtool... OTOH, if we had been following that logic -- no "mostly ported" packages allowed -- then we wouldn't have had an ncurses or readline library for the past two years (I'm including the old cygutils days) I'm starting to get a bit pissed off here. I've spent hours writing these emails, trying to point out the shortcomings in the debian-centric proposal. Each of which is answered by "oh yeah, we can work around that using..." Work around?! E.g. the debian scheme isn't powerful enough on its own, so we have to work around... I've spent hours creating examples that demonstrate the various proposals -- I chose a simple package (mktemp) at first because it was easy, but unfortunately it didn't demonstrate some of the difficulties with the debian scheme, although I have pointed out those difficulties in my emails. Then I suggested a compromise where simple packages -- MOST packages -- could be done the debian way, but still leave enough flexibility for more complicated ones. This was countered by cygbuild -- which doesn't exist yet (I hate vapor), and can't derive the requisite metadata without yet MORE complicated hoops (tarball parsing, etc). [And don't get me started on the partial ports like ncurses...] Now I've spent yet more hours creating an example using jpeg, which does exhibit some of the problems I have warned about, and I await Robert's brilliant use of shar and patches-that-both-patch-and-create-other-patches to show me how it should be done. (I'll point out that even if I WAS inclined to use shar, it wasn't an official cygwin tool until Saturday anyway). But anyway, IMNSHO, that's a recipe for maintainer's migraine. A patch that creates a shar file that when unshared creates a tarball that gets unpacked revealing a .ppm, a .jpg, and -- YES -- *another* patch, which can then be applied. (This is the lossless jpeg example). but what if my cygwin patch should be applied AFTER the semi-official lossless jpeg patch? (e.g. I have to patch files created by the ljpeg patch)? Oops, I can't do that, because there's only one "real" patch, and it was already applied. So, my "real" patch, in addition to creating the shar file mentioned previously, must also create a secondary "apply after the ljpeg patch" cygwin-specific patch. Umm...how do I regenerate this automatically? @^!#$* Now we're back to the "two cygwin-specific patches" problem that is what makes the ncurses and readline ports in their current two-step non-libtoolized form difficult. The Debian Way makes jpeg have the same problem -- unnecessarily! I realize, jpeg is a corner case. But that's my point: the one-src+one-patch method isn't powerful enough to handle THIS corner case without heroic, non-maintainer-friendly efforts. But WHY must we jump thru these hoops? Apparently, to avoid the terrible, horrifying possibility that we might have something OTHER than a single pristine source tarball and a one big patch in our -src archives, because debian is God's gift to package distribution, and anything they have done MUST be the optimum. Robert's compromise seems to be "okay, everybody can also put this useless .sh file in there too (even for complex packages [jpeg, ncurses-non-new-libtool]), which will tell you to use cygbuild." Even when cygbuild won't work (with complex packages) without hellacious tarball-parsing ugliness. I really suspect an ulterior motive, here, Robert. When will your port of dpkg be ready? Since you seem intent on forcing this process to specify The Debian Way as the official -src packaging standard for cygwin, I can only assume that the reason is you want to use dpkg eventually for all cgywin-building. I don't think that's going to be very popular with the Red Hat brass, tho. I'm ready to be proven wrong. Please, without repacking or renaming the original jpeg source tarball, show me how simple it really is to do it The Debian Way, with one source tarball and one patch. Show me a generic cygbuild script that can handle it, lossless jpeg patch and all. I'm sure it is possible -- I just really really doubt that it will be simple, or maintainer-friendly. I'm tired of this. I'm tired of spending so much useless time unproductively. I'm tired of being at loggerheads here -- with the exception of an early contribution from Corinna (which I compromised away), and recent contributions from Gareth, this has been largely a two person conversation -- and I'm tired of THAT, too. I had hoped we could reach some sort of sanity by Thanksgiving, and I had planned to spend some of my time next weekend developing new ports. No longer. Unless there is some new blood in this conversation, I'm out of it until after Thanksgiving (although I will *read* with interest Robert's shar-and-metapatch jpeg dist) --chuck