Mailing-List: contact cygwin-apps-help AT sourceware DOT cygnus DOT com; run by ezmlm Sender: cygwin-apps-owner AT sourceware DOT cygnus DOT com List-Subscribe: List-Archive: List-Post: List-Help: , Delivered-To: mailing list cygwin-apps AT sources DOT redhat DOT com Message-ID: <012301c170f2$90ae1990$0200a8c0@lifelesswks> From: "Robert Collins" To: "Charles Wilson" Cc: References: <3BF8385F DOT 8010606 AT ece DOT gatech DOT edu> <02a301c17084$9b178b50$0200a8c0 AT lifelesswks> <3BF8B243 DOT 6090200 AT ece DOT gatech DOT edu> Subject: Re: -src package standard: proposal #5 and #5a Date: Mon, 19 Nov 2001 23:06:04 +1100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 X-OriginalArrivalTime: 19 Nov 2001 12:06:03.0681 (UTC) FILETIME=[8F9B7110:01C170F2] ----- Original Message ----- From: "Charles Wilson" To: "Robert Collins" Cc: Sent: Monday, November 19, 2001 6:18 PM Subject: Re: -src package standard: proposal #5 and #5a > Robert Collins wrote: > > > > > Ok, this sounds reasonable... how about this script though. ==== > > #!/bin.sh > > echo "unpack the source archive, apply the patch, and then optionally" > > echo "build the package for you using the script found in " > > echo "/CYGWIN-PATCHES/ to build this package." > > echo "Call this script like \"cygbuild packagenameandversion\"" > > echo "add --build to cause the package to be built" ... > > If for simple packages only, fine. It's really no different than I was > saying, except that for "simple" packages, setup.exe doesn't do the > unpack/patch thing; 'cygbuild pkgnameversion' does it. As before, for > simple packages, the "real" work -- configure, build, pack, etc -- is done > by /CYGWIN-PATCHES/. I wasn't trying to be different to what you proposed, just to take something that appeared *identical* across all pacakges - including multi patch packages (which one am I to look at BTW) - and allow it to be reused rather than included each time. > Very debian. I guess :]. Seriously though, I do like the way debian manages this, and thus am drawing inspiration from there (as well as BSD ports/rpm - just debian seems to take the limelight a lot :}). ... > packages. I merely put it there to show *that it wasn't necessary*. And Ah. Ok then. Lets get rid of it then.. > However, if you intend the above script only for simple packages -- those Nope, I intended it to be able to kick off any package. > that are easily built by a simple "unpack,patch,run > CYGWIN-PATCHES/buildscript" sequence -- then your use of cygbuild is a mere > replacement for the functionality I assumed was to be put into setup.exe. > So, no meaningful differences there. Agreed. > However, you still don't address (e.g. agree or disagree or counterpropose) I thought I did, via the 'I'll fiddle a sample complex package as proof-of concept' offer. > If I'm wrong, then the rest of this message is probably of academic > interest only. However, I don't think I am. I believe you want a > window-dressing .sh file for ALL packages, instructing builders of ALL > packages to use 'cygbuild pkgnameversion'. I don't really want such a script - I was negotiating based on your input - I did not realise you were trying to prove a negative with the script. All I want is * No unneed directories to interfere with the users desires. * Completely relocatable source & build process contained in a single tree - once extracted and the initial patch applied. * The ability to ship the vendors source (maybe repackaged) unaltered for all cygwin suffixes per vendor release. This is to reduce download overhead when a packaging or local patch related change is released and the vendor version hasn't changed. <... issues with primitive cygbuild...> Thats fine, lets not have a script. Like I said, it's not been a big issue, nor has it been waiting in the wings to be sprung on you. I came up with it on the fly. IMO all we do now is temporary pending setup getting full rpm/deb support, so the less we do the better, but to lower the bar for packagers we have to have a standard, and I think we all agree there are a few things (the common things in our discussion) that should be changed from the current standard. I hereby tack onto my proposal that the pristine source archive be always repackaged as a .bz2 file that extracts into a directory with a predictable name, preventing the need for metadata. The maintainer only needs to perform this operation ONCE per vendor release, not once per cygwin package update, so the overhead should be low. > The "one big patch" leads you to (a) using shar to generate binary files > (b) including meta-patches in your "one big patch" -- the big patch creates > secondary patches and shar files... Point taken. metapatches are ugly. So are packages that need a patch applied half way through the build process, or that ship _binary_ data in their distribution WITHOUT shipping the source to that data. (*) > Sure, I see WHY you want to do this -- so that all port-specific stuff is > selfcontained, and can be completely separated from the upstream "pristine" > source -- even downloaded separately. There's another reason why you'd > want to do this -- but that can wait until I put my ranting hat on. > > I don't think that goal is worth the hoops and hassles. > ... jpeg breaking trivial script... > -- but accesing /CYGWIN-PATCHES/buildscript is not easy to fix > without additional smarts or external metadata. Granted. see above. > Sure -- all of this is *possible* to do with the "one pristine src and one > patch only" method -- provided you jump thru enough hoops: > shar to create binaries (.jpg files, additional tarballs) .jpgs without source in distributions is evil. Just my 2c. > "one big patch" creates secondary patches, along with the buildscript covered already. > cygbuild parses the tarballs to extract dir and filenames rather than > assuming that they follow a sane pattern (which jpeg *doesn't*) likewise. > Or, you drive it all with one external package-specific script that KNOWS > about the peculiarities of the package (contains the metadata itself). > RPM+package.spec. bash+package.sh. package.spec/package.sh are *custom* > to the package, and KNOW about its wackiness if any. Debian has it's own equivalent. It's in the directory /debian. I suspect they require recompression of the upstream source to address the name issue - which isn't a big onus anyway, as they have to have a copy at their own mirrors for download for GPL compliance. > As you've probably guessed, I've created a jpeg-6b-5 example at > http://www.neuro.gatech.edu/users/cwilson/cygutils/packaging/ for you to > take a crack at. (FYI: This will NOT be my real jpeg-6b-5 release; there > are other changes I want to make to this package...like autoimport and > stuff...before I make a "real" new release) Cool.. will look tonight. > > I'm starting to get a bit pissed off here. I've spent hours writing these > emails, trying to point out the shortcomings in the debian-centric > proposal. Each of which is answered by "oh yeah, we can work around that > using..." Work around?! E.g. the debian scheme isn't powerful enough on > its own, so we have to work around... Very good point. I've been trying to point out my own thing: This is not a debian centric proposal. This is a "Robert has his own axe" thing. I nearly wrote a 'lets just leave this on ice' email a day or two ago. A debian centric proposal would be substantially different. I can do one if you'd like. (but give me a day to research it). ... > to creating the shar file mentioned previously, must also create a > secondary "apply after the ljpeg patch" cygwin-specific patch. I'll look at this when I play with jpeg. > I realize, jpeg is a corner case. But that's my point: the > one-src+one-patch method isn't powerful enough to handle THIS corner case > without heroic, non-maintainer-friendly efforts. Point taken. > But WHY must we jump thru these hoops? Apparently, to avoid the terrible, > horrifying possibility that we might have something OTHER than a single > pristine source tarball and a one big patch in our -src archives, because > debian is God's gift to package distribution, and anything they have done > MUST be the optimum. Nope, sorry if it's been coming across that way, it's been my own personal opinion here, not me trying to push debian. ***** If was trying to push debian in this discussion I *WOULD* be pushing the use of debian tools which I expect deal with the difficulties you have as they regularly apply multiple patchs to the vendors source - such as the lossless jpeg patch. ***** This is simply me saying "I loath, really loath, the SPECS etc constructs". > Robert's compromise seems to be "okay, everybody can also put this useless > .sh file in there too (even for complex packages [jpeg, > ncurses-non-new-libtool]), which will tell you to use cygbuild." Even when > cygbuild won't work (with complex packages) without hellacious > tarball-parsing ugliness. So have a specific script, I really really didn't mean to add more work suggesting it gets put into a global script, it just seemed logical to me. > I really suspect an ulterior motive, here, Robert. When will your port of > dpkg be ready? Since you seem intent on forcing this process to specify I don't have a port of dpkg. I last looked at it prior to implementing FIFO's in cygwin, and have not been back since. I have recently subscribed to the debain-w32 list - where cgf is as well. > The Debian Way as the official -src packaging standard for cygwin, I can > only assume that the reason is you want to use dpkg eventually for all > cgywin-building. I don't think that's going to be very popular with the > Red Hat brass, tho. As stated, no ulterior motive. For the record: Here is my complete stance on deb vs rpm vs setup.exe vs "The debian way". 1) Technically rpm and deb are quite similar. They solve similar goals. 2) rpm is a single-distribution vendors solution to updating software on client machines. 3) deb is a federated maintainer solution to updating software on client machines. 4) setup.exe is the bootstrap installer/package installer for a volunteer run distribution. 5) Neither rpm nor deb will work on cygwin without major internal logic changes. They need to understand open file renaming issues and solve them - and still run update scripts etc. This is *hard*. 6) If we are going to leverage what someone else has got - ports/rpm/deb - I believe we should copy deb. I believe this not because I happen to prefer the way deb does a lot of things, because debian has the closest contributor structure to what we do of all the linux distributions (except slackware, which is not really around much nowadays). I.e. if redhat had no community help, they would still release RH 8 with all the packages updated. They have staff to do that. The Debian Project manages the same feat without paid staff. We have the same challenge. 7) I will accept patches to setup to make it generate a rpm backend database. 8) I will accept patches to setup to make it generate a deb backend database. 9) The code I'm writing for setup is making it more modular, more able to be changed to do either 7 or 8, or even both. This is being done to generate the library cgf requested for cygcheck. 10) If noone has written 7 or 8 when I'm happy with setup.exe's internal structure I intend to start looking at reading deb archives. 11) Possibly the most important point. The Debian Way is not the packaging tools or formats, its the whole documented policy on whats in a package and whats not, and how things are packaged. And it's not always the most easy thing for the maintainer. IF WE (not me, WE), want to go the Debian Way, then lots of things have to happen. * We have to buy into the idea. * Redhat have to buy into the idea. * AFAIK Debian have to buy into the idea. ( if we want to leverage the mirrors and auto build systems and existing huge mass of ports). * We need to have ports of the core debian utilities including dpkg and apt-get - which I am NOT working on. (And I don't intend to). * open file issues in dpkg and all it's helper libraries and scripts need addressing. * setup needs to either interface directly or bootstrap and then call the appropriate scripts to do get the correct data recorded. IMO trying to become a debian port right now is idiocy. There is far to much to do, on a technical basis alone, for this to be a discussion topic. **** NOTE THAT the bullet list above is almost identical (skip the first 3 lines) if RPM is the chosen format. **** So there you have it. I have no intention of forcing a system on you Chuck. I have been focusing tightly on the things that bugged the hell out of me in RPM land and thats all. Rob (*) Squid has icons as well. Here's what happens: CVS stores a shar archive. developers have to have sharutils. make extracts the .gifs from the shar archive. make dist includes the .gifs in the distribution. if you are patching the .gifs, you only need to patch the shar archive, because that will cause the .gifs to be extracted again, so you don't need a metapatch or a binary patch.