Subject: Re: experimental texmf packages
From: Jan Nieuwenhuizen <[email-masked]>
To: Charles Wilson <[email-masked]>
Cc: "Gerrit P. Haase" <[email-masked]>
Date: 05 Dec 2001 13:27:10 +0100

Charles Wilson <[email-masked]> writes:

First, thanks for your reply on my ranting mail, that went to the list
by accident (look at Gerrit's address).

> Silly? no.  Difficult and painful, prompting questions like "surely
> there is a better way"?  yes.

Well, I still thought it was silly to have identical forward and
backward patches, when we have a patch -R flag.  Normal mode of
operation is to (apply and) supply forward patches; now my (and
everyone else's packaging procedure) I must be rewritten to do unpack,
rename -> .orig, patch, run diff from .orig tree.  Or, be hacked to
try to determine the direction of the patch before applying, but

> You need to understand where that requirement came from: protective
> action against maintainers who flake out.
[snip]

I understand this must be avoided at all cost.

> So, we said "provide prepatched source" -- because people wanted to
> just unpack, configure, make -- but also "provide the diff between
> your source and the official version".

Sure, but I don't see any advantage in reversed patches.

> Yes, it's a pain.  Yes, there are better ways.  But silly? no.  (I think
> the silly part was insisting that the source code be PRE-patched.

Yes, I think that requirement is silly too.  Because people will be
using a download cache, the pristine source is there too.  Build
environments now hold pristine sources, patched sources, and patches
in both directions.  But, whatever gets you going :-)

> there is no iron clad, my-way-or-the-highway standard, but agreement on
> a number of principles:

> 1) official, unmodified source code should be provided
> 2) with a diff (or multiple diffs) that are used to turn the official
> source into cygwin source
> 3) there should be some sort of automated script -- shell script,
> external makefile, *something*, to drive the entire cygwin build

I'm not sure I understand.  Have the 1-diff,reversed diff, pre-patched
-src requirements been dropped?  Currently, I dist the pristine
tarball as foo-orig.tar.gz; and the pre-patched cygwin, including the
applied patches, as -src.tar.gz.

Is this sort of what it should be now:

    $ tar xzf foo-1.1-1-src.tar.gz
    $ ls foo-1.1-1-src
    foo-1.1-1-src/foo-1.1.tar.gz  # pristine src
    foo-1.1-1-src/foo-1.1-1.patch # forward diff
    foo-1.1-1-src/other.patch
    doit.sh  # how to guess this name?

    $ cat foo-1.1-1-src/doit.sh
    #!/bin/bash
    set -e
    tar xzf foo-1.1.tar.gz
    mv foo-1.1 foo-1.1-1
    cd foo-1.1-1
    patch -p1 < ../other.patch
    patch -p1 < ../foo-1.1-1.patch
    autoconf
    ./configure --prefix=/usr
    make
    make install prefix=/tmp/foo-root/usr
    cd /tmp/foo-root
    tar czf /usr/src/foo-1.1-1.tar.gz *

How about an example dummy package like this?

Now every package should contain a build script.  Isn't that a step
backwards in the evolution?  You'd want a central tool(set) to do the
building and packaging, and package specific scripts/makefile
snippets.

> That's it.

> > Three years ago, I ported RPM
> 
> You did that?  I thought *I* did that (cygutils, perl-5.005 modules, mid
> 1998).  I think it's silly for people to re-port the same applications
> for the umpteenth time... :->

:-)

> But you gloss over the *reasons* it didn't catch on:
> 
>   1) There was no true NATIVE port, so you couldn't use rpm to install
> cygwin itself.

That is a real problem.  But it solved the cross-building, the
tarball/patch issue, binary packaging, dependencies, pre/post
install/remove scripts for me.

>   2) Nobody, and I mean NOBODY, has followed thru on porting, packaging,
> and maintaining the berkeley-db and rpm packages as official,
> cygwin-mirror-system distributed packages (surely that's the FIRST step
> before attempting to use rpm as the be-all and end-all package
> management application for cygwin?  PROVIDE it and maintain an official
> port?)

I didn't have the webspace to do that.  But I did implement
.patch.rpm's (supply pristine source yourself) to work around that,
and put it up here (where it sits and rots):

    http://www.xs4all.nl/~jantien/cygnus/b20/PRPMS
   
Also, while I am/was willing to help out the Cygwin project, I'd
rather work on LilyPond than reinvent packaging/do porting.  I
hoped people to be interested, discuss and maybe take over what I
tried to start, but nothing much happened, then.

> > and I built a set of cross-build scripts
> > from the .spec shell-snippets.  
> 
> that's a neat idea.  Can I see?

Sure (it's been posted to the cygwin list a couple of times, but as
there where no reactions, I decided not to spam the list any
further).  Find the latest here:

    http://lilypond.org/people/jan/gnu-windows/cygwin-cross-1.3.6.0.tar.gz

It's a cardhouse of shell snippets.  The idea is simple: there are
three flavours of packages: binary (= untar, that's it), cross (eg,
binutils, gcc), native (guile, lilypond etc).  Each flavour has a
number of stages that should be performed: download, configure, make,
install etc.  For each stage there's a template script for a well
behaved package, but instead of this template a specific script can be
inserted, eg:

    cygwin-cross-1.3.6.0$ ls -1r stage/*.configure
    stage/template.configure
    stage/native.configure
    stage/lilypond.configure
    stage/guile.configure
    stage/gcc.configure

Then there's a script: bin/cygwin-cross.sh, that simply finds and
sources all stages for all packages for all flavours; upon first run
this recreates a cross-build environment in $HOME/usr/src/cygwin.  If
you run it, please first do it in a safe environment, because it's,
well, hairy shell scripts and evals, and you'll be only the third
user, probably.  Esp. little things like cygwin's source and binary
tarball naming, and the .gz/.bz2 make it a bit hairy.

> However, "rpm-like" is still on the table in
> the sense of:
>  1) provide pristine source
>  2) provide the patches
>  3) provide "something" to autodrive the build (shellscript+sh.exe,
> debian.rules+dpkg, spec+rpm, whatever)

Good.

> a) source code autobuilding and packing
> b) installation and unpacking
> 
> Part A assumes you already have a working cygwin development environment
> installed on your machine. Part B must be doable on a "virgin"
> system.
> Thus, the installer-unpacker must NOT rely on cygwin.

What I did, using rpm, was provide a rpm-x.y.z.bin.tar.gz, including
necessary libs.  If setup.exe can't run rpm, it gets and untars the
tarball.  After that, it installs/upgrades the cygwin, rpm, zlib
.rpm's.  Of course, that's the easy part.  But it wouldn't be hard to
let setup.exe handle the final placment of any binaries that were in
use during the rpm phase.  Is that too kludgy?

> (This sidesteps the question about "where do you put the package
> management database for a native windows port of rpm/dpkg/whatever?"
[..]
> You can't put it in /var/lib/rpm/ (because cygwin installation will
> manipulate the mount table; you don't know where /var will BE until
> after the installation is finished).

Ok, so, a native port, that should be more feasible with the current
state of mingw(?), would pose other problems.  But these seem even
easier to solve?

What if you'd only allow setup.exe to choose the cygwin root for the
first installation?

> Of all the let's-use-rpm proposals, Dario's makes the most sense --
> although Robert's eventual plan for setup.exe ain't bad either.

Ok, so I missed not only this discussions, but also actual code.
Ignore my rpm questions above, I'll have a look into these two
alternatives first.

> <rant>
> Jan, next time try CONTRIBUTING to the discussion, instead of calling
> the result silly after the fact.
> </rant>

You're right to rant here, because my mail really was a rant too, I'm
sorry.  But I always try to put code first, and then start a
discussion.  In this case, I've presented rpm stuff, my cross-build
scripts, and some months ago a patch for a setup.exe postremove
feature.  Maybe it was bad timing, maybe my code was just too bad, but
on each of these occasions I got so little sensible feedback if any at
all.  That turned me off a bit.

> I appreciate your efforts w.r.t. tetex-*; please don't take the above
> rant as criticism of THOSE contributions.

Thanks; no I won't.

> we obviously could have benefited from some additional input -- but
> got very little.  It was mostly a dialogue between Robert and I, and
> I'm still a little disappointed at the lack of wider participation
> in that discussion.

Ok.  I've not been a Cygwin user myself for some years now, I'm only
parasiting on Cygwin to provide a windows port for LilyPond.  That
makes that I don't follow all Cygwin stuff all the time; I only look
into it when there are major updates/and or when setting up a new
cross build environment fails.

About a year ago (?) I asked Christopher if I could be on the
cygwin-developers, to stay in touch a bit more and deliver the
occasional patch to setup.exe or cross-compile fix, but he judged my
contributions too low to warrant membership at that time.

As I had the impression that the only sensible discussions were at
cygwin-developers, and it my posts on cygwin were mostly ignored (even
though Christopher once tried to direct some attention to one of my
posts), I gave up.  Being only involved occasionally, and going
through the archives directly, I didn't find out about the apps list
(and open membership) until recently...

Greetings,
Jan.

-- 
Jan Nieuwenhuizen <[email-masked]> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/