Message-ID: <4C35E1E1.1080906@cwilson.fastmail.fm>
Date: Thu, 08 Jul 2010 10:34:09 -0400
From: Charles Wilson <cygwin AT cwilson DOT fastmail DOT fm>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: Re: cygport cross compile(r) support [was: Re: cygport patch: suppress libtool fixup step]
References: <4C3215FC DOT 3010309 AT cwilson DOT fastmail DOT fm>	 <1278402537 DOT 5784 DOT 8 DOT camel AT YAAKOV04>  <4C337C6E DOT 2080003 AT cwilson DOT fastmail DOT fm>	 <1278455320 DOT 4032 DOT 49 DOT camel AT YAAKOV04> <4C33EEC1 DOT 9050600 AT cwilson DOT fastmail DOT fm>	 <1278519024 DOT 1096 DOT 90 DOT camel AT YAAKOV04> <4C34D3BF DOT 7020404 AT cwilson DOT fastmail DOT fm>	 <1278548055 DOT 6960 DOT 59 DOT camel AT YAAKOV04>  <4C3534E6 DOT 90505 AT cwilson DOT fastmail DOT fm> <1278560355 DOT 6960 DOT 114 DOT camel AT YAAKOV04>
In-Reply-To: <1278560355.6960.114.camel@YAAKOV04>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com

On 7/7/2010 11:39 PM, Yaakov (Cygwin/X) wrote:
> On Wed, 2010-07-07 at 22:16 -0400, Charles Wilson wrote:
>> Hmm. That's what I *was* doing: JonY's -src provides a cygport that

> I didn't mean the .cygport(5), I meant cygport(1).  The goal is to make
> these workarounds unnecessary.

Sure. There's three "status levels" of cygport support for any desired
behavior:

 1) no way to coerce cygport(1) to do what you need without patching
    cygport(1) itself
 2) odd workarounds in .cygport(5) can make cygport(1) do what you need
 3) every thing works nice and cleanly

No question, we're at (1) right now.  With my two small patches, we can
be at (2).  It'll take more work -- and different and more complex
solutions -- to move from (1) all the way to (3).

>> Honestly, I just don't get this.  Are you participating in a live
>> #mingw-w64 session, or looking at old logs?
> 
> Live; I've been there since yesterday.

Hmm.

>> The mingw64-gcc I *just built* /is/ multilib, ...
>> So, it *does* build. ... JonY and I both built it.
>>
>> Hence my confusion.
> 
> Then I'm missing something, because the packages I grabbed from
> sourceforge didn't build multilib, and I was told that additional
> patches were necessary.

Well, NightStrike said in another thread that many of the #chat folks
had complained that it was a difficult and error prone process in
general, but that JonY has gone through and documented all of the
necessary steps, and his .cygport(5) implements those steps, so...

>> I agree (willing to be persuaded that all three compilers could be
>> NON-multilib, but it seems JonY *wants* to provide multilib -- I guess
>> for just the default-to-64bit compiler -- and I'm not gonna stand in his
>> way, if he'd doing the work.
> 
> I don't mind if it's not multilib, it would certainly make things
> easier.

Unless Corinna/cgf really put their foot down and forbid multilib, I
think we should defer to whatever JonY wants to do.

Now, his ORIGINAL proposal was 64bit only, non-multilib.  Maybe he'd be
pleased to go back to that; my feeling is he'd rather do multilib if
possible, but I'm not sure, and certainly don't speak for him.

>> But the difference is, we are NOT actually talking about two completely
>> separate platforms.  You CAN run mingw apps and use mingw DLLs from
>> "cygwin" directly (mingw64-32, or mingw.org), because it's running on
>> top of plain old windows.  Ditto mingw64-64 -- assuming you're on a
>> 64bit Windows.
> 
> I still believe that within the context of Cygwin, that mingw* is no
> different than any other cross-compiler.  This is an important
> distinction when we talk about runtime data (more on that below).

Yes...

>> And...libtool already makes allowances for this: that's why cygwin
>> specifically uses the 'cyg' prefix for all DLLs, so they won't conflict
>> with mingw libs that could be in use simultaneously on the same machine (*).
> 
> And I won't mention how many problems the 'cyg' prefix has *caused*.

That is true. However, you weren't around before its introduction, back
in the bad old days where building DLLs at all on cygwin and mingw was
extremely difficult, and required massive internal patching (no
autoimport support, and nobody in the free software economy had existing
support for MSVC).

What's worse, is every Tom Dick and Harry would create their own (native
w32) DLL versions of software, all with slightly different mechanisms
and subtly distinct APIs -- and all named "libz.dll" or "libtiff.dll".
This caused all kinds of heck when your cygwin program picked up NCSA
netscape's libjpeg.dll by mistake.

The cyg prefix -- for all the trouble it causes in dlopen() scenarios --
is a godsend for the much-more-common non-dlopen cases.  We are
unaffected by any native DLL hell episodes...

>> I would REALLY like to be able to set $PATH to include
>> /usr/sysroot/mingw32/bin and test (for example) mingw-bzip2.exe WITHOUT
>> having to go manually copy
>>    /usr/sysroot/mingw32/lib/libbz2_2.dll,
>>    /usr/lib/gcc/i686-pc-mingw32/4.6.0/libgcc_s-1.dll
>> to /usr/sysroot/mingw32/bin/ before I can!  (And then, have to remember
>> to redo that every time something gets updated in its "proper" buried
>> location).
> 
> The (sole?) purpose of /usr/$target/bin is to contain binutils'
> cross-tools.

WAIT.

When did I say "/usr/$target/bin"? I never said that. I said

   /usr/sysroot/$target-shortname/bin

And I'm not even sure that THIS "sysroot" tree should be the same as the
one you may (or may not) pass to gcc's configure with --sysroot.  It's
just: the place where non-toolchain packages that are compiled using a
particular cross compiler will go.  E.g. you'd configure mingw-zlib with
--prefix=/usr/sysroot/mingw32.


"/usr/sysroot/" as a literal string, not a virtual placeholder that
means /usr/$target.  This so-called "sysroot" is DIFFERENT than
/usr/$target/bin.

Remember, here's the fedora-mingw filesystem that was the motivating idea:
http://fedoraproject.org/wiki/Packaging/MinGW#Filesystem_layout

[root]
  |
  +- etc
  |   |
  |   +- rpm
  |       |
  |       +- macros.mingw32
  |
  +- usr
      |
      +- bin   - Links to cross compiler toolchain
      |   |
      |   +- i686-pc-mingw32-cpp
      |   +- i686-pc-mingw32-gcc
      |   +- i686-pc-mingw32-g++
      |   +- ... etc..
      |
      +- lib
      |   |
      |   +- rpm
      |       |
      |       +- mingw32-defs   - custom helper scripts for
auto-requires, binary stripping, etc
      |       +- mingw32-find-provides.sh - extra DLL names
      |       +- mingw32-find-requires.sh - discover required DLL names
      |
      +- i686-pc-mingw32  - root of mingw toolchain and binaries - see
next diagram


/usr/i686-pc-mingw32
  |
  +- bin  - Cross compiler toolchain
  |   |
  |   +- cpp
  |   +- gcc
  |   +- g++
  |   +- ... etc ...
  |
  +- lib  - Cross compiler toolchain support libraries / files
  |
  +- sys-root  - root for cross compiled binaries
      |
      +- mingw
          |
          +- bin     - cross-compiled binaries & runtime DLL parts
          +- doc     - documentation
          +- include - include files for cross compiled libs
          +- lib     - cross-compiled static libraries & linktime DLL parts
          |   |
          |   +- pkgconfig  - pkg-config definitions for libraries
          |
          +- share
              |
              +- man


Except I may have misinterpreted. I keep referring to:

   /usr/sysroot/mingw32/{bin,lib,...}
as distinct from
   /usr/$target/{bin,lib,...}

But I think a proper interpretation of the fedora diagram is

   /usr/$target/sysroot/mingw32/{bin,lib,...}
which is STILL distinct from
   /usr/$target/{bin,lib,...}

> As Cygwin native EXEs they depend on the libraries
> in /usr, not those under /usr/$target.  Therefore is it unnecessary to
> put the /usr/$target/lib DLLs in /usr/$target/bin, because the EXEs
> there don't need them.  

And I don't WANT to put them there.  I want to put them in a
$target-specific directory DISTINCT from the place that the
cross-compiler .exes go.

> Not only that, if you're not careful with PATH,
> then /usr/$target/bin/foo.exe will come before /usr/bin/foo.exe, and
> that's not good.

Well, sure, my normal cygwin $PATH wouldn't include
  /usr/sysroot/$target-shortname/bin
as a routine matter.  I said I want to be able to set it, temporarily
and as necessary, for testing.

Maybe I'd also have a batch file that starts up a cmd.exe window, with
no cygwin in sight, but with $PATH set to
C:/cygwin/usr/sysroot/mingw32/bin if I want to use some custom
mingw32-ish tool I've built. And it ought to be able to "find" all of
its DLLs...

>> Again, ONLY if you shackle Dave Korn and JonY together at the hip, and
>> require that all compiler versions, for any $target, remain in lockstep
>> forever.  I don't think that is fair to either maintainer.  (Or to John
>> Doe, if he steps forward to provide a $target=linux cross compiler).
> 
> I suspect that a matching $major.$minor would be enough for NLS to work;
> that's why we wanted to know which version Dave would package next.  I
> would think that we're getting to the point that we can keep the Cygwin
> GCC up to date.

We're still stuck at 4.3.4 -- after five(?) years stuck at 3.4.x.  4.4.0
was released in April 2009.  4.5.0 three months ago.  I'm not knocking
DaveK, he's done great work.  But the mingw64 guys have reported issues
with 4.5.0 and want to provide 4.6.x (which is still in stage 1!).  I'm
betting that DaveK has no intention of shipping a non-released version
of gcc; and especially not one in stage 1: "During this period, changes
of any nature may be made to the compiler. In particular, major changes
may be merged from branches."

mingw64 plays at the bleeding edge. Mingw.org and cygwin do not.  I
reckon there will *always* be at least a minor version number difference
between them.

>> If /opt is completely unacceptable, then I'd prefer allowing the cross
>> toolchains to use $target-specific --datarootdirs.
>>
>>    /usr/bin/$target-foo.exe
>>    /usr/$target/{bin,include,lib}
>>    /usr/$target/share/{info,man,locale} <<<--- datarootdir
>>    /usr/lib/gcc/$target/$version
> 
> Remember that (AFAIK) only binutils and gcc will have runtime data
> (locales) because they are the only ones that are meant to be run; other
> packages (mingw*-runtime, mingw*-pthreads, mingw*-zlib, etc.) need no
> runtime data because they are not intended to be run within Cygwin.

Huh? What does THAT have to do with the block of text quoted immediately
above?  I *am* talking about, solely and specifically, the way in which
the TOOLCHAIN packages are compiled (e.g. "cygport 'cross-compile'
support type #2", from an earlier email exchange).  In this section, I
am NOT talking about how *other* packages, like mingw-zlib or even
mingw-pthreads, are configured and compiled (e.g. "cygport
'cross-compile' support type #1")


Yaakov, the more I think about it, the more I believe that there really
should be two different cygclasses here:

cross-toolchain.cygclass
   + maybe a modification/extension, mingw-cross-toolchain.cygclass

for controlling how cygport builds packages that are PART of a cross
toolchain: e.g. binutils and gcc.

and

cross.cygchain

for controlling how packages that themselves are NOT intrinsic parts of
the toolchain, like mingw-xz.  Not sure if (mingw-runtime,
w32api)|(mingw64-crt-lib, mingw64-crt-headers) belongs in this category,
or the previous one.  Probably this one.


And, the behavior of those two cygclasses is *different*.

> The info and man7 pages are just documentation, not real runtime data,
> and don't need duplicates.

But if the major or even minor version differs, then they are not
duplicates. They are *different*.

>  The man1 pages are $target-prefixed so they
> don't collide.  AFAICS the only real runtime data is the locales.  I
> don't think that justifies a separate $datarootdir; either we "borrow"
> from cygwin-gcc4 (even if they're not a perfect match), or we can
> disable NLS altogether.

I really just don't understand this obsession with saving a few
kilobytes of disk space...so WHAT if there are two copies of *possibly
different* versions of gcc.info (ditto w32api...).  At least I know that
/usr/share/info/gcc.info is always in sync with /usr/bin/gcc, and
/usr/$target/share/info/gcc.info is in sync with /usr/$target/bin/gcc.

If I have four different gcc compilers (three cross, mingw64,
mingw64-32, mingw.org) and the native cygwin one...ALL of which may be
at different minor version levels (*), which version does
/usr/share/info/gcc.info match?

Just one of them.

That sux. How can I RTFM if my FM doesn't match my FCompiler?

(*) Let's assume that all cross compilers ARE kept in lock-step with
cygwin's gcc.  What if I want to stick with the prev: version of
cygwin-gcc but use the test: version of mingw64-gcc? Or vice versa?

This "keep all versions the same" idea is just not workable in the real
world -- and trying to force it will just make everybody miserable,
including DaveK and JonY.  Let's make their lives easier, instead!

>> Although the *cygwin* specific docs would still go in
>>    /usr/share/doc/Cygwin/*
>> and maybe
>>    /usr/share/doc/$pkg/
>> but perhaps that last one also belongs over in the 'new' datarootdir.
> 
> No, because that won't collide with the cygwin-native version.

Sure, it won't collide.  However, if you DO use a custom --datarootdir
to address the other issue, but insist that the non-info, non-man docs
go in /usr/share, then you have to do

   --datarootdir=/usr/$target/share --docdir=/usr/share/doc/${PN}

which is odd, but not terrible.  You could even insist that

   --mandir=/usr/share/man

since the manpages do get renamed.  IF you move the info files over to
/usr/$target, does it make more sense for non-info documentation to live
right next door, or all the way over in /usr?  I can see both arguments.

But if the version of the various compilers differ -- and they will,
especially if users are allowed to choose prev: or test: versions of
either compiler -- then the ONLY way to ensure the correct i18n files
are used is to configure with a custom --datarootdir, regardless of what
you do with the options controlling directories under it, like --docdir
and --mandir (there is no such thing as --localedir, so you can't
specifically relocate it by itself).

Or turn off i18n, which is definitely suboptimal.

>> what about this:
>>
>> mingw64-tc64-gcc.cygport:
>> ...
>> inherit cross
>> inherit mingw-cross
>> ...

> 
> The libtool fixups are not handled by a cygclass, so there's no need for
> two separate cygclasses.

But if you want those fixups to behave differently for one class of
cygports compared to another -- I know you don't currently believe this
is necessary, but assume it is for the sake of argument -- how to you
impose that distinction, from the .cygport(5) level?

inherit cross-toolchain
export MINGW_TOOLCHAIN_LIBTOOL_FIXUP_STYLE=1
export MINGW_TOOLCHAIN_ABSPATH_TO_TARGET_DLL_DIR=
	/usr/sysroot/$target-shortname/bin

?

--
Chuck

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple