Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Message-ID: <415BD855.4050205@x-ray.at> Date: Thu, 30 Sep 2004 11:56:37 +0200 From: Reini Urban User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; de-AT; rv:1.8a3) Gecko/20040817 MIME-Version: 1.0 To: Charles Wilson Cc: cygwin AT cygwin DOT com Subject: Re: How can you change the DLL name in the .idata section header of a DLL? References: <4159DD5A DOT 1070900 AT x-ray DOT at> <4159F03E DOT 2000603 AT cwilson DOT fastmail DOT fm> <20040929092843 DOT GN14978 AT cygbert DOT vinschen DOT de> <415AC7BE DOT 3010307 AT x-ray DOT at> <415AEBDF DOT 6090803 AT cwilson DOT fastmail DOT fm> <415B02CA DOT 9020906 AT x-ray DOT at> <415B270F DOT 4080105 AT cwilson DOT fastmail DOT fm> In-Reply-To: <415B270F.4080105@cwilson.fastmail.fm> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-IsSubscribed: yes [ I switched from cygwin-apps, where it is not appropriate ] Charles Wilson schrieb: > Reini Urban wrote: > >> Spoofing is a good point. >> But I might prefer to be able to update dynamic dependencies without >> breakage. openssl usually gets updated with an security issue. >> e.g curl or postgresql cannot deal with an openssl update which >> deletes cygssl-0.9.7.dll and replaces that with cygssl-0.9.8.dll. >> If we had just a cygssl.dll we can simply upgrade that and nothing >> breaks. > > > That's the way most other DLLs on cygwin are named. But cygssl is > different -- as a security/encryption DLL, considerations other that > "ease of slipstream replacement" are more important. Usually there is a > reason when a maintainer chooses to go against the common practice: > openssl is one of those. > >> Anyway, see cygperl.dll: >> DLL api's should stay binary compatible IMHO, unless it's really an >> issue (such as int32 => int64 on cygwin's switch from 1.3 => 1.5). >> You normally can trust MS DLL backwards compatibility. Newer features >> are just added, or new versioned DLL's appear - newlib2.dll - but old >> DLL's must stay. ("contract") >> With our libtool generated DLL names I'm not that happy. > > > Perl's build system is a nightmare. We're lucky we have a shared > version at all. But if you want to assist Geritt in fixing some of the > warts in the perl build system, I'm sure he'd welcome patches (and, as I > indicate below, concentrating on the ACTUAL problems rather than trying > to wholesale rework libtool/binutils/gcc/etc is probably a quicker path > to solving your issues). > > That name (cygssl-0.9.7.dll) is not libtool's fault. Libtool uses two > different mechanisms for setting the name of a DLL: > -version-info c:r:a > and > -release x.y.z > There are explicit rules governing how c, r, and a are to be updated, > and how the dll name is derived from c,r,a in order to insure that (a) > the library name changes rarely, and only when necessary to indicate > binary incompatibility, and (b) all dlls with the same name are > backwards compatible (e.g. old exe's linked against the old DLL will > still work with the new DLL). The libtool maintainers STRONGLY suggest > using the -version-info paradigm. On cygwin, this results in library > names like libintl-3.dll (NOT libintl-0.12.1.dll) Most cygwin packages > which provide DLLs obey these rules and follow this "suggestion". > > But the libtool folks do provide the -release mechanism, which is often > used to name the DLL using the major.minor.micro revision numbering > system. There are sometimes good reasons to do this, but usually it's > because the package maintainer doesn't understand library versioning, or > libtool. cygssl behave this way (even though it doesn't use libtool IIRC). > > If libtool names were used "correctly" by the packages, then you would > be "happy" -- and further, on cygwin we take GREAT PAINS to ensure that > compatibility DLLs are kept around virtually forever whenever DLL names > do change. So what, exactly, is your beef? Just perl and ssl -- and > THAT justifies reworking libtool and binutils, and rewriting the > underlying conventions of DLL names and cygwin's runtime loader? Many thanks for this explanation. I was wrong. >> Yes, indeed. >> People forget about -no-undefined in their Makefile.am, >> and then on any conflict libtool generates only the static version. >> Which was e.g. the case with having -lgdi32 in the dependencies, for >> which no syms could be found. > > > It seems that the problem is NOT, then, libtool or the > existence/non-existence of a .la file. The problem is the "conflict" > (e.g. build errors in the package in question). Focus your efforts on > fixing the problem, not band-aiding a "solution". True. 99% of all conflicts are just incorrect object and library order. Seldom they miss a .o, lib or add a wrong dependency. >> ------------------------------ >> >> BTW, a binutils question: >> "How can you change the DLL name in the .idata section header of a DLL?" > > Don't do that. > >> Rationale: >> DLL's cannot by symlinked on cygwin, unlike on other platforms. >> DLL names are usually quite version specific, which makes perfect >> sense on those other platforms. DLL names are built in the executable >> PE header, so on any DLL rename or upgrade, breakage will occur. On MS >> Win32 old DLL's just stay, so no breakage will occur. >> Not so with the cygwin setup.exe. > > > Faulty. DLLs don't often change names, and when they do, the old > versions stay around. If not, then THAT is the problem. Persuaded. > For instance, how many versions of libintl*.dll do you have on your > system? libncurses*.dll? cygtiff*.dll? cygjpeg*.dll? cyggdbm*.dll? > cygdb*.dll? > > Are you noticing a pattern yet? You caught me. Indeed. > It appears that your biggest beef is with the cygssl (and perl) DLL > naming scheme: it uses -release versioning (even tho it actually doesn't > use libtool to do it, IIRC). (perl does something even wierder; I'm not > going there). > > This presents a difficulty for packages which depend on cygssl*dll. > However, this IS one of those times when there is a good reason to > change the DLL name on every security fix (e.g. micro release version). > It means you can't silently slip an SSL bugfix into an older version > of imapd, but you also can't silently slip in NEW SLL security holes > possibly introduced by the new library. > > Each dependent app can make its own determination that "Yes, the bugfix > in version+1 outweighs the risk of new bugs, so I will now recompile > against the latest version". > > Downside: on our platform, at present, it appears that "older" versions > of cygssl*.dll are not retained, because the "openssl" package is not > "split" into independent "library" and "other" packages: e.g. > libssl0_9_7 package contains /bin/cygssl-0.9.7.dll > libssl0_9_8 package contains /bin/cygssl-0.9.8.dll > while > openssl package contains every thing else. > That is a maintainer issue, and I'm sure there is a good reason (perhaps > the openssl package itself is not amenable to this splitup. For > instance, suppose that the "everything else" is version-specific, yet > lives in /usr/share/openssl/. You'd need multiple versions of > "everything else" but they all want to own the same directory. Problems.) > >> Solution: >> Use as much generic dll names as possible (such as cygperl58.dll, >> cygssl-1.dll, ...). > > But that IS what we do, *whenever* possible. When we (cygwin pkg > maintainers) don't, there is usually a good reason. One which, quite > frankly, you obviously haven't done enough research to identify == > because you're focusing on libtool and binutils issues, when it appears > the difficulties are much more localized to the specifics of a few > packages like openssl and perl. > >> Provide an objdump option to rename a certain DLL path to fix breakage >> for older libs, until they get rebuilt. > > God no. If anyone ever does such a thing to any of the DLLs supplied by > my packages, and then comes complaining to the cygwin list because "tiff > doesn't work anymore" not only will I not support their bastardization, > but I will drop the tiff package from the distro like a bad habit. :) If nobody wants it, I will not write such a hack. Rebuilding the DLL is easier. > I do NOT have enough time to support people who take a hexeditor to the > internals of the packages I supply. Use them as is, or don't use them > at all. Thanksfully the section headers are protected by two checksums (and even more magic), so nobody can hexedit the DLL name :) I knew that there must be an explanation, but I couldn't find it. So it would only fix the perl problem, but for perl alone it will be MUCH easier to provide a package update. Agreed. And no package so far uses a embedded shared perl interpreter, to my knowledge. Just postgresql starting with v8.0, for v7.x it is broke). That are my primary concerns, because I just want to take over from Jason. >> One could also hardcode the DLL path to some /usr/lib; /usr/libexec, >> /usr//lib, so that we can rid of /usr/bin cyg*.dll pollution. >> But this will only work if the cygwin mount will not change. The >> windows process loader needs the windows path. So >> "/usr/lib/cygperl.so" might be "c:/cygwin/lib/cygperl.so". No major >> problem if objcopy will support that. Something like rebaseall. > > The windows loader will only load .dlls (and .ax but that's a special > case for the COM+ architecture AFAIK). And .dlls follow certain rules. > It will not load .so's even if the .so contains pei-386 code. You need > a special loader to do this instead; something like libltdl would have > to be built into cygwin1.dll -- but it wouldn't be exactly libltdl for, > I hope, obvious reasons. You got me wrong. My "/usr/lib/cygperl.so" up there is of course a binary DLL (Ix86 COFF with PE), with just a funny filename. In my experience LoadLibrary (dlopen) will load any filename ending with any extension, as long as it is a true DLL - relocatable Ix86 COFF with PE and some more restrictions. (PEI is just the win32 version, an old header prepended, printing the "This program cannot be run in DOS mode" warning. I believe.) As long as you provide the basename+extension. path not necessary. basename is not enough (Default: ".DLL"). Hell, Windows even thinks that the extension of "perl5.8.1.dll" is "8.1.dll", and not "dll". But this may vary, depending on the library function used internally. >> Another idea is to make those .idata sections weak and provide a hook >> to go over cygpath resolution via libltdl. If that will work. >> Weak is rather new I suppose, and I haven't looked at the >> implementation of lazy linking with ld. > > Hell, I don't even know if weak WORKS on cygwin; it was developed over > in the mingw arena; as far as I know they didn't even test it on our > side of the fence. And you want to replace the shared library mechanism > with it? Walk before you run... I believe it was developed by the WINE folks, because MSVC suddenly supported lazy loading (requested by some folks, like Autodesk) and the WINE folks had to load these beasts. gcc-core (or just binutils?) picked it up lately, but I don't think the decided on the ld switch naming yet. I don't follow these lists, just the ChangeLog. ld --help |grep delay ld --help |grep lazy It is just an early idea, but I don't think a bad one. And there's no hurry: I don't want to replace it. Just talk about it. If would make sense. > All of these ideas (like using a cygwin runtime loader instead of > relying on DLLs and the Windows Runtime Loader) have been brought up > before. But nobody has demonstrated any working code, nor demonstrated > 1/10th the amount of thought necessary to forestall the problems > inherent in those plans. Show me the code. Where? (just a rough substring would be enough to google it) I couldn't find it anywhere. Most of the things I found at WINE, and know from my former MSVC + AutoCAD + FFI days, where Autodesk developed its own DLL loading + extension scheme, just to support C++ class extensions and overloading by external DLL's. Which I hacked and extended for lisp. > Look, I know this message (and my last) have been snippy. I'm sorry for > that, but about every three months somebody comes along with a great > idea to "fix" a problem with DLLs on cygwin. (Usually -- although not > in your case -- the problem is actually between the keyboard and the > chair) They almost never demonstrate a thorough understanding of the > issues surrounding dynamic libraries, naming schemes, the Windows > Loader. Further, they often approach the current paradigms with faulty > assumptions (e.g. cygwin DLL names change often; they don't; DLLs are > removed by setup willy-nilly; they aren't [current problems with > cygwin1.dll+setup notwithstanding]) nor do they understand the reasons > things are done as they are presently. > > So please, before you continue on this path, go back to the earlist > cygwin-apps archives, and read...then, read it again. We HAVE thought > about these issues. -- Reini Urban http://xarch.tu-graz.ac.at/home/rurban/ -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/