Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Message-ID: <3E420A43.8010709@ece.gatech.edu> Date: Thu, 06 Feb 2003 02:09:55 -0500 From: Charles Wilson User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Ralf Habacker CC: cygwin AT cygwin DOT com Subject: Re: [avail for test] libtool-devel-20030121-1 References: <007901c2ccf0$f2b46080$0a1c440a AT BRAMSCHE> <3E41D181 DOT 5080808 AT ece DOT gatech DOT edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Ralf, this is the last message in the thread you mentioned. Charles Wilson said: > You've shown that win32_libid() as it is currently can take up to 5 > seconds on a single "library", while most "libraries" can be > identified in 0.3-0.6 seconds. But that doesn't address how much > longer win32_libid() adds to the total link time when building a > library. > > e.g. try this: > > You're linking libkdetext or somesuch. Halt the build right as the > link command (/bin/sh ../libtool -mode=link ....) is printed. Save > the command in a text file somewhere. > > ****REBOOT**** in order to clear your machine's memory/cache > whatever. Run the libtool -mode=link command (saved earlier) by hand, > with the current win32_libid() function -- and time it. > > ****REBOOT**** again. Then, time the link command again (but edit > your prebuilt libtool script and replace win32_libid() with this: > > win32_libid () { echo="x86 archive import" } > > (e.g. a no-op.) This will tell us exactly how much extra time is > added to a typical kde libtool-link command by the current definition > of win32_libid() AND what percentage of the total link time that > reflects. > > What I'm getting at is: is this 5 seconds for the absolute worst case > dependency (compared to 0.3 for most of them) really that bad, > compared to the total time involved in the link. How much does > absolute correctness (e.g. no prone-to-failure shortcuts) cost us? You never followed up on that. However, I'm betting that we can get most of the speedup we need by adopting the function below. It's not quite as agressive as yours, but it makes fewer assumptions. Results using the following snippet: time for fn in /usr/lib/* /usr/bin/* ; do echo $fn : `cygwin_libid $fn` done are as follows. I ran the test twice with each script and took the faster score. Hopefully that accounts for cacheing issues. old script: real 4m59.249s user 3m1.315s sys 2m8.683s Chuck's new script: real 3m40.841s user 2m1.821s sys 1m36.813s Ralf's proposal: real 4m24.808s user 2m38.784s sys 1m52.503s Ralf's proposal (with .exe fix): real 3m17.747s user 1m55.214s sys 1m24.283s Also, Ralf's could probably be sped up by using some of the 'head -n XXX' tricks in my version. 1801 files, 139MB. Note that .exe's are marked as 'x86 DLL' in both scripts, because soon the patch which allows exe's to have export tables will go into binutils, and we ought to allow linking against them. And really, there is very little difference between .dlls and .exes. The difference between this script and Ralf's proposal is that Ralf assumed that every file passed to the function was x86 and didn't check it. Also, any file named *.dll.a was assumed to be an import lib without checking. I believe that this loophole would tempt people to rename .a's as .dll.a's which would defeat the purpose of the entire system -- and would probably end up breaking tertiary programs. (**) I *don't* think folks will be tempted to rename .a's as .dll's, however. Also, Ralf's version (as proposed) marks .exe's as 'unknown' -- but that's easy to fix, and would have the side effect of improving Ralf's scores on my "benchmark". The downside to my version is that this symlink: libfoo.dll.a -> ../bin/cygfoo.dll will be marked "unknown".(*) Ralf's version will mark it as 'x86 archive import' -- which is acceptable given how the result is used, but isn't really correct. The original script would correctly identify it as 'x86 DLL'. (*) This could be fixed with some low-impact special case code (detect symlink, follow it, check the filename of the target), but I'll leave that as an excercise... (**) The way libtool works, if cygA.dll depend on cygB.dll, then the flag '-lB' is automatically appended to the link command whenever I try to build C.exe which depends on cygA.dll. But, if somebody broke the rules which building A.dll because they only had a static libB.a (and renamed it to libB.dll.a to fool libtool), the A.dll will re-export the symobls in libB.a. And you don't want that. But perhaps this simply falls under the heading of "the user is allowed to shoot themselves in the foot(head)". But this is a mighty shiny and tempting bullet..."Hmm, I can't link my dll because I only have a static version of this dependency. Hmmm...let me rename it!" --Chuck P.S. I'll be out of contact for a while, so "discuss". :-) Chuck's new version. ############################################## #!/bin/sh # changes: # 0) use extension to identify .dlls (user can then shoot # self in foot, but it is their own fault). Upside: # much faster AND allows successful link against a # *symlink* to a dll; original code did not. # 1) only call objdump and nm once, and only when necessary # 2) only need the first 3 lines of $OBJDUMP output; keep 10 # and throw away the rest. # 3) don't call awk # 4) only check to distinguish between static and import libs # within the first 100 lines of $NM output. This seems safe. # Normal import libs have an " I " symbol on line 5--8. # libcygwin.a [a hybrid import/static lib] has it on line 32. # only truly pathological cases would appear static for 100 # symbols before an import symbol appears... # 5) 'file' is fast; use it first to decide that $1 is an archive # before using $OBJDUMP to insure x86 and / or $NM to distinguish # between static / import. # 6) both .dll's and .exe's are marked as 'x86 DLL' -- because # they are, and because .exe's will soon be able to have export # tables, so we should allow linking to them. win32_libid () { win32_libid_type="unknown" if echo $1 | grep -E "\.dll$|\.exe$" >/dev/null; then win32_libid_type="x86 DLL" else if eval file $1 2>/dev/null | \ grep -E 'ar archive' >/dev/null; then if eval $OBJDUMP -f $1 | head -n 10 2>/dev/null | \ grep -E 'file format pe-i386(.*architecture: i386)?' >/dev/null ; then if eval $NM -f posix -A $1 | head -n 100 | grep " I " >/dev/null ; then win32_libid_type="x86 archive import" else win32_libid_type="x86 archive static" fi fi fi echo $win32_libid_type } OBJDUMP=objdump NM=nm win32_libid $1 ############################################## Ralf's proposal ############################################## #!/bin/sh # changes: # 1) only use nm for .a files # win32_libid () { win32_libid_type="unknown" if echo $1 | grep -E "\.dll$" >/dev/null; then win32_libid_type="x86 DLL" else if echo $1 | grep -E "\.dll.a$" >/dev/null; then win32_libid_type="x86 archive import" else if echo $1 | grep -E "\.a$" >/dev/null; then if eval $NM -f posix -A $1 | grep " I " >/dev/null ; then win32_libid_type="x86 archive import" else win32_libid_type="x86 archive static" fi fi fi fi echo $win32_libid_type } OBJDUMP=objdump NM=nm win32_libid $1 ############################################## -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/