Mail Archives: cygwin/2003/02/06/02:11:52
Ralf, this is the last message in the thread you mentioned.
Charles Wilson said:
> You've shown that win32_libid() as it is currently can take up to 5
> seconds on a single "library", while most "libraries" can be
> identified in 0.3-0.6 seconds. But that doesn't address how much
> longer win32_libid() adds to the total link time when building a
> library.
>
> e.g. try this:
>
> You're linking libkdetext or somesuch. Halt the build right as the
> link command (/bin/sh ../libtool -mode=link ....) is printed. Save
> the command in a text file somewhere.
>
> ****REBOOT**** in order to clear your machine's memory/cache
> whatever. Run the libtool -mode=link command (saved earlier) by hand,
> with the current win32_libid() function -- and time it.
>
> ****REBOOT**** again. Then, time the link command again (but edit
> your prebuilt libtool script and replace win32_libid() with this:
>
> win32_libid () { echo="x86 archive import" }
>
> (e.g. a no-op.) This will tell us exactly how much extra time is
> added to a typical kde libtool-link command by the current definition
> of win32_libid() AND what percentage of the total link time that
> reflects.
>
> What I'm getting at is: is this 5 seconds for the absolute worst case
> dependency (compared to 0.3 for most of them) really that bad,
> compared to the total time involved in the link. How much does
> absolute correctness (e.g. no prone-to-failure shortcuts) cost us?
You never followed up on that. However,
I'm betting that we can get most of the speedup we need by adopting the
function below. It's not quite as agressive as yours, but it makes
fewer assumptions. Results using the following snippet:
time for fn in /usr/lib/* /usr/bin/* ; do
echo $fn : `cygwin_libid $fn`
done
are as follows. I ran the test twice with each script and took the
faster score. Hopefully that accounts for cacheing issues.
old script:
real 4m59.249s
user 3m1.315s
sys 2m8.683s
Chuck's new script:
real 3m40.841s
user 2m1.821s
sys 1m36.813s
Ralf's proposal:
real 4m24.808s
user 2m38.784s
sys 1m52.503s
Ralf's proposal (with .exe fix):
real 3m17.747s
user 1m55.214s
sys 1m24.283s
Also, Ralf's could probably be sped up by using some of the 'head -n
XXX' tricks in my version.
1801 files, 139MB.
Note that .exe's are marked as 'x86 DLL' in both scripts, because soon
the patch which allows exe's to have export tables will go into
binutils, and we ought to allow linking against them. And really, there
is very little difference between .dlls and .exes.
The difference between this script and Ralf's proposal is that Ralf
assumed that every file passed to the function was x86 and didn't check
it. Also, any file named *.dll.a was assumed to be an import lib
without checking. I believe that this loophole would tempt people to
rename .a's as .dll.a's which would defeat the purpose of the entire
system -- and would probably end up breaking tertiary programs. (**) I
*don't* think folks will be tempted to rename .a's as .dll's, however.
Also, Ralf's version (as proposed) marks .exe's as 'unknown' -- but
that's easy to fix, and would have the side effect of improving Ralf's
scores on my "benchmark".
The downside to my version is that this symlink:
libfoo.dll.a -> ../bin/cygfoo.dll
will be marked "unknown".(*) Ralf's version will mark it as 'x86
archive import' -- which is acceptable given how the result is used, but
isn't really correct. The original script would correctly identify it
as 'x86 DLL'.
(*) This could be fixed with some low-impact special case code (detect
symlink, follow it, check the filename of the target), but I'll leave
that as an excercise...
(**) The way libtool works, if cygA.dll depend on cygB.dll, then the
flag '-lB' is automatically appended to the link command whenever I try
to build C.exe which depends on cygA.dll. But, if somebody broke the
rules which building A.dll because they only had a static libB.a (and
renamed it to libB.dll.a to fool libtool), the A.dll will re-export the
symobls in libB.a. And you don't want that. But perhaps this simply
falls under the heading of "the user is allowed to shoot themselves in
the foot(head)". But this is a mighty shiny and tempting bullet..."Hmm,
I can't link my dll because I only have a static version of this
dependency. Hmmm...let me rename it!"
--Chuck
P.S. I'll be out of contact for a while, so "discuss". :-)
Chuck's new version.
##############################################
#!/bin/sh
# changes:
# 0) use extension to identify .dlls (user can then shoot
# self in foot, but it is their own fault). Upside:
# much faster AND allows successful link against a
# *symlink* to a dll; original code did not.
# 1) only call objdump and nm once, and only when necessary
# 2) only need the first 3 lines of $OBJDUMP output; keep 10
# and throw away the rest.
# 3) don't call awk
# 4) only check to distinguish between static and import libs
# within the first 100 lines of $NM output. This seems safe.
# Normal import libs have an " I " symbol on line 5--8.
# libcygwin.a [a hybrid import/static lib] has it on line 32.
# only truly pathological cases would appear static for 100
# symbols before an import symbol appears...
# 5) 'file' is fast; use it first to decide that $1 is an archive
# before using $OBJDUMP to insure x86 and / or $NM to distinguish
# between static / import.
# 6) both .dll's and .exe's are marked as 'x86 DLL' -- because
# they are, and because .exe's will soon be able to have export
# tables, so we should allow linking to them.
win32_libid () {
win32_libid_type="unknown"
if echo $1 | grep -E "\.dll$|\.exe$" >/dev/null; then
win32_libid_type="x86 DLL"
else
if eval file $1 2>/dev/null | \
grep -E 'ar archive' >/dev/null; then
if eval $OBJDUMP -f $1 | head -n 10 2>/dev/null | \
grep -E 'file format pe-i386(.*architecture: i386)?' >/dev/null
; then
if eval $NM -f posix -A $1 | head -n 100 | grep " I "
>/dev/null ; then
win32_libid_type="x86 archive import"
else
win32_libid_type="x86 archive static"
fi
fi
fi
echo $win32_libid_type
}
OBJDUMP=objdump
NM=nm
win32_libid $1
##############################################
Ralf's proposal
##############################################
#!/bin/sh
# changes:
# 1) only use nm for .a files
#
win32_libid () {
win32_libid_type="unknown"
if echo $1 | grep -E "\.dll$" >/dev/null; then
win32_libid_type="x86 DLL"
else
if echo $1 | grep -E "\.dll.a$" >/dev/null; then
win32_libid_type="x86 archive import"
else
if echo $1 | grep -E "\.a$" >/dev/null; then
if eval $NM -f posix -A $1 | grep " I " >/dev/null ; then
win32_libid_type="x86 archive import"
else
win32_libid_type="x86 archive static"
fi
fi
fi
fi
echo $win32_libid_type
}
OBJDUMP=objdump
NM=nm
win32_libid $1
##############################################
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Bug reporting: http://cygwin.com/bugs.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
- Raw text -