Subject: Re: Is multithreaded profiling on cygwin possible?
From: "peter garrone" <[email-masked]>
To: "Brian Ford" <[email-masked]>
Cc: [email-masked]
Date: Wed, 15 Oct 2003 17:57:31 +0800

Brian Ford wrote:
>
>On Tue, 14 Oct 2003, peter garrone wrote:
>
>> A list of active threads is maintained. A thread calling moncontrol(1) gets
>> put in the list. When a call to SuspendThread fails, the thread is assumed
>> to be defunct and taken off the list.
>>
>Seems reasonable.
>
>I guess I was originally thinking "just profile all threads all the
>time", but I guess you method is more flexable.  That could
>probably be an option somehow after you polish this up.

There is no other way to detect a dead thread. Nothing like "atexit" or
anything I could find anyway in the win32 api, of which my experience
is limited, and ordinary.

>
>> One of the fields in the thread is a counter corresponding to the sum of cpu
>> returned by GetThreadTimes. This function has fields corresponding to
>> kernel cpu and user cpu. The amount of time consumed by every thread is
>> saved.
>>
>> Generally only one thread will have consumed CPU. However to be general,
>> and in case the profiling thread is inadvertently delayed, all threads are
>> considered.
>>
>> There is a partial tick problem. Suppose that a thread has consumed say
>> 155% of the cpu time corresponding to a tick. I would assign one tick
>> and use a local random number generator to assign an extra tick on
>> average 55% of the time.
>>
>I'm getting lost here.
>
>What is your tick definition?  A sampling interval?

Yes.
The amount of CPU usage corresponging to a loop of the profiling sampler.
The amount represented by incrementing the sample data by one.
The amount of time represented by (1.0/PROF_HZ)

>
>How can a thread ever consume more that 100%?  I can see how two or more
>threads might on a multi CPU system.

The profiling thread calls Sleep. That function is guaranteed to sleep for
at least its argument. It can always sleep longer. So the actual profiling interval
could be longer than the nominal profiling interval. In the meantime,
a thread could have consumed more than one interval worth of cpu.

>
>I'm lost in the random number generator application too.
>

1) The rng algorithm itself

Its a linear congruential rng algorithm.
Given a 31 bit seed, do a 64 bit multiply by 69069,
add 1, and mask off the lower 31 bits. 
I dont think the application calls for anything special.
If it bothers you, call rand(); 
I dont have references for this, but I'm preety sure it has been
academically described and extensively tested. As an RNG,
it does have severe limitations but it is very fast.

2) Why use one? 

What else do you do if only a fraction of a tick
is used. If a thread has consumed say 70% of a complete ticks worth of cpu,
then roll the dice and 70% of the time assign a tick. I dont think I can
explain it better. It all averages out in the end, and its basically an
estimate anyway.

>> I tried getting the program counter for all threads, but this was found
>> not to work very well, consuming excessive cpu, on average 50 milliseconds.
>>
>I thought there might be an overhead issue.

Definitely. This was causing the scheduler to assign very long profile thread sleeptimes,
much longer than nominal,
and is the reason for all the probabilistic stuff. That scheduler is a bit sus.

>Please do (come back with your results, that is).  I'm definately
>interested.
>

Concerning the DLL import function profile assignment solution,
I can get it to work, but only for toy situations unfortunately.
If I create a simple test DLL, then it apparently works OK, giving about 70 nanoseconds per
DLL function call, 7 seconds for 100 million, compared with
about 20 nanoseconds for a simple unoptimised profiled function call,
and some nice looking gprof output indeed.

However when I try things like the cygwin dll or kernel32,
I just get a segmentation violation on startup. I think there are some issues with the import
library format which I dont understand. I have changed the little bit of code found in
dlltool.c that goes in each import library function to cause a call to the profiler code,
which is always at a fixed address since I cant get ld to accept an extra relocation.
Also I might have some assembler issues. I am currently trashing eax and edx in the profiler
call, which C functions appear to do.

My cygwin installation is behaving rather oddly at the moment so it could be something to do with that.
I have a snapshot, and I have to have my own version of bash, compiled with that
snapshot unfortunately. gdb always has an application initialization
failure on startup. Setup hangs at the last. I think I'll try a new snapshot.

Any suggestions on this, by anyone at all, would be greatly appreciated.
 Cheers,
  Peter

-- 
______________________________________________
Check out the latest SMS services @ http://www.linuxmail.org 
This allows you to send and receive SMS through your mailbox.

Powered by Outblaze

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/