Subject: Possible race condition bug in pthread_cond_broadcast/pthread_con d_wait?
From: Jonathan Wackley <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Date: Mon, 9 Jun 2003 13:11:43 -0400

Hi,

Created small test program showing possible implementation options for
barriers.  This test program works on a number of platforms and machines.  I
found that it does not work as expected under cygwin, while it does work
using mingw.  This implementation is improved upon in later versions and is
intended to show how development could progress through iterations.  Please
not that cygcheck.out is attached to this e-mail.

Here is the source code;

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <sys/types.h>
#include <semaphore.h>
#include <pthread.h>

#define NUMBER_OF_THREADS (3)
#define COUNTER_MAX (10)

pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t barrier_condition = PTHREAD_COND_INITIALIZER;
sem_t barrier_semaphore;

void *
pthread_func( void * startup_argument ) {

	static int thread_counter = 0;
	int counter = 0;
	long thread_id = ( long ) startup_argument;
	long dummy = 0;

	pthread_mutex_lock( &mutex );
	printf( "Started (%d)\n", thread_id );
	if ( ++thread_counter == NUMBER_OF_THREADS ) {
		printf( "rendezvous\n" );
		sem_post( &barrier_semaphore );
	}
	pthread_cond_wait( &barrier_condition, &mutex );
	pthread_mutex_unlock( &mutex );

	for ( counter = 0; counter < COUNTER_MAX; counter++ ) {
		dummy = counter * counter;
		pthread_mutex_lock( &mutex );
		printf( "Got here (%ld): %d %ld\n", thread_id, counter,
dummy );
		pthread_mutex_unlock( &mutex );
	}

	pthread_mutex_lock( &mutex );
	if ( --thread_counter == 0 ) {
		printf( "Posting for last thread\n" );
		sem_post( &barrier_semaphore );
	}
	pthread_mutex_unlock( &mutex );

	return 0;
}

int
main( int argc, char * argv[ ] ) {

	int status = 0;
	long counter = 0;

	void * join_status = 0;

	pthread_attr_t thread_attributes;
	pthread_t thread_id[ NUMBER_OF_THREADS ];

	status = sem_init( &barrier_semaphore, 0, 0 );

	if ( status != 0 ) {
		printf( "Semaphore failed: %s\n", strerror( status ) );
		exit( 0 );
	}

	pthread_attr_init( &thread_attributes );
	pthread_attr_setscope( &thread_attributes, PTHREAD_SCOPE_SYSTEM );

	for ( counter = 0; counter < NUMBER_OF_THREADS; counter++ ) {
		status = pthread_create( thread_id + counter,
&thread_attributes, pthread_func, ( void * ) counter );
		if ( status != 0 ) {
			printf( "Thread create failed: %s\n", strerror(
status ) );
			exit( 0 );
		}
		pthread_mutex_lock( &mutex );
		printf( "Thread creation status(%d): %d\n", counter, status
);
		pthread_mutex_unlock( &mutex );
	}

	sem_wait( &barrier_semaphore );
	pthread_mutex_lock( &mutex );
	printf( "About to broadcast to threads\n" );
	pthread_cond_broadcast( &barrier_condition );
	pthread_mutex_unlock( &mutex );
	sem_wait( &barrier_semaphore );

	for ( counter = 0; counter < NUMBER_OF_THREADS; counter++ ) {
		status = pthread_join( thread_id[ counter ], &join_status );
		pthread_mutex_lock( &mutex );
		printf( "Thread join status(%d): %d\n", counter, status );
		pthread_mutex_unlock( &mutex );
	}

	return 0;

}

cygwin compile command line: gcc -D_POSIX_C_SOURCE=199506L -o pthread_test
pthread_test.c -lpthread
mingw compile command line: $ gcc -mno-cygwin -D_POSIX_C_SOURCE=199506L -o
pthread_test pthread_test.c -lpthreadGC

mingw output;

$ ./pthread_test
Thread creation status(0): 0
Thread creation status(1): 0
Thread creation status(2): 0
Started (0)
Started (1)
Started (2)
rendezvous
About to broadcast to threads
Got here (0): 0 0
Got here (1): 0 0
Got here (0): 1 1
Got here (1): 1 1
Got here (0): 2 4
Got here (1): 2 4
Got here (0): 3 9
Got here (1): 3 9
Got here (0): 4 16
Got here (1): 4 16
Got here (0): 5 25
Got here (1): 5 25
Got here (0): 6 36
Got here (1): 6 36
Got here (0): 7 49
Got here (1): 7 49
Got here (0): 8 64
Got here (1): 8 64
Got here (0): 9 81
Got here (1): 9 81
Got here (2): 0 0
Got here (2): 1 1
Got here (2): 2 4
Got here (2): 3 9
Got here (2): 4 16
Got here (2): 5 25
Got here (2): 6 36
Got here (2): 7 49
Got here (2): 8 64
Got here (2): 9 81
Posting for last thread
Thread join status(0): 0
Thread join status(1): 0
Thread join status(2): 0

Output from cygwin;

$ ./pthread_test
Thread creation status(0): 0
Thread creation status(1): 0
Thread creation status(2): 0
Started (0)
Started (1)
Started (2)
rendezvous
About to broadcast to threads
Got here (0): 0 0
Got here (0): 1 1
Got here (0): 2 4
Got here (0): 3 9
Got here (0): 4 16
Got here (0): 5 25
Got here (0): 6 36
Got here (0): 7 49
Got here (0): 8 64
Got here (0): 9 81
Got here (1): 0 0
Got here (1): 1 1
Got here (1): 2 4
Got here (1): 3 9
Got here (1): 4 16
Got here (1): 5 25
Got here (1): 6 36
Got here (1): 7 49
Got here (1): 8 64
Got here (1): 9 81

Where it hangs, last thread does not start.  I modified the program to sleep
for a second and re-broadcast on the condition variable where it then works.
Here is the modification;

        sem_wait( &barrier_semaphore );
        pthread_mutex_lock( &mutex );
        printf( "About to broadcast to threads\n" );
        pthread_cond_broadcast( &barrier_condition );
        pthread_mutex_unlock( &mutex );
        sleep( 1 );
        pthread_mutex_lock( &mutex );
        printf( "About to broadcast to threads (again)\n" );
        pthread_cond_broadcast( &barrier_condition );
        pthread_mutex_unlock( &mutex );
        sem_wait( &barrier_semaphore );

And here is the output;

$ ./pthread_test
Thread creation status(0): 0
Thread creation status(1): 0
Thread creation status(2): 0
Started (0)
Started (1)
Started (2)
rendezvous
About to broadcast to threads
Got here (0): 0 0
Got here (0): 1 1
Got here (0): 2 4
Got here (0): 3 9
Got here (0): 4 16
Got here (0): 5 25
Got here (0): 6 36
Got here (0): 7 49
Got here (0): 8 64
Got here (0): 9 81
Got here (1): 0 0
Got here (1): 1 1
Got here (1): 2 4
Got here (1): 3 9
Got here (1): 4 16
Got here (1): 5 25
Got here (1): 6 36
Got here (1): 7 49
Got here (1): 8 64
Got here (1): 9 81
About to broadcast to threads (again)
Got here (2): 0 0
Got here (2): 1 1
Got here (2): 2 4
Got here (2): 3 9
Got here (2): 4 16
Got here (2): 5 25
Got here (2): 6 36
Got here (2): 7 49
Got here (2): 8 64
Got here (2): 9 81
Posting for last thread
Thread join status(0): 0
Thread join status(1): 0
Thread join status(2): 0

As I mentioned earlier there are much better ways to implement a barrier.
From what I can see this code should work, though it may not be elegant.  As
a demonstration it works on AIX, HP, Tru64, Win32 (pthread-win32), mingw
(with pthread-win32) Linux and Mac OS/X.  It is somewhat surprising that it
does not work as expected on cygwin.  It may be a silly oversight on my
side, I just can't find what it may be.  If it matters, I am not on the
mailing list so please send comments/directions directly to me.

Thanks,
jonw

 <<cygcheck.out>> 
--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/