X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:from:to:subject:date:message-id:content-type
	:content-transfer-encoding:mime-version; q=dns; s=default; b=NuA
	w5j8NcmClBpUlAZZjqfVxhWMEehphFKBKfcK5kxsxnFu9aCW2Xwyq5UTlm5gSy+P
	lfXPxsm6iEC5oSP0uFrJ3Nqqpanq+Cl+SFIzIphQigKJvGiIPPnAeR5xNhK2AFMm
	KXDw4080ENA29nClg1YRpVSmnEjvOZlHYPDVwZDY=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:from:to:subject:date:message-id:content-type
	:content-transfer-encoding:mime-version; s=default; bh=BO7QjyfJL
	/LS0jcfHcAk8PJNk3U=; b=XfNveLXAiUiVWD9WttXERl9kTmywqXxvCxV1XwSQz
	Y6dv/C4myC5x8wsjGEK+LHFEstXyjSX/n20PON73QH/dLvk1LZSwB+jXh7yDy2Z2
	fDe0YWeqpHIoFcc3QeA57RUPyl5B+ldJH256+k+rnEadvLVAKQEDnECRJJP6sfLl
	SM=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Virus-Found:	No
X-Spam-SWARE-Status:	No, score=4.9 required=5.0 tests=AWL,BAYES_50,KAM_LAZY_DOMAIN_SECURITY autolearn=no version=3.3.2
X-HELO:	mx-relay39-dus.antispameurope.com
From:	Vermessung AVT - Wolfgang Rieger <w DOT rieger AT avt DOT at>
To:	"cygwin AT cygwin DOT com" <cygwin AT cygwin DOT com>
Subject:	Re: gawk: Bad File Descriptor error with concurrent readonly access to a network file
Date:	Sat, 26 Sep 2015 10:00:39 +0000
Message-ID:	<AB495CE313664A489959F8DEF45069A90188AA0788@EXSRV01.avt-imst.local>
MIME-Version:	1.0
X-cloud-security-sender:w DOT rieger AT avt DOT at :
X-cloud-security-recipient:cygwin AT cygwin DOT com :
X-cloud-security-Virusscan:CLEAN :
X-cloud-security-disclaimer:	This E-Mail was scanned by E-Mailservice on mx-gate39-dus with E6AE12018007
X-cloud-security-connect:	46-243-105-14.cni.at[46.243.105.14], TLS=1, IP=46.243.105.14
X-cloud-security:scantime:.1583 :
X-MIME-Autoconverted:	from quoted-printable to 8bit by delorie.com id t8QA16sU005320

On Fri, 25 Sep 2015 18:58:57 +0200, Marco Atzeri wrote:
>> "Bad file descriptor" just arose recently in another problem
>> https://cygwin.com/ml/cygwin/2015-09/msg00374.html
>> https://cygwin.com/ml/cygwin/2015-09/msg00436.html
>>
I don't think this applies to our case. We use massive parallel processing, and the problem is related to that as the test case shows in our environment. In single thread operation we don't have any problems at all. I don't use fork or other of the tools mentioned. We don't have Chrome or Comodo or so installed. We have an encapsulated environment with not even an anti-virus sw running in the power workstations and as little stuff as possible because computing speed is our main issue.

>> Have you by chance some potential suspect like usual ones
>>   https://cygwin.com/faq/faq.html#faq.using.bloda
I did not find there anything that seems related to our problem.

>> On your cygcheck output I notice nothing strange.
I do not think there is anything strange. I have been using Cygwin for 15+ years now. We started parallelizing our jobs some 12 years ago. Of course, hardware was not comparable then to what we have today. But the Bad File Descriptor issues only started some 3 or 4 years ago with an update of Cygwin (I really don't remember when; there must have been some major change in the Cygwin-dll: E. g., since then the type-ahead buffer of cmd.exe is no longer useable when Cygwin programs run in the shell). Since these errors were fairly rare (say, 1 in >1000 tiles), we did not dig into it deeper. However, it is an ongoing issue.

With raising workload at the file server and new workstations with more cores (allowing for more parallel processes) it became more frequent during last years. A server upgrade last winter reduced the problem, but with recently massively increasing work load it raise again.

>> Can you provide the type of network disk with
>> /usr/lib/csih/getVolInfo <volumename>
I am sorry, I have a very small installation of Cygwin running with no getVolInfo. In which package can I find that? We have MS Windows Server 2008 that provides network shares.


Again I want to stress: Running the jobs in single thread we never experienced any such problems at all. Only with several jobs running in parallel (the same batch job is started in several cmd-shell windows independently) we have these errors. The reason is obviously when by chance two processes try to access the same file at the same time which happens not often, but it happens. I assume access to local files is better synchronized by the CPU, whereas at the server there may arise these conflicts.

The major question is, what is the underlying access problem within Cygwin? As mentioned, the MS programs (e. g. copy) never show a similar problem.
 
Thanks for your help,
Wolfgang


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

- Raw text -