X-Recipient: archive-cygwin AT delorie DOT com
DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:subject:mime-version:content-type:from
	:in-reply-to:date:cc:content-transfer-encoding:message-id
	:references:to; q=dns; s=default; b=mEDAvnjAXnghg7lXekg6fGvo7WAf
	uTnwW8VFxKESZXRYDJQaAt0F4WJUIoa0dN6FNc+WaxBtsRSNN5riwkvNwlBEFaMb
	A7LasBs+1Nen3j9x/bHlvqnwj6NETg8VoH7i5DCMwySS5AwpK7Vp2bHAclVNK1xw
	4uC8kSyA0pb2Dt4=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:subject:mime-version:content-type:from
	:in-reply-to:date:cc:content-transfer-encoding:message-id
	:references:to; s=default; bh=k76qM/ce6niyegjNPQcygZ8MWEs=; b=ge
	PtqIpHUtztArc74Tdc6t2EkjLFcn32wCWV+AbWm/f0+/a7ok4/oExc3toqMwlGcu
	NBzBTZ8nId/naYTzSi4nqvNY8mFFPAbgfFd5F1rPtJILCAHfDiUErHD0Zvy47MK5
	Zwfutg+UjZVScVT5SqQwkwRmXX0FlhtQ1mxn0u1Sc=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Authentication-Results: sourceware.org; auth=none
X-Virus-Found: No
X-Spam-SWARE-Status: No, score=-0.1 required=5.0 tests=AWL,BAYES_50,RDNS_NONE autolearn=no version=3.3.2
X-HELO: smtp6-g21.free.fr
Subject: Antivirus strikes back (probably) (Was: Intermittent failures retrieving process exit codes)
Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1822\))
Content-Type: text/plain; charset=windows-1252
From: Denis Excoffier <cygwin AT Denis-Excoffier DOT org>
In-Reply-To: <20131125215842.0ff53134@tukaani.org>
Date: Tue, 26 Nov 2013 00:11:20 +0100
Cc: Tom Honermann <thonermann AT coverity DOT com>,
        Cygwin Mailing List <cygwin AT cygwin DOT com>
Message-Id: <9E1F6369-88DF-4EDB-A752-1340ACBA7564@Denis-Excoffier.org>
References: <50C2498C DOT 2000003 AT coverity DOT com> <50C276AC DOT 9090301 AT mailme DOT ath DOT cx> <50D401EF DOT 9040705 AT coverity DOT com> <52844B2E DOT 5050902 AT coverity DOT com> <EFA04305-A94A-46F4-BCCE-8FB3ADA45E72 AT Denis-Excoffier DOT org> <20131125215842 DOT 0ff53134 AT tukaani DOT org>
To: Lasse Collin <lasse DOT collin AT tukaani DOT org>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by delorie.com id rAPNDAvG007883

On 2013-11-25 à 21:58 +02:00, Lasse Collin wrote:
> On 2013-11-15 Denis Excoffier wrote:
>> Very briefly, my problem is that when i "tar xf
>> —use-compress-program=xz", i get:
>> tar: Unexpected EOF in archive
>> tar: Unexpected EOF in archive
>> tar: Error is not recoverable: exiting now
>> and the last file of the archive is truncated at some 512bytes block.
>> This occurs on Windows 7 (not on XP); with xz-5.1.3alpha (not with
>> xz-5.1.2alpha or xz-5.0.5); never on most tar.xz files; almost always
>> on some (rare) tar.xz files (one notable example is
>> bc-1.06.95.tar.bz2 bunzip2’ed and then xz’ed); depends on the .tar
>> file itself, not on the option (like -9e, -0) used to create
>> the .tar.xz; never with "tar tf"; and with all tar’s i have tested.
>> The return code of all the involved xz -d commands is always zero
>> though. Perhaps after all, this is unrelated?
> 
> xz 5.1.3alpha has some new file I/O code that uses non-blocking file
> descriptors, the self-pipe trick, and poll(). It's there to fix a race
> condition in signal handling. Since you say it works with 5.1.2alpha, I
> wonder could there be a bug with the new I/O code in xz or if the code
> in xz triggers a bug in Cygwin or Windows.
> 
> If you haven't already tried, please compile both 5.1.2alpha and
> 5.1.3alpha from source while keeping everything else unchanged, and see
> if the bug really only occurs with 5.1.3alpha.
Already done. I did some strace-ing, and since i’m not so fluent with the
result, i’ll send it there in a while (when i’m back on cygwin) if someone is
interested. But the bug (contrary to what i said before) also _sometimes_
occurs with 5.1.2alpha or 5.0.5 and this makes me think now that:

a) my antivirus-anti-intrusion-whatever-software (that i can’t remove of
course) creates some kind of "background noise" where a certain percentage
of such ‘tar xf —use-compress-program’ commands will always fail

b) nevertheless, xz-5.1.3alpha (with its new file I/O code etc.) triggers some
untypical configuration inside the antivirus that increases drastically the
percentage, making the failure almost certain for some files.

It is not extraordinary that i cannot observe the failure on XP since
i do not have this particular antivirus on XP.

You might however want some more detail. Test plan is: perform
'tar xf file.xz --use-compress-program=xz -bx', where x varies from 1 to 200.
There are two kinds of results:

1) usual situation is where you observe max 1 or 2 failures (on a maximum of 200).
If you launch the same plan, you still report max 1 or 2 failures, usually not
with the same numbers. Very often you have no failure at all. Very often the
-b20 (the default) does not fail.
-> this situation occurs with 5.1.2alpha or 5.0.5 with all input files, or with
5.1.3alpha with most input files.

2) pathological situation is where you observe, say, 30 failures (on a maximum of 200).
If you launch the same plan, you report nearly the same failures, ie mostly the same
ones, with some minor variability analogous to the variability observed in the usual
situation above
-> this situation occurs with 5.1.3alpha only, with some selected input files,
eg bc-1.06.95.tar.xz (see above how to create bc-1.06.95.tar.xz)

When it fails (usually or pathologically), the last file of the archive gets
truncated (see above), and _this_ is strange from an antivirus behaviour. After
all, perhaps some flush() or similar is missing inside 5.1.3alpha.

Denis Excoffier.
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple