X-Recipient: archive-cygwin AT delorie DOT com
DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:content-type:content-transfer-encoding;
	 q=dns; s=default; b=J2/G4JPnuDCzYvMVdmGkDyUE1tIF/PONrBXdJ/RvjwW
	KMWyFrCS2DfvFRnzWTQTDczWlCiEhb5moTAlG9Gr8eSqTvnL4o8jPMmCnvr1DHYh
	JVbZNf2Dsk93xT4dBjpSH62wP1ptywgnWSiek1zRA6sH1NWiLPEDRqFApmbvM4pA
	=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:content-type:content-transfer-encoding;
	 s=default; bh=t4xzkY1bHsMasC+04WjFE6euKjY=; b=Wf20rhmv68YB69SOE
	R6tDdh2MzETyiSEWS8IQ3VRDoSvPCTfNjIuLY7xXcfDV640JK8GQa8LYO7rsOf3k
	CXIENJB3/tgNWKgz+X8VXVfR6Pw+69QJcxySja9cFoRP64FYGexNaJXvmtqMcqsN
	1hBqRoUnrsuXgwDR45SWm4QRAI=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Authentication-Results: sourceware.org; auth=none
X-Virus-Found: No
X-Spam-SWARE-Status: No, score=-1.1 required=5.0 tests=BAYES_00,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,RCVD_IN_SORBS_SPAM,SPF_PASS autolearn=no version=3.3.2 spammy=sum
X-HELO: mail-ot0-f175.google.com
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;        d=1e100.net; s=20161025;        h=x-gm-message-state:mime-version:in-reply-to:references:from:date         :message-id:subject:to:content-transfer-encoding;        bh=Ky1QZQfYVtsigqqjYPCVz28oAR8xKHpXG4okxyCDZ4c=;        b=D3Txh4z2JIa7F5kgGQ9kqxgr3vp/ZQGt8Ptpy2ewBRPsK0jgbET/R7XcG51yqCDfvN         vJNnxQ8HpHeY8jCMTIE3iCSQbSlnQ2TChuA6RnWHPf2mSE2uDG1hQEhvf7u9RiBjrenN         V6ZwDvGuZecyK1kvOY7tg2E5dBU0nva3Ut20ifUC4VwCYfNHULPHrc/GyxYFn3pgOBdQ         ZRos39GmdQe/vGduuCUrIBHAn3Uv9m2NRMtPYJgKPi2+T0OmLwkViTAgz6xoGUJ9Ix5S         3YDLRFRPftLRosRG7oE+IpnedoxQJ2/FZpoEfcRbcsK8Gi/lICLsljZbzPJqj1gdOExW         jlYA==
X-Gm-Message-State: AMke39moNWVTvCORYUFPjltnGXiVLMxCi7XrLkJEi3TGvyjmNuLq1HC9d8LQtXwPAE0MEe3kZJq2PG3OWyIbWw==
X-Received: by 10.157.0.102 with SMTP id 93mr2666754ota.141.1485984348222; Wed, 01 Feb 2017 13:25:48 -0800 (PST)
MIME-Version: 1.0
In-Reply-To: <CAPJ9Yc9rzY4nZ+5tTU32EjPvPs4Ct5BQj_bvJnNg7pVubiMGEg@mail.gmail.com>
References: <CAPJ9Yc8SQ_btEtyrZhqR22W4AUY1=xveQnM1_A0ENdyUUEgP=A AT mail DOT gmail DOT com> <CAPJ9Yc9rzY4nZ+5tTU32EjPvPs4Ct5BQj_bvJnNg7pVubiMGEg AT mail DOT gmail DOT com>
From: =?UTF-8?Q?David_Bala=C5=BEic?= <xerces9 AT gmail DOT com>
Date: Wed, 1 Feb 2017 22:25:47 +0100
Message-ID: <CAPJ9Yc-dxGfahRH-zYiHb+A5LRKuzTUFNKFzuk-9a3Go0705Dw@mail.gmail.com>
Subject: Re: Bug in lrzip 0.631-1 (32 bit version) with -d -o - options
To: cygwin AT cygwin DOT com
Content-Type: text/plain; charset=UTF-8
X-IsSubscribed: yes
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by delorie.com id v11LQCs4032102

I tried the  2.7.0-0.1  test release and now the behavior changed.

Before I got consistently a wrong MD5 sum of 8bd6ad48f2cea6a710af70b434d57673
With this release I get c43a02c309fa5e0abe778201e9ceec46.

So something changed.

Either the problem is in cygwin or lrzip, I guess.

Regards,
David


On 30 January 2017 at 16:23, David Balažic <xerces9 AT gmail DOT com> wrote:
> I tried in Ubuntu 32 bit (both the packaged lrzip  and a self compiled
> one) and there the problem does not happen, so it looks like either:
>  - bad lrzip in cygwin
>  - cygwin pipe issues?
>
> Regards,
> David
>
>
> On 25 January 2017 at 23:15, David Balažic <xerces9 AT gmail DOT com> wrote:
>> Hi!
>>
>> The 32 bit version of lrzip 0.631-1 contains a bug that corrupts the
>> decompressed dat in some circumstances.
>>
>> I reproduced the problem on 2 PCs (the md5sum of the broken output was
>> the same on both systems).
>>
>> I seems to happen when the (de)compressed file size is bigger than the
>> available RAM (note that the 32 bit version uses max 4GB in any case)
>> and lrzip resorts to using a temporary file.
>>
>> See below for reproducing:
>>
>> $ lrzip -i sda.img.lrz2
>> sda.img.lrz2:
>> lrzip version: 0.6 file
>> Compression: rzip + lzma
>> Decompressed file size: 64017212928
>> Compressed file size: 7210541304
>> Compression ratio: 8.878
>> MD5 used for integrity testing
>> MD5: 6594f5b0d22efd345003260054165842
>>
>> $ date; df -h ; TMP=/cygdrive/i/t/tmp/  lrzip -v  -d  -o -
>> sda.img.lrz2  | tee >(md5sum --tag) >(sha1sum --tag) > /dev/null   ;
>> date
>> Tue Jan 24 21:29:01 CET 2017
>> Filesystem      Size  Used Avail Use% Mounted on
>> C:/cygwin       114G   94G   21G  83% /
>> D:              541G  534G  7.1G  99% /cygdrive/d
>> I:              391G  279G  113G  72% /cygdrive/i
>> Q:               60G   57G  2.8G  96% /cygdrive/q
>> The following options are in effect for this DECOMPRESSION.
>> Threading is ENABLED. Number of CPUs detected: 4
>> Detected 17160601600 bytes ram
>> Compression level 7
>> Nice Value: 19
>> Show Progress
>> Verbose
>> Output Filename Specified: -
>> Temporary Directory set as: /cygdrive/i/t/tmp/
>> Outputting to stdout.
>> Detected lrzip version 0.6 file.
>> MD5 being used for integrity testing.
>> Decompressing...
>> Unable to decompress entirely in ram, will use physical files
>> Dumping temporary file to control->outFILE.
>>
>> [1]+  Stopped                 TMP=/cygdrive/i/t/tmp/ lrzip -v -d -o -
>> sda.img.lrz2 | tee >(md5sum --tag) >(sha1sum --tag) > /dev/null
>> Tue Jan 24 21:31:39 CET 2017
>>
>> stein AT hofer8 /cygdrive/i/Zotac_bak
>> $ fg
>> TMP=/cygdrive/i/t/tmp/ lrzip -v -d -o - sda.img.lrz2 | tee >(md5sum
>> --tag) >(sha1sum --tag) > /dev/null
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>> Dumping temporary file to control->outFILE.
>>
>> Average DeCompression Speed:  0.668MB/s
>> Dumping temporary file to control->outFILE.
>> [OK] - 64017212928 bytes
>> Total time: 25:22:26.25
>> SHA1 (-) = 6c519210541eb128c03b7c0f803adb2b46ee2a72
>> MD5 (-) = 8bd6ad48f2cea6a710af70b434d57673
>>
>>
>> The correct md5sum is 6594f5b0d22efd345003260054165842.
>>
>>
>> Simply decompressing the file (lrzip -d -o sda.img sda.img.lrz2) to
>> filesystem works fine, only when piped to stdout the problem happens.
>>
>> The 64 bit version does not have this problem.
>>
>>
>> I will check if the same problem happens with the native linux build
>> of lrzip (it takes a day...).
>>
>>
>> I tried to reproduce the problem with a smaller file, but there it did
>> not happen. Maybe my first test file has some corruption that causes
>> this (unlikely).
>>
>> Some version information (complete cygcheck -s -v -r output attached):
>>
>> base-cygwin                           3.8-1
>> cygwin                                2.6.1-1
>> lrzip                                 0.631-1
>>
>> Regards,
>> David

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple