X-Recipient:	archive-cygwin AT delorie DOT com
DomainKey-Signature:	a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:content-type:content-transfer-encoding;
	q=dns; s=default; b=VUkY6Dt9Ewvp732/bq7j8Bi6/MI1oWQ33ZbPZL1a4eu
	w518wSliC3m6AhenrxP+NsoEqSJaKTHp4noMFS1wWKZ7nTreCuA14Ivexr3VvAks
	PthXSvlFlwP79o2I5VhxeZCwsUy5kSIv43fgeae3jgx+cS6Yq64gDxIsA4cS3v2w
	=
DKIM-Signature:	v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:content-type:content-transfer-encoding;
	s=default; bh=rG9LQBTcVAEz0PiXVldscyCGyok=; b=OC/6k5poHyPy7t/t/
	oxkSoDt19gGfcI+oIvwVIQrmax1pWbbfnGtdMvqay1H63PmmM0+UWXKS9YL0WZ8x
	76HycWE+yfw8AzFV+mjuRepj6Qj7qy8bFcERxEfE6pTrQzVDlX7AzZhGBQn8yJn1
	Ysq0cDF4gyXYsV/S3ZmYMR23SE=
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com
Authentication-Results:	sourceware.org; auth=none
X-Virus-Found:	No
X-Spam-SWARE-Status:	No, score=-1.1 required=5.0 tests=BAYES_00,FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,RCVD_IN_SORBS_SPAM,SPF_PASS autolearn=no version=3.3.2 spammy=Stopped, PCs, base-cygwin, basecygwin
X-HELO:	mail-ot0-f172.google.com
X-Google-DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-transfer-encoding; bh=Ghdo+kTu/GSzDwmGnFDRBNccHAdAM5uNQjV5Xa5cW7Q=; b=anG9X/MGboH3sufRJygzkJHOGrP0u73kSou3b08rexlyRYqV6WjWrBu6tWE+HHxyFX AgqEkdkSa1XBOKr+KfkM/9LaXeTLT18rYQffhsfcpEdbjR5a1QD7+IRQLJF8OAolQPII /+bynGFfWtaoYvdKKXve4eGMsPVP9FzkVtr8ed1OAhZjmtcBTMhIExVbW3EA+Ln5TSDX VMTzWXShL432Kvh67HgmlvfCvsOzTjrV0Y/NiAMrz8lyqyLL7nCozbm9gkzA/CopC0r+ rifqRGz3k+veMI8Y3skcbCNZkvbvTuUhAv1CSIMaUG+rA1OG4NDREiCe9hjaMXcdiHWf uYsA==
X-Gm-Message-State:	AIkVDXKQOnIMhJ58dCYOWsr9xn0zjbcdwMpASYF26/shxmWrrPTMxQHa6hk73xpgEOiHPYhtod2E7zlhJ4Jv7A==
X-Received:	by 10.157.49.38 with SMTP id e35mr10021211otc.196.1485789829315; Mon, 30 Jan 2017 07:23:49 -0800 (PST)
MIME-Version:	1.0
In-Reply-To:	<CAPJ9Yc8SQ_btEtyrZhqR22W4AUY1=xveQnM1_A0ENdyUUEgP=A@mail.gmail.com>
References:	<CAPJ9Yc8SQ_btEtyrZhqR22W4AUY1=xveQnM1_A0ENdyUUEgP=A AT mail DOT gmail DOT com>
From:	=?UTF-8?Q?David_Bala=C5=BEic?= <xerces9 AT gmail DOT com>
Date:	Mon, 30 Jan 2017 16:23:48 +0100
Message-ID:	<CAPJ9Yc9rzY4nZ+5tTU32EjPvPs4Ct5BQj_bvJnNg7pVubiMGEg@mail.gmail.com>
Subject:	Re: Bug in lrzip 0.631-1 (32 bit version) with -d -o - options
To:	cygwin AT cygwin DOT com
X-IsSubscribed:	yes
X-MIME-Autoconverted:	from quoted-printable to 8bit by delorie.com id v0UFO7M9004836

I tried in Ubuntu 32 bit (both the packaged lrzip  and a self compiled
one) and there the problem does not happen, so it looks like either:
 - bad lrzip in cygwin
 - cygwin pipe issues?

Regards,
David


On 25 January 2017 at 23:15, David Balažic <xerces9 AT gmail DOT com> wrote:
> Hi!
>
> The 32 bit version of lrzip 0.631-1 contains a bug that corrupts the
> decompressed dat in some circumstances.
>
> I reproduced the problem on 2 PCs (the md5sum of the broken output was
> the same on both systems).
>
> I seems to happen when the (de)compressed file size is bigger than the
> available RAM (note that the 32 bit version uses max 4GB in any case)
> and lrzip resorts to using a temporary file.
>
> See below for reproducing:
>
> $ lrzip -i sda.img.lrz2
> sda.img.lrz2:
> lrzip version: 0.6 file
> Compression: rzip + lzma
> Decompressed file size: 64017212928
> Compressed file size: 7210541304
> Compression ratio: 8.878
> MD5 used for integrity testing
> MD5: 6594f5b0d22efd345003260054165842
>
> $ date; df -h ; TMP=/cygdrive/i/t/tmp/  lrzip -v  -d  -o -
> sda.img.lrz2  | tee >(md5sum --tag) >(sha1sum --tag) > /dev/null   ;
> date
> Tue Jan 24 21:29:01 CET 2017
> Filesystem      Size  Used Avail Use% Mounted on
> C:/cygwin       114G   94G   21G  83% /
> D:              541G  534G  7.1G  99% /cygdrive/d
> I:              391G  279G  113G  72% /cygdrive/i
> Q:               60G   57G  2.8G  96% /cygdrive/q
> The following options are in effect for this DECOMPRESSION.
> Threading is ENABLED. Number of CPUs detected: 4
> Detected 17160601600 bytes ram
> Compression level 7
> Nice Value: 19
> Show Progress
> Verbose
> Output Filename Specified: -
> Temporary Directory set as: /cygdrive/i/t/tmp/
> Outputting to stdout.
> Detected lrzip version 0.6 file.
> MD5 being used for integrity testing.
> Decompressing...
> Unable to decompress entirely in ram, will use physical files
> Dumping temporary file to control->outFILE.
>
> [1]+  Stopped                 TMP=/cygdrive/i/t/tmp/ lrzip -v -d -o -
> sda.img.lrz2 | tee >(md5sum --tag) >(sha1sum --tag) > /dev/null
> Tue Jan 24 21:31:39 CET 2017
>
> stein AT hofer8 /cygdrive/i/Zotac_bak
> $ fg
> TMP=/cygdrive/i/t/tmp/ lrzip -v -d -o - sda.img.lrz2 | tee >(md5sum
> --tag) >(sha1sum --tag) > /dev/null
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
> Dumping temporary file to control->outFILE.
>
> Average DeCompression Speed:  0.668MB/s
> Dumping temporary file to control->outFILE.
> [OK] - 64017212928 bytes
> Total time: 25:22:26.25
> SHA1 (-) = 6c519210541eb128c03b7c0f803adb2b46ee2a72
> MD5 (-) = 8bd6ad48f2cea6a710af70b434d57673
>
>
> The correct md5sum is 6594f5b0d22efd345003260054165842.
>
>
> Simply decompressing the file (lrzip -d -o sda.img sda.img.lrz2) to
> filesystem works fine, only when piped to stdout the problem happens.
>
> The 64 bit version does not have this problem.
>
>
> I will check if the same problem happens with the native linux build
> of lrzip (it takes a day...).
>
>
> I tried to reproduce the problem with a smaller file, but there it did
> not happen. Maybe my first test file has some corruption that causes
> this (unlikely).
>
> Some version information (complete cygcheck -s -v -r output attached):
>
> base-cygwin                           3.8-1
> cygwin                                2.6.1-1
> lrzip                                 0.631-1
>
> Regards,
> David

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

- Raw text -