Subject: RE: Problem to open big selfextracting Zip files from bash
From: Barry Smith at SourceLink <[email-masked]>
To: <[email-masked]>
Cc: "'Dirk Napierala'" <[email-masked]>, "'ZELL,VOLKER'" <[email-masked]>
Date: Mon, 22 Sep 2008 09:32:14 -0400
List-Id: <cygwin.cygwin.com>

Dirk --

Fortunately, the issue with zip corruption (in dealing with files over 1
gig) 
is well documented in multiple web locations with test results.  I don't
know 
the code for Zip, so I can't speak to exactly where in the source the
corruption 
occurs, but based upon my knowledge of compression tools, I would assume
that
the checksum value in the Zip standard isn't large enough. RAR and WinRAR 
(other compression tools based upon Tar) both completely support the ZIP
spec.  
The point that I'm trying to make is. "It's not the Zip tool, it's the Zip 
specification ." 

The RAR spec is a lot more stable for large archives, as is the TAR spec.

Just a note on what advantages the RAR format has :
- ZIP uses LZW compression (with perhaps some enhancements). RAR goes one 
step further using pattern compression, ideal for multimedia data which 
already uses some form of compression.
- RAR has easy-to-use segmenting options
- RAR incoporates some error checking which will flag an archive as bad if 
there's even a bit that's out of place.
- RAR also allows a recovery record which can be used to retrieve files even

when an archive is damage.
- RAR offers solid archiving (files encoded in one go) which gains a bit of 
space. The downside to this is that the whole archive needs to be
decompressed 
to extract just one file.
- RAR archives have better passwording although I'm sure that there are 
utilities that can be used to crack them.
- RAR Authenticity verification is a very handy feature that allows you to
check 
where an archive came from.
- RAR can conserve original file dates.
- RAR takes into account NTFS file permissions.
- RAR allows you to store paths and reproduce them on a destination machine.

In short, RAR is more secure and more evolved compression standard.

Found the following testing results (WinAce vs WinRAR vs WinZip) at
SoftPedia...
http://news.softpedia.com/news/WinAce-VS-WinRAR-VS-WinZip-17365.shtml
Scroll down to the Details section at the bottom of the article --

   For those of you who would still like to know some details about these 
   tests here are some important facts: 

   The testing was done using two computers, one with a 3200 AMD64
processor, 
   the other one with a 1500 Pentium processor. The results mentioned above 
   were from a tiff file, a mov file and a wav file.

   WinAce compressed the video file in 17-18 seconds to 188KB. To compress 
   the audio file to 46.6MB it took 48 seconds at maximum compression and 
   24 seconds at normal compression. The image file was compressed at 
   47.9MB in 44 seconds at normal compression and 47 seconds at maximum 
   compression. Please notice the fact that the although compressed to 
   the maximum, the file size did not change significantly.

   WinRAR compressed the mov file to 458KB in 17 seconds at maximum
compression 
   and to 399KB in 14 seconds at normal compression. The wav was archived in

   18-19 seconds to 44.9MB at maximum compression and 45.9 at normal
compression. 
   The tiff file was compressed in 47 seconds to 47.6MB.

   In 6 seconds, WinZip archived the same video file to 2.05MB at maximum 
   compression and 5.59MB at normal compression in 3 seconds. Using bzip2 
   compression the audio file was shrunk to 59.2MB in 30 seconds for maximum

   compression and to 61.7MB in 9 seconds using normal compression. As for 
   the image file, using again bzip2 compression, it was archived to 47.7MB 
   in 27 seconds at maximum compression and in 7 seconds using normal 
   compression. 

Personally, both for personal AND business use, I switched over to RAR (and 
WinRar) exclusively over three years ago.

RAR (and WinRAR) both create self-extracting archives.

Also, what Zip calls "spanning" (or compressing to multiple volumes) is 
perfectly supported in Rar.

If you don't want to give up the Zip ghost (and presuming that the SE-Zip
can 
be re-created from the original files), try zip-spanning.  If I remember 
correctly, the first file will be an exe, and the other parts will be zip
files.

Best of luck.

Barry Smith
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Dirk Napierala
Sent: Friday, September 19, 2008 9:59 AM
To: [email-masked]
Cc: EVERS,DIRK; ZELL,VOLKER
Subject: Problem to open big selfextracting Zip files from bash

Hi there,

We discovered an issue  trying to open big selfextracting zip files.
Trying to do so result in the following error:

./*selfextracting_zipfile*.exe
bash: ./*selfextracting_zipfile*.exe: Cannot allocate memory

This only happens to huge files (guess above 1.5GB size) Smaller once are
working fine.

This issue happens on the current version available for download 1.5.25-15
and also with the new 1.7.0 (base-files version 3.7-1)(thanks to Volker Zell
providing us this one for testing)

If we are using an older version instead (1.5.18 base-files version
3.6-1) this issue doesn't show up.
We tried the same selfextracting zip file on each of the above mentioned
versions on a Dell Optiplex GX280 with 2GB memory.

After discussing this issue with Volker Zell he mentioned to send this
incident to your mailing list.
Hopefully someone can assist us with that and provide a fix.

Best Regard Dirk Napierala

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/