From: "Juan Manuel Guerrero" Organization: Darmstadt University of Technology To: eliz AT is DOT elta DOT co DOT il Date: Mon, 4 Dec 2000 13:01:37 +0200 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: Re: bzip2 support for DJTAR CC: djgpp-workers AT delorie DOT com In-reply-to: <2950-Sat25Nov2000131328+0200-eliz@is.elta.co.il> X-mailer: Pegasus Mail for Windows (v2.54DE) Message-ID: <73466C5D23@HRZ1.hrz.tu-darmstadt.de> Reply-To: djgpp-workers AT delorie DOT com Errors-To: nobody AT delorie DOT com X-Mailing-List: djgpp-workers AT delorie DOT com X-Unsubscribes-To: listserv AT delorie DOT com Precedence: bulk Patch for bzip support for DJTAR. Patch 4/4 diff -acprNC5 djgpp.orig/src/utils/djtar/bzlib.c djgpp/src/utils/djtar/bzlib.c *** djgpp.orig/src/utils/djtar/bzlib.c Thu Jan 1 00:00:00 1970 --- djgpp/src/utils/djtar/bzlib.c Sun Dec 3 00:57:30 2000 *************** *** 0 **** --- 1,512 ---- + /* Copyright (C) 2000 DJ Delorie, see COPYING.DJ for details */ + + /*-------------------------------------------------------------*/ + /*--- Library top-level functions. ---*/ + /*--- bzlib.c ---*/ + /*-------------------------------------------------------------*/ + + /*-- + This file is a part of bzip2 and/or libbzip2, a program and + library for lossless, block-sorting data compression. + + Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + + 3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. + + 4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + Julian Seward, Cambridge, UK. + jseward AT acm DOT org + bzip2/libbzip2 version 1.0 of 21 March 2000 + + This program is based on (at least) the work of: + Mike Burrows + David Wheeler + Peter Fenwick + Alistair Moffat + Radford Neal + Ian H. Witten + Robert Sedgewick + Jon L. Bentley + + For more information on these sources, see the manual. + --*/ + + /*-- + CHANGES + ~~~~~~~ + 0.9.0 -- original version. + + 0.9.0a/b -- no changes in this file. + + 0.9.0c + * made zero-length BZ_FLUSH work correctly in bzCompress(). + * fixed bzWrite/bzRead to ignore zero-length requests. + * fixed bzread to correctly handle read requests after EOF. + * wrong parameter order in call to bzDecompressInit in + bzBuffToBuffDecompress. Fixed. + + Minor changes for DJTAR program, Juan Manuel Guerrero December 2000 + --*/ + + #include "bzlib_private.h" + + + /*---------------------------------------------------*/ + #ifndef BZ_NO_STDIO + void BZ2_bz__AssertH__fail ( int errcode ) + { + fprintf(stderr, + "\n\nbzip2/libbzip2: internal error number %d.\n" + "This is a bug in bzip2/libbzip2, %s.\n" + "Please report it to me at: jseward AT acm DOT org. If this happened\n" + "when you were using some program which uses libbzip2 as a\n" + "component, you should also report this bug to the author(s)\n" + "of that program. Please make an effort to report this bug;\n" + "timely and accurate bug reports eventually lead to higher\n" + "quality software. Thanks. Julian Seward, 21 March 2000.\n\n", + errcode, + BZ2_bzlibVersion() + ); + exit(3); + } + #endif + + + /*---------------------------------------------------*/ + static + int bz_config_ok ( void ) + { + if (sizeof(int) != 4) return 0; + if (sizeof(short) != 2) return 0; + if (sizeof(char) != 1) return 0; + return 1; + } + + + /*---------------------------------------------------*/ + static + void* default_bzalloc ( void* opaque, Int32 items, Int32 size ) + { + void* v = malloc ( items * size ); + return v; + } + + static + void default_bzfree ( void* opaque, void* addr ) + { + if (addr != NULL) free ( addr ); + } + + + /*---------------------------------------------------*/ + /*--- Decompression stuff ---*/ + /*---------------------------------------------------*/ + + /*---------------------------------------------------*/ + int BZ_API(BZ2_bzDecompressInit) + ( bz_stream* strm, + int verbosity, + int small ) + { + DState* s; + + if (!bz_config_ok()) return BZ_CONFIG_ERROR; + + if (strm == NULL) return BZ_PARAM_ERROR; + if (small != 0 && small != 1) return BZ_PARAM_ERROR; + if (verbosity < 0 || verbosity > 4) return BZ_PARAM_ERROR; + + if (strm->bzalloc == NULL) strm->bzalloc = default_bzalloc; + if (strm->bzfree == NULL) strm->bzfree = default_bzfree; + + s = BZALLOC( sizeof(DState) ); + if (s == NULL) return BZ_MEM_ERROR; + s->strm = strm; + strm->state = s; + s->state = BZ_X_MAGIC_1; + s->bsLive = 0; + s->bsBuff = 0; + s->calculatedCombinedCRC = 0; + strm->total_in_lo32 = 0; + strm->total_in_hi32 = 0; + strm->total_out_lo32 = 0; + strm->total_out_hi32 = 0; + s->smallDecompress = (Bool)small; + s->ll4 = NULL; + s->ll16 = NULL; + s->tt = NULL; + s->currBlockNo = 0; + s->verbosity = verbosity; + + return BZ_OK; + } + + + /*---------------------------------------------------*/ + static + void unRLE_obuf_to_output_FAST ( DState* s ) + { + UChar k1; + + if (s->blockRandomised) { + + while (True) { + /* try to finish existing run */ + while (True) { + if (s->strm->avail_out == 0) return; + if (s->state_out_len == 0) break; + *( (UChar*)(s->strm->next_out) ) = s->state_out_ch; + BZ_UPDATE_CRC ( s->calculatedBlockCRC, s->state_out_ch ); + s->state_out_len--; + s->strm->next_out++; + s->strm->avail_out--; + s->strm->total_out_lo32++; + if (s->strm->total_out_lo32 == 0) s->strm->total_out_hi32++; + } + + /* can a new run be started? */ + if (s->nblock_used == s->save_nblock+1) return; + + + s->state_out_len = 1; + s->state_out_ch = s->k0; + BZ_GET_FAST(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + s->state_out_len = 2; + BZ_GET_FAST(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + s->state_out_len = 3; + BZ_GET_FAST(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + BZ_GET_FAST(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + s->state_out_len = ((Int32)k1) + 4; + BZ_GET_FAST(s->k0); BZ_RAND_UPD_MASK; + s->k0 ^= BZ_RAND_MASK; s->nblock_used++; + } + + } else { + + /* restore */ + UInt32 c_calculatedBlockCRC = s->calculatedBlockCRC; + UChar c_state_out_ch = s->state_out_ch; + Int32 c_state_out_len = s->state_out_len; + Int32 c_nblock_used = s->nblock_used; + Int32 c_k0 = s->k0; + UInt32* c_tt = s->tt; + UInt32 c_tPos = s->tPos; + char* cs_next_out = s->strm->next_out; + unsigned int cs_avail_out = s->strm->avail_out; + /* end restore */ + + UInt32 avail_out_INIT = cs_avail_out; + Int32 s_save_nblockPP = s->save_nblock+1; + unsigned int total_out_lo32_old; + + while (True) { + + /* try to finish existing run */ + if (c_state_out_len > 0) { + while (True) { + if (cs_avail_out == 0) goto return_notr; + if (c_state_out_len == 1) break; + *( (UChar*)(cs_next_out) ) = c_state_out_ch; + BZ_UPDATE_CRC ( c_calculatedBlockCRC, c_state_out_ch ); + c_state_out_len--; + cs_next_out++; + cs_avail_out--; + } + s_state_out_len_eq_one: + { + if (cs_avail_out == 0) { + c_state_out_len = 1; goto return_notr; + }; + *( (UChar*)(cs_next_out) ) = c_state_out_ch; + BZ_UPDATE_CRC ( c_calculatedBlockCRC, c_state_out_ch ); + cs_next_out++; + cs_avail_out--; + } + } + /* can a new run be started? */ + if (c_nblock_used == s_save_nblockPP) { + c_state_out_len = 0; goto return_notr; + }; + c_state_out_ch = c_k0; + BZ_GET_FAST_C(k1); c_nblock_used++; + if (k1 != c_k0) { + c_k0 = k1; goto s_state_out_len_eq_one; + }; + if (c_nblock_used == s_save_nblockPP) + goto s_state_out_len_eq_one; + + c_state_out_len = 2; + BZ_GET_FAST_C(k1); c_nblock_used++; + if (c_nblock_used == s_save_nblockPP) continue; + if (k1 != c_k0) { c_k0 = k1; continue; }; + + c_state_out_len = 3; + BZ_GET_FAST_C(k1); c_nblock_used++; + if (c_nblock_used == s_save_nblockPP) continue; + if (k1 != c_k0) { c_k0 = k1; continue; }; + + BZ_GET_FAST_C(k1); c_nblock_used++; + c_state_out_len = ((Int32)k1) + 4; + BZ_GET_FAST_C(c_k0); c_nblock_used++; + } + + return_notr: + total_out_lo32_old = s->strm->total_out_lo32; + s->strm->total_out_lo32 += (avail_out_INIT - cs_avail_out); + if (s->strm->total_out_lo32 < total_out_lo32_old) + s->strm->total_out_hi32++; + + /* save */ + s->calculatedBlockCRC = c_calculatedBlockCRC; + s->state_out_ch = c_state_out_ch; + s->state_out_len = c_state_out_len; + s->nblock_used = c_nblock_used; + s->k0 = c_k0; + s->tt = c_tt; + s->tPos = c_tPos; + s->strm->next_out = cs_next_out; + s->strm->avail_out = cs_avail_out; + /* end save */ + } + } + + + + /*---------------------------------------------------*/ + __inline__ Int32 BZ2_indexIntoF ( Int32 indx, Int32 *cftab ) + { + Int32 nb, na, mid; + nb = 0; + na = 256; + do { + mid = (nb + na) >> 1; + if (indx >= cftab[mid]) nb = mid; else na = mid; + } + while (na - nb != 1); + return nb; + } + + + /*---------------------------------------------------*/ + static + void unRLE_obuf_to_output_SMALL ( DState* s ) + { + UChar k1; + + if (s->blockRandomised) { + + while (True) { + /* try to finish existing run */ + while (True) { + if (s->strm->avail_out == 0) return; + if (s->state_out_len == 0) break; + *( (UChar*)(s->strm->next_out) ) = s->state_out_ch; + BZ_UPDATE_CRC ( s->calculatedBlockCRC, s->state_out_ch ); + s->state_out_len--; + s->strm->next_out++; + s->strm->avail_out--; + s->strm->total_out_lo32++; + if (s->strm->total_out_lo32 == 0) s->strm->total_out_hi32++; + } + + /* can a new run be started? */ + if (s->nblock_used == s->save_nblock+1) return; + + + s->state_out_len = 1; + s->state_out_ch = s->k0; + BZ_GET_SMALL(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + s->state_out_len = 2; + BZ_GET_SMALL(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + s->state_out_len = 3; + BZ_GET_SMALL(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + BZ_GET_SMALL(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + s->state_out_len = ((Int32)k1) + 4; + BZ_GET_SMALL(s->k0); BZ_RAND_UPD_MASK; + s->k0 ^= BZ_RAND_MASK; s->nblock_used++; + } + + } else { + + while (True) { + /* try to finish existing run */ + while (True) { + if (s->strm->avail_out == 0) return; + if (s->state_out_len == 0) break; + *( (UChar*)(s->strm->next_out) ) = s->state_out_ch; + BZ_UPDATE_CRC ( s->calculatedBlockCRC, s->state_out_ch ); + s->state_out_len--; + s->strm->next_out++; + s->strm->avail_out--; + s->strm->total_out_lo32++; + if (s->strm->total_out_lo32 == 0) s->strm->total_out_hi32++; + } + + /* can a new run be started? */ + if (s->nblock_used == s->save_nblock+1) return; + + s->state_out_len = 1; + s->state_out_ch = s->k0; + BZ_GET_SMALL(k1); s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + s->state_out_len = 2; + BZ_GET_SMALL(k1); s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + s->state_out_len = 3; + BZ_GET_SMALL(k1); s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + BZ_GET_SMALL(k1); s->nblock_used++; + s->state_out_len = ((Int32)k1) + 4; + BZ_GET_SMALL(s->k0); s->nblock_used++; + } + + } + } + + + /*---------------------------------------------------*/ + int BZ_API(BZ2_bzDecompress) ( bz_stream *strm ) + { + DState* s; + if (strm == NULL) return BZ_PARAM_ERROR; + s = strm->state; + if (s == NULL) return BZ_PARAM_ERROR; + if (s->strm != strm) return BZ_PARAM_ERROR; + + while (True) { + if (s->state == BZ_X_IDLE) return BZ_SEQUENCE_ERROR; + if (s->state == BZ_X_OUTPUT) { + if (s->smallDecompress) + unRLE_obuf_to_output_SMALL ( s ); else + unRLE_obuf_to_output_FAST ( s ); + if (s->nblock_used == s->save_nblock+1 && s->state_out_len == 0) { + BZ_FINALISE_CRC ( s->calculatedBlockCRC ); + if (s->verbosity >= 3) + VPrintf2 ( " {0x%x, 0x%x}", s->storedBlockCRC, + s->calculatedBlockCRC ); + if (s->verbosity >= 2) VPrintf0 ( "]" ); + if (s->calculatedBlockCRC != s->storedBlockCRC) + return BZ_DATA_ERROR; + s->calculatedCombinedCRC + = (s->calculatedCombinedCRC << 1) | + (s->calculatedCombinedCRC >> 31); + s->calculatedCombinedCRC ^= s->calculatedBlockCRC; + s->state = BZ_X_BLKHDR_1; + } else { + return BZ_OK; + } + } + if (s->state >= BZ_X_MAGIC_1) { + Int32 r = BZ2_decompress ( s ); + if (r == BZ_STREAM_END) { + if (s->verbosity >= 3) + VPrintf2 ( "\n combined CRCs: stored = 0x%x, computed = 0x%x", + s->storedCombinedCRC, s->calculatedCombinedCRC ); + if (s->calculatedCombinedCRC != s->storedCombinedCRC) + return BZ_DATA_ERROR; + return r; + } + if (s->state != BZ_X_OUTPUT) return r; + } + } + + AssertH ( 0, 6001 ); + + return 0; /*NOTREACHED*/ + } + + + /*---------------------------------------------------*/ + int BZ_API(BZ2_bzDecompressEnd) ( bz_stream *strm ) + { + DState* s; + if (strm == NULL) return BZ_PARAM_ERROR; + s = strm->state; + if (s == NULL) return BZ_PARAM_ERROR; + if (s->strm != strm) return BZ_PARAM_ERROR; + + if (s->tt != NULL) BZFREE(s->tt); + if (s->ll16 != NULL) BZFREE(s->ll16); + if (s->ll4 != NULL) BZFREE(s->ll4); + + BZFREE(strm->state); + strm->state = NULL; + + return BZ_OK; + } + + + + /*---------------------------------------------------*/ + /*-- + return version like "0.9.0c". + --*/ + const char * BZ_API(BZ2_bzlibVersion)(void) + { + return BZ_VERSION; + } + + + /*-------------------------------------------------------------*/ + /*--- end bzlib.c ---*/ + /*-------------------------------------------------------------*/ diff -acprNC5 djgpp.orig/src/utils/djtar/zmethod.c djgpp/src/utils/djtar/zmethod.c *** djgpp.orig/src/utils/djtar/zmethod.c Tue Nov 28 21:21:30 2000 --- djgpp/src/utils/djtar/zmethod.c Sat Dec 2 20:36:52 2000 *************** get_method(void *in) *** 188,197 **** --- 188,203 ---- } else if (memcmp(magic, LZH_MAGIC, 2) == 0) { decompressor = unlzh; method = LZHED; last_member = 1; + } else if (memcmp(magic, BZIP2_MAGIC, 2) == 0 && inptr == 2 + && memcmp((char*)inbuf, BZIP2_MAGIC, 3) == 0) { + decompressor = unbzip2; + method = BZIP2ED; + last_member = 1; + } else if (part_nb == 1) { /* pass input unchanged by default */ method = STORED; decompressor = copy; inptr = 0; last_member = 1; diff -acprNC5 djgpp.orig/src/utils/djtar/zread.h djgpp/src/utils/djtar/zread.h *** djgpp.orig/src/utils/djtar/zread.h Tue Nov 28 21:21:58 2000 --- djgpp/src/utils/djtar/zread.h Sat Dec 2 22:48:52 2000 *************** typedef unsigned long ulg; *** 29,39 **** #define COMPRESSED 1 #define PACKED 2 #define LZHED 3 /* methods 4 to 7 reserved */ #define DEFLATED 8 ! #define MAX_METHODS 9 extern int method; /* compression method */ extern FILE *log_out; /* the stream to output messages */ /* To save memory for 16 bit systems, some arrays are overlaid between --- 29,40 ---- #define COMPRESSED 1 #define PACKED 2 #define LZHED 3 /* methods 4 to 7 reserved */ #define DEFLATED 8 ! #define BZIP2ED 9 ! #define MAX_METHODS 10 extern int method; /* compression method */ extern FILE *log_out; /* the stream to output messages */ /* To save memory for 16 bit systems, some arrays are overlaid between *************** extern int pkzip; /* set for a *** 84,94 **** #define PACK_MAGIC "\037\036" /* Magic header for packed files */ #define GZIP_MAGIC "\037\213" /* Magic header for gzip files, 1F 8B */ #define OLD_GZIP_MAGIC "\037\236" /* Magic header for gzip 0.5 = freeze 1.x */ #define LZH_MAGIC "\037\240" /* Magic header for SCO LZH Compress files*/ ! #define PKZIP_MAGIC "\120\113\003\004" /* Magic header for pkzip files */ /* gzip flag byte */ #define ASCII_FLAG 0x01 /* bit 0 set: file probably ascii text */ #define CONTINUATION 0x02 /* bit 1 set: continuation of multi-part gzip file */ #define EXTRA_FIELD 0x04 /* bit 2 set: extra field present */ --- 85,96 ---- #define PACK_MAGIC "\037\036" /* Magic header for packed files */ #define GZIP_MAGIC "\037\213" /* Magic header for gzip files, 1F 8B */ #define OLD_GZIP_MAGIC "\037\236" /* Magic header for gzip 0.5 = freeze 1.x */ #define LZH_MAGIC "\037\240" /* Magic header for SCO LZH Compress files*/ ! #define PKZIP_MAGIC "\120\113\003\004" /* Magic header for pkzip files */ ! #define BZIP2_MAGIC "\102\132\150" /* Magic header for bzip2 files, BZh */ /* gzip flag byte */ #define ASCII_FLAG 0x01 /* bit 0 set: file probably ascii text */ #define CONTINUATION 0x02 /* bit 1 set: continuation of multi-part gzip file */ #define EXTRA_FIELD 0x04 /* bit 2 set: extra field present */ *************** extern int check_zipfile (void); *** 183,192 **** --- 185,197 ---- extern int unpack (void *); /* in unlzh.c */ extern int unlzh (void *); + /* in unbzip2.c */ + extern int unbzip2 (void *); + /* in util.c: */ extern int copy (void *); extern ulg updcrc (uch *s, unsigned n); extern void clear_bufs (void); extern int fill_inbuf (int eof_ok); diff -acprNC5 djgpp.orig/src/utils/utils.tex djgpp/src/utils/utils.tex *** djgpp.orig/src/utils/utils.tex Mon Nov 27 06:43:34 2000 --- djgpp/src/utils/utils.tex Mon Dec 4 05:29:00 2000 *************** space, because DOS implements pipes as t *** 143,152 **** --- 143,157 ---- @code{gzip}, namely, deflation (by @code{gzip}), LZW compression (by @code{compress}), LZH compression (as with @code{compress -H} available with some versions of @code{compress}), or Huffman compression (by @code{pack}). + @code{djtar} can also decompress @file{.tar} archive files compressed + with @code{bzip2}, even if multiple of this @code{bzip2} compressed + @file{.tar.bz2} or @file{.tbz} archive files have been concatenated + into a single one. + @code{djtar} can also decompress files compressed with @code{zip} or @code{pkzip}, but only if the @file{.zip} file contains only files compressed with either the @code{deflation} or @code{stored} (i.e., uncompressed, like with @samp{pkzip -e0}) methods.