压缩算法(二)——QuickLZ算法
详细直接官网看就好,传送:http://www.quicklz.com
这里目前仅是搬运。
目录
一、介绍
二、结构
三、C源码
quicklz.h
quicklz.c
四、sample
压缩:
解压
一、介绍
网址:http://www.quicklz.com
其中包括C、C#、JAVA等版本。
C版本QuickLZ包括以下几个文件:
注意最后一句,它仅依赖于string.h
二、结构
library:
quicklz.h
quicklz.c
三、C源码
源码:
quicklz.h
#ifndef QLZ_HEADER
#define QLZ_HEADER// Fast data compression library
// Copyright (C) 2006-2011 Lasse Mikkel Reinhold
// lar@quicklz.com
//
// QuickLZ can be used for free under the GPL 1, 2 or 3 license (where anything
// released into public must be open source) or under a commercial license if such
// has been acquired (see http://www.quicklz.com/order.html). The commercial license
// does not cover derived or ported versions created by third parties under GPL.// You can edit following user settings. Data must be decompressed with the same
// setting of QLZ_COMPRESSION_LEVEL and QLZ_STREAMING_BUFFER as it was compressed
// (see manual). If QLZ_STREAMING_BUFFER > 0, scratch buffers must be initially
// zeroed out (see manual). First #ifndef makes it possible to define settings from
// the outside like the compiler command line.// 1.5.0 final#ifndef QLZ_COMPRESSION_LEVEL// 1 gives fastest compression speed. 3 gives fastest decompression speed and best// compression ratio. #define QLZ_COMPRESSION_LEVEL 1//#define QLZ_COMPRESSION_LEVEL 2//#define QLZ_COMPRESSION_LEVEL 3// If > 0, zero out both states prior to first call to qlz_compress() or qlz_decompress() // and decompress packets in the same order as they were compressed#define QLZ_STREAMING_BUFFER 0//#define QLZ_STREAMING_BUFFER 100000//#define QLZ_STREAMING_BUFFER 1000000// Guarantees that decompression of corrupted data cannot crash. Decreases decompression// speed 10-20%. Compression speed not affected.//#define QLZ_MEMORY_SAFE
#endif#define QLZ_VERSION_MAJOR 1
#define QLZ_VERSION_MINOR 5
#define QLZ_VERSION_REVISION 0// Using size_t, memset() and memcpy()
#include <string.h>// Verify compression level
#if QLZ_COMPRESSION_LEVEL != 1 && QLZ_COMPRESSION_LEVEL != 2 && QLZ_COMPRESSION_LEVEL != 3
#error QLZ_COMPRESSION_LEVEL must be 1, 2 or 3
#endiftypedef unsigned int ui32;
typedef unsigned short int ui16;// Decrease QLZ_POINTERS for level 3 to increase compression speed. Do not touch any other values!
#if QLZ_COMPRESSION_LEVEL == 1
#define QLZ_POINTERS 1
#define QLZ_HASH_VALUES 4096
#elif QLZ_COMPRESSION_LEVEL == 2
#define QLZ_POINTERS 4
#define QLZ_HASH_VALUES 2048
#elif QLZ_COMPRESSION_LEVEL == 3
#define QLZ_POINTERS 16
#define QLZ_HASH_VALUES 4096
#endif// Detect if pointer size is 64-bit. It's not fatal if some 64-bit target is not detected because this is only for adding an optional 64-bit optimization.
#if defined _LP64 || defined __LP64__ || defined __64BIT__ || _ADDR64 || defined _WIN64 || defined __arch64__ || __WORDSIZE == 64 || (defined __sparc && defined __sparcv9) || defined __x86_64 || defined __amd64 || defined __x86_64__ || defined _M_X64 || defined _M_IA64 || defined __ia64 || defined __IA64__#define QLZ_PTR_64
#endif// hash entry
typedef struct
{
#if QLZ_COMPRESSION_LEVEL == 1ui32 cache;
#if defined QLZ_PTR_64 && QLZ_STREAMING_BUFFER == 0unsigned int offset;
#elseconst unsigned char *offset;
#endif
#elseconst unsigned char *offset[QLZ_POINTERS];
#endif} qlz_hash_compress;typedef struct
{
#if QLZ_COMPRESSION_LEVEL == 1const unsigned char *offset;
#elseconst unsigned char *offset[QLZ_POINTERS];
#endif
} qlz_hash_decompress;// states
typedef struct
{#if QLZ_STREAMING_BUFFER > 0unsigned char stream_buffer[QLZ_STREAMING_BUFFER];#endifsize_t stream_counter;qlz_hash_compress hash[QLZ_HASH_VALUES];unsigned char hash_counter[QLZ_HASH_VALUES];
} qlz_state_compress;#if QLZ_COMPRESSION_LEVEL == 1 || QLZ_COMPRESSION_LEVEL == 2typedef struct{
#if QLZ_STREAMING_BUFFER > 0unsigned char stream_buffer[QLZ_STREAMING_BUFFER];
#endifqlz_hash_decompress hash[QLZ_HASH_VALUES];unsigned char hash_counter[QLZ_HASH_VALUES];size_t stream_counter;} qlz_state_decompress;
#elif QLZ_COMPRESSION_LEVEL == 3typedef struct{
#if QLZ_STREAMING_BUFFER > 0unsigned char stream_buffer[QLZ_STREAMING_BUFFER];
#endif
#if QLZ_COMPRESSION_LEVEL <= 2qlz_hash_decompress hash[QLZ_HASH_VALUES];
#endifsize_t stream_counter;} qlz_state_decompress;
#endif#if defined (__cplusplus)
extern "C" {
#endif// Public functions of QuickLZ
size_t qlz_size_decompressed(const char *source);
size_t qlz_size_compressed(const char *source);
size_t qlz_compress(const void *source, char *destination, size_t size, qlz_state_compress *state);
size_t qlz_decompress(const char *source, void *destination, qlz_state_decompress *state);
int qlz_get_setting(int setting);#if defined (__cplusplus)
}
#endif#endif
quicklz.c
// Fast data compression library
// Copyright (C) 2006-2011 Lasse Mikkel Reinhold
// lar@quicklz.com
//
// QuickLZ can be used for free under the GPL 1, 2 or 3 license (where anything
// released into public must be open source) or under a commercial license if such
// has been acquired (see http://www.quicklz.com/order.html). The commercial license
// does not cover derived or ported versions created by third parties under GPL.// 1.5.0 final#include "quicklz.h"#if QLZ_VERSION_MAJOR != 1 || QLZ_VERSION_MINOR != 5 || QLZ_VERSION_REVISION != 0#error quicklz.c and quicklz.h have different versions
#endif#if (defined(__X86__) || defined(__i386__) || defined(i386) || defined(_M_IX86) || defined(__386__) || defined(__x86_64__) || defined(_M_X64))#define X86X64
#endif#define MINOFFSET 2
#define UNCONDITIONAL_MATCHLEN 6
#define UNCOMPRESSED_END 4
#define CWORD_LEN 4#if QLZ_COMPRESSION_LEVEL == 1 && defined QLZ_PTR_64 && QLZ_STREAMING_BUFFER == 0#define OFFSET_BASE source#define CAST (ui32)(size_t)
#else#define OFFSET_BASE 0#define CAST
#endifint qlz_get_setting(int setting)
{switch (setting){case 0: return QLZ_COMPRESSION_LEVEL;case 1: return sizeof(qlz_state_compress);case 2: return sizeof(qlz_state_decompress);case 3: return QLZ_STREAMING_BUFFER;
#ifdef QLZ_MEMORY_SAFEcase 6: return 1;
#elsecase 6: return 0;
#endifcase 7: return QLZ_VERSION_MAJOR;case 8: return QLZ_VERSION_MINOR;case 9: return QLZ_VERSION_REVISION;}return -1;
}#if QLZ_COMPRESSION_LEVEL == 1
static int same(const unsigned char *src, size_t n)
{while(n > 0 && *(src + n) == *src)n--;return n == 0 ? 1 : 0;
}
#endifstatic void reset_table_compress(qlz_state_compress *state)
{int i;for(i = 0; i < QLZ_HASH_VALUES; i++){
#if QLZ_COMPRESSION_LEVEL == 1state->hash[i].offset = 0;
#elsestate->hash_counter[i] = 0;
#endif}
}static void reset_table_decompress(qlz_state_decompress *state)
{int i;(void)state;(void)i;
#if QLZ_COMPRESSION_LEVEL == 2for(i = 0; i < QLZ_HASH_VALUES; i++){state->hash_counter[i] = 0;}
#endif
}static __inline ui32 hash_func(ui32 i)
{
#if QLZ_COMPRESSION_LEVEL == 2return ((i >> 9) ^ (i >> 13) ^ i) & (QLZ_HASH_VALUES - 1);
#elsereturn ((i >> 12) ^ i) & (QLZ_HASH_VALUES - 1);
#endif
}static __inline ui32 fast_read(void const *src, ui32 bytes)
{
#ifndef X86X64unsigned char *p = (unsigned char*)src;switch (bytes){case 4:return(*p | *(p + 1) << 8 | *(p + 2) << 16 | *(p + 3) << 24);case 3: return(*p | *(p + 1) << 8 | *(p + 2) << 16);case 2:return(*p | *(p + 1) << 8);case 1: return(*p);}return 0;
#elseif (bytes >= 1 && bytes <= 4)return *((ui32*)src);elsereturn 0;
#endif
}static __inline ui32 hashat(const unsigned char *src)
{ui32 fetch, hash;fetch = fast_read(src, 3);hash = hash_func(fetch);return hash;
}static __inline void fast_write(ui32 f, void *dst, size_t bytes)
{
#ifndef X86X64unsigned char *p = (unsigned char*)dst;switch (bytes){case 4: *p = (unsigned char)f;*(p + 1) = (unsigned char)(f >> 8);*(p + 2) = (unsigned char)(f >> 16);*(p + 3) = (unsigned char)(f >> 24);return;case 3:*p = (unsigned char)f;*(p + 1) = (unsigned char)(f >> 8);*(p + 2) = (unsigned char)(f >> 16);return;case 2:*p = (unsigned char)f;*(p + 1) = (unsigned char)(f >> 8);return;case 1:*p = (unsigned char)f;return;}
#elseswitch (bytes){case 4: *((ui32*)dst) = f;return;case 3:*((ui32*)dst) = f;return;case 2:*((ui16 *)dst) = (ui16)f;return;case 1:*((unsigned char*)dst) = (unsigned char)f;return;}
#endif
}size_t qlz_size_decompressed(const char *source)
{ui32 n, r;n = (((*source) & 2) == 2) ? 4 : 1;r = fast_read(source + 1 + n, n);r = r & (0xffffffff >> ((4 - n)*8));return r;
}size_t qlz_size_compressed(const char *source)
{ui32 n, r;n = (((*source) & 2) == 2) ? 4 : 1;r = fast_read(source + 1, n);r = r & (0xffffffff >> ((4 - n)*8));return r;
}size_t qlz_size_header(const char *source)
{size_t n = 2*((((*source) & 2) == 2) ? 4 : 1) + 1;return n;
}static __inline void memcpy_up(unsigned char *dst, const unsigned char *src, ui32 n)
{// Caution if modifying memcpy_up! Overlap of dst and src must be special handled.
#ifndef X86X64unsigned char *end = dst + n;while(dst < end){*dst = *src;dst++;src++;}
#elseui32 f = 0;do{*(ui32 *)(dst + f) = *(ui32 *)(src + f);f += MINOFFSET + 1;}while (f < n);
#endif
}static __inline void update_hash(qlz_state_decompress *state, const unsigned char *s)
{
#if QLZ_COMPRESSION_LEVEL == 1ui32 hash;hash = hashat(s);state->hash[hash].offset = s;state->hash_counter[hash] = 1;
#elif QLZ_COMPRESSION_LEVEL == 2ui32 hash;unsigned char c;hash = hashat(s);c = state->hash_counter[hash];state->hash[hash].offset[c & (QLZ_POINTERS - 1)] = s;c++;state->hash_counter[hash] = c;
#endif(void)state;(void)s;
}#if QLZ_COMPRESSION_LEVEL <= 2
static void update_hash_upto(qlz_state_decompress *state, unsigned char **lh, const unsigned char *max)
{while(*lh < max){(*lh)++;update_hash(state, *lh);}
}
#endifstatic size_t qlz_compress_core(const unsigned char *source, unsigned char *destination, size_t size, qlz_state_compress *state)
{const unsigned char *last_byte = source + size - 1;const unsigned char *src = source;unsigned char *cword_ptr = destination;unsigned char *dst = destination + CWORD_LEN;ui32 cword_val = 1U << 31;const unsigned char *last_matchstart = last_byte - UNCONDITIONAL_MATCHLEN - UNCOMPRESSED_END; ui32 fetch = 0;unsigned int lits = 0;(void) lits;if(src <= last_matchstart)fetch = fast_read(src, 3);while(src <= last_matchstart){if ((cword_val & 1) == 1){// store uncompressed if compression ratio is too lowif (src > source + (size >> 1) && dst - destination > src - source - ((src - source) >> 5))return 0;fast_write((cword_val >> 1) | (1U << 31), cword_ptr, CWORD_LEN);cword_ptr = dst;dst += CWORD_LEN;cword_val = 1U << 31;fetch = fast_read(src, 3);}
#if QLZ_COMPRESSION_LEVEL == 1{const unsigned char *o;ui32 hash, cached;hash = hash_func(fetch);cached = fetch ^ state->hash[hash].cache;state->hash[hash].cache = fetch;o = state->hash[hash].offset + OFFSET_BASE;state->hash[hash].offset = CAST(src - OFFSET_BASE);#ifdef X86X64if ((cached & 0xffffff) == 0 && o != OFFSET_BASE && (src - o > MINOFFSET || (src == o + 1 && lits >= 3 && src > source + 3 && same(src - 3, 6)))){if(cached != 0){
#elseif (cached == 0 && o != OFFSET_BASE && (src - o > MINOFFSET || (src == o + 1 && lits >= 3 && src > source + 3 && same(src - 3, 6)))){if (*(o + 3) != *(src + 3)){
#endifhash <<= 4;cword_val = (cword_val >> 1) | (1U << 31);fast_write((3 - 2) | hash, dst, 2);src += 3;dst += 2;}else{const unsigned char *old_src = src;size_t matchlen;hash <<= 4;cword_val = (cword_val >> 1) | (1U << 31);src += 4;if(*(o + (src - old_src)) == *src){src++;if(*(o + (src - old_src)) == *src){size_t q = last_byte - UNCOMPRESSED_END - (src - 5) + 1;size_t remaining = q > 255 ? 255 : q;src++; while(*(o + (src - old_src)) == *src && (size_t)(src - old_src) < remaining)src++;}}matchlen = src - old_src;if (matchlen < 18){fast_write((ui32)(matchlen - 2) | hash, dst, 2);dst += 2;} else{fast_write((ui32)(matchlen << 16) | hash, dst, 3);dst += 3;}}fetch = fast_read(src, 3);lits = 0;}else{lits++;*dst = *src;src++;dst++;cword_val = (cword_val >> 1);
#ifdef X86X64fetch = fast_read(src, 3);
#elsefetch = (fetch >> 8 & 0xffff) | (*(src + 2) << 16);
#endif}}
#elif QLZ_COMPRESSION_LEVEL >= 2{const unsigned char *o, *offset2;ui32 hash, matchlen, k, m, best_k = 0;unsigned char c;size_t remaining = (last_byte - UNCOMPRESSED_END - src + 1) > 255 ? 255 : (last_byte - UNCOMPRESSED_END - src + 1);(void)best_k;//hash = hashat(src);fetch = fast_read(src, 3);hash = hash_func(fetch);c = state->hash_counter[hash];offset2 = state->hash[hash].offset[0];if(offset2 < src - MINOFFSET && c > 0 && ((fast_read(offset2, 3) ^ fetch) & 0xffffff) == 0){ matchlen = 3;if(*(offset2 + matchlen) == *(src + matchlen)){matchlen = 4;while(*(offset2 + matchlen) == *(src + matchlen) && matchlen < remaining)matchlen++;}}elsematchlen = 0;for(k = 1; k < QLZ_POINTERS && c > k; k++){o = state->hash[hash].offset[k];
#if QLZ_COMPRESSION_LEVEL == 3if(((fast_read(o, 3) ^ fetch) & 0xffffff) == 0 && o < src - MINOFFSET)
#elif QLZ_COMPRESSION_LEVEL == 2if(*(src + matchlen) == *(o + matchlen) && ((fast_read(o, 3) ^ fetch) & 0xffffff) == 0 && o < src - MINOFFSET)
#endif{ m = 3;while(*(o + m) == *(src + m) && m < remaining)m++;
#if QLZ_COMPRESSION_LEVEL == 3if ((m > matchlen) || (m == matchlen && o > offset2))
#elif QLZ_COMPRESSION_LEVEL == 2if (m > matchlen)
#endif{offset2 = o;matchlen = m;best_k = k;}}}o = offset2;state->hash[hash].offset[c & (QLZ_POINTERS - 1)] = src;c++;state->hash_counter[hash] = c;#if QLZ_COMPRESSION_LEVEL == 3if(matchlen > 2 && src - o < 131071){ui32 u;size_t offset = src - o;for(u = 1; u < matchlen; u++){ hash = hashat(src + u);c = state->hash_counter[hash]++;state->hash[hash].offset[c & (QLZ_POINTERS - 1)] = src + u;}cword_val = (cword_val >> 1) | (1U << 31);src += matchlen;if(matchlen == 3 && offset <= 63){*dst = (unsigned char)(offset << 2);dst++;}else if (matchlen == 3 && offset <= 16383){ui32 f = (ui32)((offset << 2) | 1);fast_write(f, dst, 2);dst += 2;} else if (matchlen <= 18 && offset <= 1023){ui32 f = ((matchlen - 3) << 2) | ((ui32)offset << 6) | 2;fast_write(f, dst, 2);dst += 2;}else if(matchlen <= 33){ui32 f = ((matchlen - 2) << 2) | ((ui32)offset << 7) | 3;fast_write(f, dst, 3);dst += 3;}else{ui32 f = ((matchlen - 3) << 7) | ((ui32)offset << 15) | 3;fast_write(f, dst, 4);dst += 4;}}else{*dst = *src;src++;dst++;cword_val = (cword_val >> 1);}
#elif QLZ_COMPRESSION_LEVEL == 2if(matchlen > 2){cword_val = (cword_val >> 1) | (1U << 31);src += matchlen; if (matchlen < 10){ ui32 f = best_k | ((matchlen - 2) << 2) | (hash << 5);fast_write(f, dst, 2);dst += 2;}else{ui32 f = best_k | (matchlen << 16) | (hash << 5);fast_write(f, dst, 3);dst += 3;}}else{*dst = *src;src++;dst++;cword_val = (cword_val >> 1);}
#endif}
#endif}while (src <= last_byte){if ((cword_val & 1) == 1){fast_write((cword_val >> 1) | (1U << 31), cword_ptr, CWORD_LEN);cword_ptr = dst;dst += CWORD_LEN;cword_val = 1U << 31;}
#if QLZ_COMPRESSION_LEVEL < 3if (src <= last_byte - 3){
#if QLZ_COMPRESSION_LEVEL == 1ui32 hash, fetch;fetch = fast_read(src, 3);hash = hash_func(fetch);state->hash[hash].offset = CAST(src - OFFSET_BASE);state->hash[hash].cache = fetch;
#elif QLZ_COMPRESSION_LEVEL == 2ui32 hash;unsigned char c;hash = hashat(src);c = state->hash_counter[hash];state->hash[hash].offset[c & (QLZ_POINTERS - 1)] = src;c++;state->hash_counter[hash] = c;
#endif}
#endif*dst = *src;src++;dst++;cword_val = (cword_val >> 1);}while((cword_val & 1) != 1)cword_val = (cword_val >> 1);fast_write((cword_val >> 1) | (1U << 31), cword_ptr, CWORD_LEN);// min. size must be 9 bytes so that the qlz_size functions can take 9 bytes as argumentreturn dst - destination < 9 ? 9 : dst - destination;
}static size_t qlz_decompress_core(const unsigned char *source, unsigned char *destination, size_t size, qlz_state_decompress *state, const unsigned char *history)
{const unsigned char *src = source + qlz_size_header((const char *)source);unsigned char *dst = destination;const unsigned char *last_destination_byte = destination + size - 1;ui32 cword_val = 1;const unsigned char *last_matchstart = last_destination_byte - UNCONDITIONAL_MATCHLEN - UNCOMPRESSED_END;unsigned char *last_hashed = destination - 1;const unsigned char *last_source_byte = source + qlz_size_compressed((const char *)source) - 1;static const ui32 bitlut[16] = {4, 0, 1, 0, 2, 0, 1, 0, 3, 0, 1, 0, 2, 0, 1, 0};(void) last_source_byte;(void) last_hashed;(void) state;(void) history;for(;;) {ui32 fetch;if (cword_val == 1){
#ifdef QLZ_MEMORY_SAFEif(src + CWORD_LEN - 1 > last_source_byte)return 0;
#endifcword_val = fast_read(src, CWORD_LEN);src += CWORD_LEN;}#ifdef QLZ_MEMORY_SAFEif(src + 4 - 1 > last_source_byte)return 0;
#endiffetch = fast_read(src, 4);if ((cword_val & 1) == 1){ui32 matchlen;const unsigned char *offset2;#if QLZ_COMPRESSION_LEVEL == 1ui32 hash;cword_val = cword_val >> 1;hash = (fetch >> 4) & 0xfff;offset2 = (const unsigned char *)(size_t)state->hash[hash].offset;if((fetch & 0xf) != 0){matchlen = (fetch & 0xf) + 2;src += 2;}else{matchlen = *(src + 2);src += 3; } #elif QLZ_COMPRESSION_LEVEL == 2ui32 hash;unsigned char c;cword_val = cword_val >> 1;hash = (fetch >> 5) & 0x7ff;c = (unsigned char)(fetch & 0x3);offset2 = state->hash[hash].offset[c];if((fetch & (28)) != 0){matchlen = ((fetch >> 2) & 0x7) + 2;src += 2;}else{matchlen = *(src + 2);src += 3; } #elif QLZ_COMPRESSION_LEVEL == 3ui32 offset;cword_val = cword_val >> 1;if ((fetch & 3) == 0){offset = (fetch & 0xff) >> 2;matchlen = 3;src++;}else if ((fetch & 2) == 0){offset = (fetch & 0xffff) >> 2;matchlen = 3;src += 2;}else if ((fetch & 1) == 0){offset = (fetch & 0xffff) >> 6;matchlen = ((fetch >> 2) & 15) + 3;src += 2;}else if ((fetch & 127) != 3){offset = (fetch >> 7) & 0x1ffff;matchlen = ((fetch >> 2) & 0x1f) + 2;src += 3;}else{offset = (fetch >> 15);matchlen = ((fetch >> 7) & 255) + 3;src += 4;}offset2 = dst - offset;
#endif#ifdef QLZ_MEMORY_SAFEif(offset2 < history || offset2 > dst - MINOFFSET - 1)return 0;if(matchlen > (ui32)(last_destination_byte - dst - UNCOMPRESSED_END + 1))return 0;
#endifmemcpy_up(dst, offset2, matchlen);dst += matchlen;#if QLZ_COMPRESSION_LEVEL <= 2update_hash_upto(state, &last_hashed, dst - matchlen);last_hashed = dst - 1;
#endif}else{if (dst < last_matchstart){unsigned int n = bitlut[cword_val & 0xf];
#ifdef X86X64*(ui32 *)dst = *(ui32 *)src;
#elsememcpy_up(dst, src, 4);
#endifcword_val = cword_val >> n;dst += n;src += n;
#if QLZ_COMPRESSION_LEVEL <= 2update_hash_upto(state, &last_hashed, dst - 3);
#endif}else{ while(dst <= last_destination_byte){if (cword_val == 1){src += CWORD_LEN;cword_val = 1U << 31;}
#ifdef QLZ_MEMORY_SAFEif(src >= last_source_byte + 1)return 0;
#endif*dst = *src;dst++;src++;cword_val = cword_val >> 1;}#if QLZ_COMPRESSION_LEVEL <= 2update_hash_upto(state, &last_hashed, last_destination_byte - 3); // todo, use constant
#endifreturn size;}}}
}size_t qlz_compress(const void *source, char *destination, size_t size, qlz_state_compress *state)
{size_t r;ui32 compressed;size_t base;if(size == 0 || size > 0xffffffff - 400)return 0;if(size < 216)base = 3;elsebase = 9;#if QLZ_STREAMING_BUFFER > 0if (state->stream_counter + size - 1 >= QLZ_STREAMING_BUFFER)
#endif{reset_table_compress(state);r = base + qlz_compress_core((const unsigned char *)source, (unsigned char*)destination + base, size, state);
#if QLZ_STREAMING_BUFFER > 0reset_table_compress(state);
#endifif(r == base){memcpy(destination + base, source, size);r = size + base;compressed = 0;}else{compressed = 1;}state->stream_counter = 0;}
#if QLZ_STREAMING_BUFFER > 0else{unsigned char *src = state->stream_buffer + state->stream_counter;memcpy(src, source, size);r = base + qlz_compress_core(src, (unsigned char*)destination + base, size, state);if(r == base){memcpy(destination + base, src, size);r = size + base;compressed = 0;reset_table_compress(state);}else{compressed = 1;}state->stream_counter += size;}
#endifif(base == 3){*destination = (unsigned char)(0 | compressed);*(destination + 1) = (unsigned char)r;*(destination + 2) = (unsigned char)size;}else{*destination = (unsigned char)(2 | compressed);fast_write((ui32)r, destination + 1, 4);fast_write((ui32)size, destination + 5, 4);}*destination |= (QLZ_COMPRESSION_LEVEL << 2);*destination |= (1 << 6);*destination |= ((QLZ_STREAMING_BUFFER == 0 ? 0 : (QLZ_STREAMING_BUFFER == 100000 ? 1 : (QLZ_STREAMING_BUFFER == 1000000 ? 2 : 3))) << 4);// 76543210
// 01SSLLHCreturn r;
}size_t qlz_decompress(const char *source, void *destination, qlz_state_decompress *state)
{size_t dsiz = qlz_size_decompressed(source);#if QLZ_STREAMING_BUFFER > 0if (state->stream_counter + qlz_size_decompressed(source) - 1 >= QLZ_STREAMING_BUFFER)
#endif{if((*source & 1) == 1){reset_table_decompress(state);dsiz = qlz_decompress_core((const unsigned char *)source, (unsigned char *)destination, dsiz, state, (const unsigned char *)destination);}else{memcpy(destination, source + qlz_size_header(source), dsiz);}state->stream_counter = 0;reset_table_decompress(state);}
#if QLZ_STREAMING_BUFFER > 0else{unsigned char *dst = state->stream_buffer + state->stream_counter;if((*source & 1) == 1){dsiz = qlz_decompress_core((const unsigned char *)source, dst, dsiz, state, (const unsigned char *)state->stream_buffer);}else{memcpy(dst, source + qlz_size_header(source), dsiz);reset_table_decompress(state);}memcpy(destination, dst, dsiz);state->stream_counter += dsiz;}
#endifreturn dsiz;
}
四、sample
压缩:
// Sample demo for QuickLZ 1.5.x// Remember to define QLZ_COMPRESSION_LEVEL and QLZ_STREAMING_MODE to the same values for the compressor and decompressor#include <stdio.h>
#include <stdlib.h>#include "quicklz.h"int main(int argc, char* argv[])
{FILE *ifile, *ofile;char *src, *dst;qlz_state_compress *state_compress = (qlz_state_compress *)malloc(sizeof(qlz_state_compress));size_t len, len2; ifile = fopen(argv[1], "rb");ofile = fopen(argv[2], "wb");// allocate source buffer and read filefseek(ifile, 0, SEEK_END);len = ftell(ifile);fseek(ifile, 0, SEEK_SET);src = (char*) malloc(len);fread(src, 1, len, ifile);// allocate "uncompressed size" + 400 for the destination bufferdst = (char*) malloc(len + 400);// compress and write resultlen2 = qlz_compress(src, dst, len, state_compress);fwrite(dst, len2, 1, ofile);fclose(ifile);fclose(ofile);return 0;
}
解压
// Sample demo for QuickLZ 1.5.x// Remember to define QLZ_COMPRESSION_LEVEL and QLZ_STREAMING_MODE to the same values for the compressor and decompressor#include <stdio.h>
#include <stdlib.h>#include "quicklz.h"int main(int argc, char* argv[])
{FILE *ifile, *ofile;char *src, *dst, *scratch;qlz_state_decompress *state_decompress = (qlz_state_decompress *)malloc(sizeof(qlz_state_decompress));unsigned int len;ifile = fopen(argv[1], "rb");ofile = fopen(argv[2], "wb");// allocate source bufferfseek(ifile, 0, SEEK_END);len = ftell(ifile);fseek(ifile, 0, SEEK_SET);src = (char*) malloc(len);// read file and allocate destination bufferfread(src, 1, len, ifile);len = qlz_size_decompressed(src);dst = (char*) malloc(len);// decompress and write resultlen = qlz_decompress(src, dst, state_decompress);fwrite(dst, len, 1, ofile);fclose(ifile);fclose(ofile);return 0;
}
压缩算法(二)——QuickLZ算法相关推荐
- ML之LiRSGDR:基于二种算法(LiR、SGDR)对Boston(波士顿房价)数据集(506,13+1)进行价格回归预测并对比各自性能
ML之LiR&SGDR:基于二种算法(LiR.SGDR)对Boston(波士顿房价)数据集(506,13+1)进行价格回归预测并对比各自性能 目录 输出结果 设计思路 核心代码 输出结果 Bo ...
- Algorithm:【Algorithm算法进阶之路】之数据结构二十多种算法演示
Algorithm:[Algorithm算法进阶之路]之数据结构二十多种算法演示 目录 一.数据结构算法 1.顺序表 2.链表 3.栈和队列 4.串的模式匹配 5.稀疏矩阵 6.广义表 7.二叉树 8 ...
- JavaScript实现数乘以二multiplyByTwo算法(附完整源码)
JavaScript实现数乘以二multiplyByTwo算法(附完整源码) multiplyByTwo.js完整源代码 multiplyByTwo.js完整源代码 /*** @param {numb ...
- JavaScript实现数除以二divideByTwo算法(附完整源码)
JavaScript实现数除以二divideByTwo算法(附完整源码) divideByTwo.js完整源代码 divideByTwo.js完整源代码 /*** @param {number} nu ...
- 数据结构(二)算法基础与复杂度
一.算法基础 1.算法是解决特定问题求解步骤的描述,在计算机中表现为指令的有限序列,并且每条指令表示一个或多个操作. 2.算法具有五个基本特性:输入.输出.有穷性.确定性和可行性. (1)输入输出:算 ...
- #1098 : 最小生成树二·Kruscal算法
#1098 : 最小生成树二·Kruscal算法 时间限制:10000ms 单点时限:1000ms 内存限制:256MB 描述 随着小Hi拥有城市数目的增加,在之间所使用的Prim算法已经无法继续使用 ...
- 深入解析GBDT二分类算法(附代码实现)
目录: GBDT分类算法简介 GBDT二分类算法 2.1 逻辑回归的对数损失函数 2.2 GBDT二分类原理 GBDT二分类算法实例 手撕GBDT二分类算法 4.1 用Python3实现GBDT二分类 ...
- 重新学习 React (二) Diffing 算法
前几天面试问道 react 的相关知识,对我打击比较大,感觉对 react 认识非常肤浅,所以在这里重新梳理一下,想想之前没有仔细思考过的东西. 另外有说的不对的地方还请帮我指正一下,先谢谢各位啦. ...
- python复现感知机的二分类算法
本博客复现感知机的二分类算法 import pandas as pd import numpy as npclass Perceptron:def __init__(self, w=None, b=0 ...
- hihoCoder-1098最小生成树二·Kruscal算法 (最小生成树)
最小生成树二·Kruscal算法 描述 随着小Hi拥有城市数目的增加,在之间所使用的Prim算法已经无法继续使用了--但是幸运的是,经过计算机的分析,小Hi已经筛选出了一些比较适合建造道路的路线,这个 ...
最新文章
- NBT:宏基因组二、三代混合组装软件OPERA-MS
- 快速指南:在DevOps中实现持续交付
- Tomcat通过Redis实现session共享的完整部署记录
- 【Linux】一步一步学Linux——dpkg-split命令(274)
- Jdk1.8新特性(一) 接口的default方法
- MySQL year()函数
- 在vue-cli 中使用 axios
- nginx配置和前端跨域问题
- Oracle -----视图
- Libra 新编程语言 :Move 所有权模型灵感竟是来自它……
- SQL 数据定义语句(DDL)
- 智慧能源:浅谈新一代信息技术在智慧能源的应
- esp连接服务器的协议,【零知ESP8266教程】WIFI TCP协议通信 TCP服务器示例
- Matlab调用Python函数,出现OSError: [WinError 126] 找不到指定的模块报错
- MATLAB 计算 确定系数R2
- 福昕扫描王将取代扫描仪,OCR文字识别应用办公标配
- Mysql 事务锁等待时间超时
- 嵌入式开发—交叉编译OpenCV
- 二十三、正则表达式中的“r”含义
- 为什么我们放弃了 Vue?Vue 和 React 深度对比