Big log-file compression test


File type : Large traffic log file of a popular website
# of files to compress in this test : 1
Total File Size (bytes) : 20,617,071
Sample of data :

216.154.140.251 - - [03/Feb/2003:04:21:02 +0100] "GET /thumbn/cf104_r.jpg HTTP/1.1" 200 4559 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
216.154.140.251 - - [03/Feb/2003:04:21:07 +0100] "GET /full/mig17.jpg HTTP/1.1" 200 11756 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
68.113.117.251 - - [03/Feb/2003:04:21:20 +0100] "GET /info/f16.htm HTTP/1.1" 200 17687 "http://www.fighter-planes.com/data7080.htm" "Mozilla/4.0 (compatible; MSIE 5.5; MSN 2.5; Windows 98)"
68.113.117.251 - - [03/Feb/2003:04:21:21 +0100] "GET /thumbn/f16_r.jpg HTTP/1.1" 304 - "http://www.fighter-planes.com/info/f16.htm" "Mozilla/4.0 (compatible; MSIE 5.5; MSN 2.5; Windows 98)"
202.156.2.164 - - [03/Feb/2003:04:21:35 +0100] "GET /thumbn/f16_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /data9099.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /thumbn/yf23_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /thumbn/idf_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:41 +0100] "GET /thumbn/su35_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:43 +0100] "GET /info/f22.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
216.154.140.251 - - [03/Feb/2003:04:21:45 +0100] "GET /full/cf105.jpg HTTP/1.1" 200 26803 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
202.156.2.164 - - [03/Feb/2003:04:21:50 +0100] "GET /info/ef2000.htm HTTP/1.1" 200 9066 "http://www.fighter-planes.com/data99.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)"
202.156.2.164 - - [03/Feb/2003:04:22:04 +0100] "GET /info/f18e.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"

Conclusion: Because of the many repeating patterns the LOG-file is the best compressable file in the test-set. Although nearly all compressors are capable of compressing this file by at least 90%, the best program in this test, PAQ8, has a resulting filesize which is 5 times smaller then WinZip 8.0 and more then 3 times smaller then Microsofts CABARC. The resulting archive size with PAQ8 is only 1.25% of the log-files' original size (that's 80 times smaller!!). This test is again dominated by compressors using a PAQ(like) engine.

Number of different compressors/archivers listed in this test: 211


Pos. Name Compressor Best switches combination Compressed Compress Bits per
  Size Ratio Byte
  (bytes) (%) (b/B)
001PAQ8PX-725719398.75 0.0998
002WinRK 3.1.2MAX (PWCM)27162898.68 0.1054
003PAQAR 4.5-831518298.47 0.1223
004SLIM 0.23d-o41 -m400 -tt-33889498.36 0.1315
005DURILCA 0.5-o34 -t2(2)34136898.34 0.1325
006PPMonstr J rev.1-o64 -m91235572298.27 0.1380
007LPAQ8835726398.27 0.1386
008NanoZip 0.07a-cc -m640m36080298.25 0.1400
009EPM r9c04060129800708699197007 -m91236970798.21 0.1435
010ASH 07/o42 /m91238292898.14 0.1486
011Ocamyd 1.66test1-m9 -s038863098.12 0.1508
012RKC 1.02-M912m -o29 -n- -ts39657698.08 0.1539
013ZPAQ 1.09max.cfg 240142998.05 0.1558
014HIPP 0.5819/so14 /o4440543798.03 0.1573
015CCM 1.30cCMMx -642867897.92 0.1663
016CMM4 0.2b5643054497.91 0.1671
017PIMPLE2(none)43575797.89 0.1691
018BIT 0.7-p=543681597.88 0.1695
019PPMY SSE (9A9)/o41 /m92044030497.86 0.1709
020ENC 0.15abe -o24 -d25644291697.85 0.1719
021BEE 0.7.9-m3 -d546238997.76 0.1794
022RK 1.04.1-c -mx1 -M52 -B2000047474497.70 0.1842
023PPMZ 8.3-c9 -l47488397.70 0.1843
024FreeARC 0.60-m5t48755997.64 0.1892
025RINGS 1.61048841097.63 0.1895
026RKUC 1.04-x -o1650062897.57 0.1943
027RZM 0.07h(none)50642797.54 0.1965
028COMPRESSIA 1.0bBS15 SE50691297.54 0.1967
0297-Zip 9.10-m0=ppmd:o=26:mem=2951586597.50 0.2002
030SQUEEZ 5.63-PPM2 -PPMM1024 -PPMO2651614197.50 0.2003
031BWMonstr 0.02(none)51764297.49 0.2009
032Ultra7z Opt 0.05(none)51905297.48 0.2014
033BMA 1.35b-m32m52713897.44 0.2045
034UHBC 1.0-m3 -b128m -d53002297.43 0.2057
035HOOK 1.419253136497.42 0.2062
036SZIP 1.12-b41o2453631097.40 0.2081
037M03BS=32Mb53741897.39 0.2085
038TarsaLZP 21Aug2007(none)54392697.36 0.2111
039BBB ver1fm18054941697.34 0.2132
040DARK 0.51-b128m55190297.32 0.2142
041STUFFIT 14BWT L3 M2355339997.32 0.2147
042MNZIP955505397.31 0.2154
043GRZipII 0.2.4-l -b8m55516797.31 0.2154
044PPMVC 1.2-o16 -m256 -b55752097.30 0.2163
045LZPX(J) 1.2h-955789397.29 0.2165
046BSSC 0.95a-b16383ze55795697.29 0.2165
047Blizzard 0.24b3000000055876097.29 0.2168
048PPMZ2 0.81-e -t56070997.28 0.2176
049BCM 0.10-b10056153197.28 0.2179
050FlashZIP 0.99b5-m2 -c756594297.25 0.2196
051ZZIP 0.36c-40m56594997.25 0.2196
052YBS 0.03f-m16m -v56817297.24 0.2205
053DC 0.99.307b-fb -b1628856882597.24 0.2207
054PPMN 1.00b1 km-O9 -M:10 -MT1 -RLd57117197.23 0.2216
055MARbwt -g57273697.22 0.2222
056BALZ 1.15ex57729197.20 0.2240
057PIM 2.90(none)57730697.20 0.2240
058BWTZIP6000000057938697.19 0.2248
059M99 2.2.1-m 120m57974197.19 0.2250
060WinRAR 3.91-ep -m5 -mdA -mc25:128t+57995197.19 0.2250
061PPMd rev J-o16 -m4258201897.18 0.2258
062ABC 2.4-cv58230997.18 0.2260
063WINZIP 14Best Method58573597.16 0.2273
064BA 1.01-m -5058860997.15 0.2284
065ACB 2.00cu58871797.14 0.2284
066CTXf 0.75 b1-me59444497.12 0.2307
067M1 0.3btext2 profile59621497.11 0.2313
068QLFC 6.6w059767697.10 0.2319
069PPMX 0.05(none)59785597.10 0.2320
070CHILE 0.5b13107260366197.07 0.2342
071ICEOWS 4.20bMaximum60688897.06 0.2355
072Quark 0.95r-l13 -d2661363697.02 0.2381
073Quad 1.12x61970196.99 0.2405
074SBC 0.970 rev3-b42 -ad62375596.97 0.2420
075QAZAR 0.0pre5-d5 -x7 -l763064896.94 0.2447
076UHARC 0.6b-mx -md-63950296.90 0.2481
077QC 0.050-864143196.89 0.2489
078WinHKI 1.74HKI2 Max64175696.89 0.2490
079TC 5.2 dev2(none)64765996.86 0.2513
080SR3a(none)65576696.82 0.2545
081BioArc 1.9Hyper Auto66297996.78 0.2573
082GRZIP 0.7.3(none)66408296.78 0.2577
083RKIVE 1.92-p0 -mb3 -t16384 -b60000 -mm167014896.75 0.2600
084NNTCP667537996.72 0.2621
085BZP 0.3(none)69950496.61 0.2714
086ShipInBottle 1.0 b17alg:ppm mem:116 len:3970159696.60 0.2722
087TRANSFORM 1.02Maximum70522896.58 0.2736
088CTW 0.1-n16M -f16M -d12 -s70585896.58 0.2739
089BOA 0.58b-m570601696.58 0.2740
090WINIMP 1.21M2, Block 1020,SUS 16 Mb, MM71202296.55 0.2763
091IMP 1.12-271497796.53 0.2774
092OrangeArchiver 1.05(none)71690096.52 0.2782
093BZIP 0.21(none)71779896.52 0.2785
094LZTurbo 0.95-5971813596.52 0.2787
095BZIP2 1.0.5-972341596.49 0.2807
096EXP 1.0(none)72348596.49 0.2807
097DACT 0.8.41-b2200000072350496.49 0.2807
098PAR 2.00(none)72357896.49 0.2808
099UFA 0.04b1-m573301396.44 0.2844
100777 0.04b1-m573301396.44 0.2844
101DGCA 1.10(none)73564896.43 0.2855
102WinTurtle 1.6.0(none)74122296.40 0.2876
103PAC 17apr2004comp375514496.34 0.2930
10412Ghosts 7.0(none)76159796.31 0.2955
105DLC 0.6.1(none)76244496.30 0.2958
106PSA 0.91a-o8 -m524288K76295296.30 0.2960
107ARHANGEL 1.40-mo676728896.28 0.2977
108TURTLE 0.07(none)76888796.27 0.2983
109X1 0.95aam6l378479296.19 0.3045
110BruteCM 0.1d(none)78674996.18 0.3053
111BAR 1.1.2(none)80813596.08 0.3136
112BICOM 1.01(none)82130396.02 0.3187
113LZPM 0.16e82882095.98 0.3216
114DST 0.91b-mm -484036295.92 0.3261
115HuffComp 1.3(none)84313795.91 0.3272
116DCGA b8(none)84604895.90 0.3283
117LZXQ 0.4extreme85694595.84 0.3325
118GCA 0.9k(none)86411795.81 0.3353
119WinACE 2.69Max 4096Kb87667495.75 0.3402
120ACE 2.6-m5 -d409687667495.75 0.3402
121QUANTUM 0.97-c7 -t2188690095.70 0.3441
122DeepFreezer 1.06(none)90262695.62 0.3502
123CABARC 1.00.0106-m LZX:2190451395.61 0.3510
124PACKET 0.91a-m6 -s990478395.61 0.3511
125LZ2ASd104857690903495.59 0.3527
126ASD 0.2.0-m6 -mdh91579295.56 0.3554
127LZDS v2.1-s1024 -m692452295.52 0.3587
128Tornado 0.4a-12 -l819293207395.48 0.3617
129JAR 1.02-m493408695.47 0.3625
130THOR 0.96e493501695.46 0.3628
131EXTREME 1.06-t894413595.42 0.3664
132RZIP 2.1-994759395.40 0.3677
133CSC 3.1-m3 -d597856795.25 0.3797
134BIX 1.00b7-mdG97918995.25 0.3800
135PPMT 0.1(none)98516195.22 0.3823
136LGHA 1.1g-2100120995.14 0.3885
137HA 0.999ba21100120995.14 0.3885
138CODEC 3.21-c10102070395.05 0.3961
139SLUG X(none)102202595.04 0.3966
140BVI 1.70-m5103136595.00 0.4002
141Archiver 1.0Dict=16M105763294.87 0.4104
142RDMC 0.06c(none)112153994.56 0.4352
143ARI 2.2(none)113551394.49 0.4406
144YAC 1.02(none)114741594.43 0.4452
145SEMONE 0.6(none)117063094.32 0.4542
146DCA 1.0.1bBest118078494.27 0.4582
147LCW 0.2-l9118878694.23 0.4613
148LZAP 0.20.0b(none)119519094.20 0.4638
149JCALG1 5.32-9120945494.13 0.4693
150LHA32 1.88.3.14-jm4 -je65536120960794.13 0.4694
151LHARK 0.4d-tob121343294.11 0.4708
152UC II v3.05b-tst121529294.11 0.4716
153ZAP32 0.15.0b(none)122126994.08 0.4739
154SYMBRA 0.2-c6 -p2125676793.90 0.4877
155KZIP 14-APR-2007/b400126126793.88 0.4894
156BJWFLATE 1.54(none)129244493.73 0.5015
157LCSSR 0.2-l9131041093.64 0.5085
158SQUEEZE 1.08.4/p1 /q4 /m2131990393.60 0.5122
159LIMIT 1.2-mx132939593.55 0.5158
160ZET 0.10b-eh132963793.55 0.5159
161AMG 2.2Max compression132982993.55 0.5160
162HiP beta 16133083793.54 0.5164
163PKZIP 2.50-exx133172493.54 0.5167
164DZIP 2.90-9133316493.53 0.5173
165vuZIP 1.8Maximum133321793.53 0.5173
166EAZEL 1.0(best)133324793.53 0.5173
167BSA 2.00-+0133581093.52 0.5183
168GZIP 1.3.5-9133791493.51 0.5191
169ZIP 2.2-9133799993.51 0.5192
170WINZIP 8.0(Max Compression)133799993.51 0.5192
171LHA 2.67(none)133806593.51 0.5192
172ESP 1.92(none)134159693.49 0.5206
173AIN 2.32/m1134169493.49 0.5206
174RAX 1.02-m7134281993.49 0.5211
175aPLib 0.43(none)134353393.48 0.5213
176HIT 2.10-x134370393.48 0.5214
177XPA 1.0.2(none)134993693.45 0.5238
178AKT 1.00a3(none)135379993.43 0.5253
179HAP 3.06(none)136094993.40 0.5281
180HYPER 2.5(none)137117793.35 0.5321
181AI 1.1-mt137631093.32 0.5340
182ARJ 2.85-jm -e -jh65535137955293.31 0.5353
183WIN-GZ 1.2(None)144946792.97 0.5624
184File2Pack 2.0(none)144957892.97 0.5625
185 BCArchive 1.08.7(none)147353692.85 0.5718
186Chaos Comp 3.0(none)147911592.83 0.5739
187Windows XP built-in(none)151807592.64 0.5891
188LZA 1.01/C1159377692.27 0.6184
189ALZip 7.0Normal161446692.17 0.6265
190LZC 0.0810162716592.11 0.6314
191SAR 1.0(none)162739392.11 0.6315
192AR 1.0(none)162739392.11 0.6315
193ZOO 2.1ah162753092.11 0.6315
194QuickLZ 1.40b9mode2163256292.08 0.6335
195LZOP 1.02rc1-9168620491.82 0.6543
196Zhuff 0.2(none)185005591.03 0.7179
197Secura 1.7(none)200382890.28 0.7775
198CODER 1.1-ew 20971520200936890.25 0.7797
199ARX 1.0(none)210219289.80 0.8157
200CA-ZIP 3.4-a212611289.69 0.8250
201LZ 1.0(none)215528789.55 0.8363
202QPress 0.38b-L3216626789.49 0.8406
203LZP2 0.7d(none)216730389.49 0.8410
204BigCrunch 0.4a1(none)217246689.46 0.8430
205BriefLZ 1.04(none)222115089.23 0.8619
206LZBW1 0.8(none)241265288.30 0.9362
207SRANK 1.0c8280998986.37 1.0904
208LZRW1(none)491083776.18 1.9055
209ERI 5.1fre(none)533229674.14 2.0691
210SHcodec 1.0.1(none)1400703732.06 5.4351
211Shindlet(none)1403761531.91 5.4470
212fp.log206170710.00 8.0000

Show historic data

Lossless data compression ratio's of the best and some well know compression programs for a large LOG-file.

Next Test Home Previous Test


©2003-2009 MaximumCompression (lossless data compression software benchmarks)