Big log-file compression test


File type : Large traffic log file of a popular website
# of files to compress in this test : 1
Total File Size (bytes) : 20,617,071
Sample of data :

216.154.140.251 - - [03/Feb/2003:04:21:02 +0100] "GET /thumbn/cf104_r.jpg HTTP/1.1" 200 4559 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
216.154.140.251 - - [03/Feb/2003:04:21:07 +0100] "GET /full/mig17.jpg HTTP/1.1" 200 11756 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
68.113.117.251 - - [03/Feb/2003:04:21:20 +0100] "GET /info/f16.htm HTTP/1.1" 200 17687 "http://www.fighter-planes.com/data7080.htm" "Mozilla/4.0 (compatible; MSIE 5.5; MSN 2.5; Windows 98)"
68.113.117.251 - - [03/Feb/2003:04:21:21 +0100] "GET /thumbn/f16_r.jpg HTTP/1.1" 304 - "http://www.fighter-planes.com/info/f16.htm" "Mozilla/4.0 (compatible; MSIE 5.5; MSN 2.5; Windows 98)"
202.156.2.164 - - [03/Feb/2003:04:21:35 +0100] "GET /thumbn/f16_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /data9099.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /thumbn/yf23_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /thumbn/idf_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:41 +0100] "GET /thumbn/su35_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:43 +0100] "GET /info/f22.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
216.154.140.251 - - [03/Feb/2003:04:21:45 +0100] "GET /full/cf105.jpg HTTP/1.1" 200 26803 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
202.156.2.164 - - [03/Feb/2003:04:21:50 +0100] "GET /info/ef2000.htm HTTP/1.1" 200 9066 "http://www.fighter-planes.com/data99.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)"
202.156.2.164 - - [03/Feb/2003:04:22:04 +0100] "GET /info/f18e.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"

Conclusion: Because of the many repeating patterns the LOG-file is the best compressable file in the test-set. Although nearly all compressors are capable of compressing this file by at least 90%, the best program in this test, PAQ8, has a resulting filesize which is 5 times smaller then WinZip 8.0 and more then 3 times smaller then Microsofts CABARC. The resulting archive size with PAQ8 is only 1.25% of the log-files' original size (that's 80 times smaller!!). This test is again dominated by compressors using a PAQ(like) engine.

Number of different compressors/archivers listed in this test: 215


Current Version Previous Version
Pos. Name Compressor Best switch combination Size
(bytes)		 Ratio
(%)		 Bits per Byte (b/B) Name Size
(bytes)		 Delta
(bytes)
001PAQ8PX-725719398.75 0.0998 PAQ8P2649827789
002WinRK 3.1.2MAX (PWCM)27162898.68 0.1054 WinRK 3.0.3271227-401
003PAQAR 4.5-831518298.47 0.1223 PAQAR 4.13189963814
004SLIM 0.23d-o41 -m400 -tt-33889498.36 0.1315 SLIM 00223417622868
005DURILCA 0.5-o34 -t2(2)34136898.34 0.1325 DURILCA 0.4b3447773409
006PPMonstr J rev.1-o64 -m91235572298.27 0.1380 PPMonstr Ir137318417462
007LPAQ8835726398.27 0.1386 LPAQ73621944931
008NanoZip 0.08a-cc -m640m36032698.25 0.1398 NanoZip 0.07a360802476
009EPM r9c04060129800708699197007 -m91236970798.21 0.1435 EPM r83726792972
010ASH 07/o42 /m91238292898.14 0.1486 ASH 06a382353-575
011Ocamyd 1.66test1-m9 -s038863098.12 0.1508 Ocamyd 1.66 final387467-1163
012RKC 1.02-M912m -o29 -n- -ts39657698.08 0.1539    
013ZPAQ 1.10max.cfg 240142998.05 0.1558 ZPAQ 1.0040145728
014HIPP 0.5819/so14 /o4440543798.03 0.1573    
015CCM 1.30cCMMx -642867897.92 0.1663 CCM 1.26b4376178939
016CMM4 0.2b5643054497.91 0.1671 CMM4 0.1e44628615742
017PIMPLE2(none)43575797.89 0.1691 PIMPLE 1.43b555402119645
018BIT 0.7-p=543681597.88 0.1695 BIT 0.3576981140166
019PPMY SSE (9A9)/o41 /m92044030497.86 0.1709    
020ENC 0.15abe -o24 -d25644291697.85 0.1719    
021BEE 0.7.9-m3 -d546238997.76 0.1794 BEE 0.7.7b49590533516
022RK 1.04.1-c -mx1 -M52 -B2000047474497.70 0.1842    
023PPMZ 8.3-c9 -l47488397.70 0.1843    
024RINGS 1.61048841097.63 0.1895 RINGS 1.5c486886-1524
025RKUC 1.04-x -o1650062897.57 0.1943    
026FreeARC 0.666-m5t50241597.56 0.1950 FreeARC 0.60487559-14856
027RZM 0.07h(none)50642797.54 0.1965 RZM 0.0453029723870
028COMPRESSIA 1.0bBS15 SE50691297.54 0.1967    
029BSC 2.2.5-H1051143997.52 0.1985 BSC 1.0.2510721-718
0307-Zip 9.15-m0=ppmd:o=26:mem=2951586597.50 0.2002 7-Zip 9.105158650
031SQUEEZ 5.63-PPM2 -PPMM1024 -PPMO2651614197.50 0.2003 SQUEEZ 4.2965846449705
032BWMonstr 0.02(none)51764297.49 0.2009 BWMonstr 0.015244186776
033Ultra7z Opt 0.05(none)51905297.48 0.2014    
034BMA 1.35b-m32m52713897.44 0.2045 BMA 1.34b58837161233
035UHBC 1.0-m3 -b128m -d53002297.43 0.2057    
036HOOK 1.419253136497.42 0.2062 HOOK 1.354578114417
037SZIP 1.12-b41o2453631097.40 0.2081 SZIP 1.05f533995-2315
038M03BS=32Mb53741897.39 0.2085    
039TarsaLZP 21Aug2007(none)54392697.36 0.2111 TarsaLZP 07Aug2007541908-2018
040BBB ver1fm18054941697.34 0.2132    
041DARK 0.51-b128m55190297.32 0.2142 DARK 0.50c552015113
042STUFFIT 14BWT L3 M2355339997.32 0.2147 STUFFIT 12.05547371338
043MNZIP955505397.31 0.2154    
044GRZipII 0.2.4-l -b8m55516797.31 0.2154    
045PPMVC 1.2-o16 -m256 -b55752097.30 0.2163 PPMVC 1.15643646844
046LZPX(J) 1.2h-955789397.29 0.2165 LZPX(J) 1.2g59196934076
047BSSC 0.95a-b16383ze55795697.29 0.2165 BSSC 0.93a5594591503
048Blizzard 0.24b3000000055876097.29 0.2168    
049BCM 0.11(none)55965897.29 0.2172 BCM 0.105615311873
050PPMZ2 0.81-e -t56070997.28 0.2176 PPMZ2 0.85629932284
051ZZIP 0.36c-40m56594997.25 0.2196    
052YBS 0.03f-m16m -v56817297.24 0.2205    
053DC 0.99.307b-fb -b1628856882597.24 0.2207    
054PPMN 1.00b1 km-O9 -M:10 -MT1 -RLd57117197.23 0.2216    
055MARbwt -g57273697.22 0.2222    
056BALZ 1.15ex57729197.20 0.2240 BALZ 1.13551055-26236
057PIM 2.90(none)57730697.20 0.2240 PIM 2.105820204714
058BWTZIP6000000057938697.19 0.2248    
059M99 2.2.1-m 120m57974197.19 0.2250 M99 2.1579740-1
060WinRAR 3.91-ep -m5 -mdA -mc25:128t+57995197.19 0.2250 WinRAR 3.62579766-185
061FlashZIP 0.99b8-m3 -c758042097.18 0.2252 FlashZIP 0.94566090-14330
062PPMd rev J-o16 -m4258201897.18 0.2258 PPMd I rev 15856093591
063ABC 2.4-cv58230997.18 0.2260    
064WINZIP 14Best Method58573597.16 0.2273 WINZIP 125857350
065BA 1.01-m -5058860997.15 0.2284    
066ACB 2.00cu58871797.14 0.2284    
067CTXf 0.75 b1-me59444497.12 0.2307    
068M1 0.3btext2 profile59621497.11 0.2313 M1 0.1a65525659042
069QLFC 6.6w059767697.10 0.2319    
070PPMX 0.05(none)59785597.10 0.2320    
071CHILE 0.5b13107260366197.07 0.2342 CHILE 0.3d69746193800
072ICEOWS 4.20bMaximum60688897.06 0.2355    
073Quark 0.95r-l13 -d2661363697.02 0.2381 Quark 0.9363820324567
074Quad 1.12x61970196.99 0.2405 Quad 1.07b63263012929
075SBC 0.970 rev3-b42 -ad62375596.97 0.2420    
076QAZAR 0.0pre5-d5 -x7 -l763064896.94 0.2447 QAZAR 0.0pre4d68020449556
077UHARC 0.6b-mx -md-63950296.90 0.2481 UHARC 0.56493949892
078QC 0.050-864143196.89 0.2489 QC 0.03366165320222
079WinHKI 1.74HKI2 Max64175696.89 0.2490 WinHKI 1.3g6417560
080TC 5.2 dev2(none)64765996.86 0.2513 TC 5.1 dev7585245-62414
081SR3a(none)65576696.82 0.2545 SR36557693
082BioArc 1.9Hyper Auto66297996.78 0.2573    
083GRZIP 0.7.3(none)66408296.78 0.2577    
084RKIVE 1.92-p0 -mb3 -t16384 -b60000 -mm167014896.75 0.2600    
085NNTCP667537996.72 0.2621    
086BZP 0.3(none)69950496.61 0.2714    
087ShipInBottle 1.0 b17alg:ppm mem:116 len:3970159696.60 0.2722 ShipInBottle 1.0 b16701594-2
088TRANSFORM 1.02Maximum70522896.58 0.2736    
089CTW 0.1-n16M -f16M -d12 -s70585896.58 0.2739    
090BOA 0.58b-m570601696.58 0.2740    
091WINIMP 1.21M2, Block 1020,SUS 16 Mb, MM71202296.55 0.2763    
092IMP 1.12-271497796.53 0.2774    
093OrangeArchiver 1.05(none)71690096.52 0.2782    
094BZIP 0.21(none)71779896.52 0.2785    
095LZTurbo 0.95-5971813596.52 0.2787 LZTurbo 0.9278902170886
096BZIP2 1.0.5-972341596.49 0.2807 BZIP2 1.0.37234150
097EXP 1.0(none)72348596.49 0.2807    
098DACT 0.8.41-b2200000072350496.49 0.2807    
099PAR 2.00(none)72357896.49 0.2808    
100777 0.04b1-m573301396.44 0.2844    
101UFA 0.04b1-m573301396.44 0.2844    
102DGCA 1.10(none)73564896.43 0.2855 DGCA 1.087356480
103WinTurtle 1.6.0(none)74122296.40 0.2876 WinTurtle 1.3.07462064984
104PAC 17apr2004comp375514496.34 0.2930    
10512Ghosts 7.0(none)76159796.31 0.2955    
106DLC 0.6.1(none)76244496.30 0.2958    
107PSA 0.91a-o8 -m524288K76295296.30 0.2960    
108ARHANGEL 1.40-mo676728896.28 0.2977    
109TURTLE 0.07(none)76888796.27 0.2983    
110X1 0.95aam6l378479296.19 0.3045 X1 0.94h7847920
111BruteCM 0.1d(none)78674996.18 0.3053 BruteCM 0.1a80225115502
112BAR 1.1.2(none)80813596.08 0.3136    
113BICOM 1.01(none)82130396.02 0.3187    
114LZPM 0.16e82882095.98 0.3216 LZPM 0.15608101-220719
115DST 0.91b-mm -484036295.92 0.3261    
116HuffComp 1.3(none)84313795.91 0.3272    
117DCGA b8(none)84604895.90 0.3283    
118LZXQ 0.4extreme85694595.84 0.3325 LZXQ 0.1997853140908
119GCA 0.9k(none)86411795.81 0.3353    
120WinACE 2.69Max 4096Kb87667495.75 0.3402 WINACE 2.6b18766740
121ACE 2.6-m5 -d409687667495.75 0.3402 ACE 2.048766740
122QUANTUM 0.97-c7 -t2188690095.70 0.3441 QUANTUM 0.9691208525185
123DeepFreezer 1.06(none)90262695.62 0.3502    
124CABARC 1.00.0106-m LZX:2190451395.61 0.3510    
125PACKET 0.91a-m6 -s990478395.61 0.3511 PACKET 0.90b9079573174
126LZ2ASd104857690903495.59 0.3527    
127YZX 0.04-m2 -c591372295.57 0.3545    
128ASD 0.2.0-m6 -mdh91579295.56 0.3554 ASD 0.1.529908702075078
129LZDS v2.1-s1024 -m692452295.52 0.3587    
130Tornado 0.4a-12 -l819293207395.48 0.3617 Tornado 0.49345472474
131JAR 1.02-m493408695.47 0.3625    
132THOR 0.96e493501695.46 0.3628 THOR 0.959350160
133EXTREME 1.06-t894413595.42 0.3664    
134RZIP 2.1-994759395.40 0.3677 RZIP 2.0948083490
135BIX 1.00b7-mdG97918995.25 0.3800    
136PPMT 0.1(none)98516195.22 0.3823    
137LGHA 1.1g-2100120995.14 0.3885    
138HA 0.999ba21100120995.14 0.3885    
139CODEC 3.21-c10102070395.05 0.3961    
140SLUG X(none)102202595.04 0.3966 SLUG 1.27b107616854143
141BVI 1.70-m5103136595.00 0.4002    
142Archiver 1.0Dict=16M105763294.87 0.4104    
143CSC 3.2a6-m3 -d20 -fo110317994.65 0.4281 CSC 3.1978567-124612
144RDMC 0.06c(none)112153994.56 0.4352    
145ARI 2.2(none)113551394.49 0.4406    
146YAC 1.02(none)114741594.43 0.4452    
147SEMONE 0.6(none)117063094.32 0.4542    
148DCA 1.0.1bBest118078494.27 0.4582    
149LCW 0.2-l9118878694.23 0.4613    
150LZAP 0.20.0b(none)119519094.20 0.4638    
151JCALG1 5.32-9120945494.13 0.4693    
152LHA32 1.88.3.14-jm4 -je65536120960794.13 0.4694    
153LHARK 0.4d-tob121343294.11 0.4708    
154UC II v3.05b-tst121529294.11 0.4716    
155ZAP32 0.15.0b(none)122126994.08 0.4739    
156ULZ 0.0.2c6125510593.91 0.4870    
157SYMBRA 0.2-c6 -p2125676793.90 0.4877    
158KZIP 14-APR-2007/b400126126793.88 0.4894 KZIP 11-OCT-200612612670
159BJWFLATE 1.54(none)129244493.73 0.5015    
160LCSSR 0.2-l9131041093.64 0.5085    
161SQUEEZE 1.08.4/p1 /q4 /m2131990393.60 0.5122    
162LIMIT 1.2-mx132939593.55 0.5158    
163ZET 0.10b-eh132963793.55 0.5159    
164AMG 2.2Max compression132982993.55 0.5160    
165HiP beta 16133083793.54 0.5164    
166PKZIP 2.50-exx133172493.54 0.5167 PKZIP 2.0613387837059
167DZIP 2.90-9133316493.53 0.5173    
168vuZIP 1.8Maximum133321793.53 0.5173    
169EAZEL 1.0(best)133324793.53 0.5173    
170BSA 2.00-+0133581093.52 0.5183    
171GZIP 1.3.5-9133791493.51 0.5191 GZIP 1.2.41338486572
172WINZIP 8.0(Max Compression)133799993.51 0.5192    
173ZIP 2.2-9133799993.51 0.5192    
174LHA 2.67(none)133806593.51 0.5192    
175ESP 1.92(none)134159693.49 0.5206 ESP 1.913415960
176AIN 2.32/m1134169493.49 0.5206    
177RAX 1.02-m7134281993.49 0.5211    
178aPLib 0.43(none)134353393.48 0.5213    
179HIT 2.10-x134370393.48 0.5214    
180XPA 1.0.2(none)134993693.45 0.5238    
181Etincelle RC2(none)135021793.45 0.5239 Etincelle beta413517141497
182AKT 1.00a3(none)135379993.43 0.5253    
183HAP 3.06(none)136094993.40 0.5281    
184HYPER 2.5(none)137117793.35 0.5321    
185AI 1.1-mt137631093.32 0.5340    
186ARJ 2.85-jm -e -jh65535137955293.31 0.5353 ARJ 2.82b11379532-20
187WIN-GZ 1.2(None)144946792.97 0.5624    
188File2Pack 2.0(none)144957892.97 0.5625    
189 BCArchive 1.08.7(none)147353692.85 0.5718 BCArchive 1.00b14735360
190Chaos Comp 3.0(none)147911592.83 0.5739    
191Windows XP built-in(none)151807592.64 0.5891    
192LZA 1.01/C1159377692.27 0.6184    
193ALZip 7.0Normal161446692.17 0.6265 ALZip 6.3216144660
194LZC 0.0810162716592.11 0.6314 LZC 0.0616271650
195AR 1.0(none)162739392.11 0.6315    
196SAR 1.0(none)162739392.11 0.6315    
197ZOO 2.1ah162753092.11 0.6315    
198QuickLZ 1.40b9mode2163256292.08 0.6335 QuickLZ 1.40b6166258330021
199LZOP 1.02rc1-9168620491.82 0.6543    
200Zhuff 0.2(none)185005591.03 0.7179    
201Secura 1.7(none)200382890.28 0.7775    
202CODER 1.1-ew 20971520200936890.25 0.7797    
203ARX 1.0(none)210219289.80 0.8157    
204CA-ZIP 3.4-a212611289.69 0.8250    
205LZ 1.0(none)215528789.55 0.8363    
206QPress 0.38b-L3216626789.49 0.8406 QPress 0.35b21662670
207LZP2 0.7d(none)216730389.49 0.8410 LZP22434562267259
208BigCrunch 0.4a1(none)217246689.46 0.8430    
209BriefLZ 1.04(none)222115089.23 0.8619    
210LZBW1 0.8(none)241265288.30 0.9362    
211SRANK 1.0c8280998986.37 1.0904    
212LZRW1(none)491083776.18 1.9055    
213ERI 5.1fre(none)533229674.14 2.0691    
214SHcodec 1.0.1(none)1400703732.06 5.4351    
215Shindlet(none)1403761531.91 5.4470    
216fp.log206170710.00 8.0000    

Hide historic data

Lossless data compression ratio's of the best and some well know compression programs for a large LOG-file.

Next Test Home Previous Test


©2003-2009 MaximumCompression (lossless data compression software benchmarks)