Big log-file compression test


File type : Large traffic log file of a popular website
# of files to compress in this test : 1
Total File Size (bytes) : 20,617,071
Sample of data :

216.154.140.251 - - [03/Feb/2003:04:21:02 +0100] "GET /thumbn/cf104_r.jpg HTTP/1.1" 200 4559 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
216.154.140.251 - - [03/Feb/2003:04:21:07 +0100] "GET /full/mig17.jpg HTTP/1.1" 200 11756 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
68.113.117.251 - - [03/Feb/2003:04:21:20 +0100] "GET /info/f16.htm HTTP/1.1" 200 17687 "http://www.fighter-planes.com/data7080.htm" "Mozilla/4.0 (compatible; MSIE 5.5; MSN 2.5; Windows 98)"
68.113.117.251 - - [03/Feb/2003:04:21:21 +0100] "GET /thumbn/f16_r.jpg HTTP/1.1" 304 - "http://www.fighter-planes.com/info/f16.htm" "Mozilla/4.0 (compatible; MSIE 5.5; MSN 2.5; Windows 98)"
202.156.2.164 - - [03/Feb/2003:04:21:35 +0100] "GET /thumbn/f16_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /data9099.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /thumbn/yf23_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /thumbn/idf_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:41 +0100] "GET /thumbn/su35_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:43 +0100] "GET /info/f22.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
216.154.140.251 - - [03/Feb/2003:04:21:45 +0100] "GET /full/cf105.jpg HTTP/1.1" 200 26803 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
202.156.2.164 - - [03/Feb/2003:04:21:50 +0100] "GET /info/ef2000.htm HTTP/1.1" 200 9066 "http://www.fighter-planes.com/data99.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)"
202.156.2.164 - - [03/Feb/2003:04:22:04 +0100] "GET /info/f18e.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"

Conclusion: Because of the many repeating patterns the LOG-file is the best compressable file in the test-set. Although nearly all compressors are capable of compressing this file by at least 90%, the best program in this test, PAQ8, has a resulting filesize which is 5 times smaller then WinZip 8.0 and more then 3 times smaller then Microsofts CABARC. The resulting archive size with PAQ8 is only 1.25% of the log-files' original size (that's 80 times smaller!!). This test is again dominated by compressors using a PAQ(like) engine.

Number of different compressors/archivers listed in this test: 219


Pos. Name Compressor Best switches combination Compressed Compress Bits per
  Size Ratio Byte
  (bytes) (%) (b/B)
001PAQ8PX-725719398.75 0.0998
002WinRK 3.1.2MAX (PWCM)27162898.68 0.1054
003PAQAR 4.5-831518298.47 0.1223
004SLIM 0.23d-o41 -m400 -tt-33889498.36 0.1315
005DURILCA 0.5-o34 -t2(2)34136898.34 0.1325
006PPMonstr J rev.1-o64 -m91235572298.27 0.1380
007LPAQ8835726398.27 0.1386
008NanoZip 0.09a-cc -m900m35883098.26 0.1392
009EPM r9c04060129800708699197007 -m91236970798.21 0.1435
010ASH 07/o42 /m91238292898.14 0.1486
011Ocamyd 1.66test1-m9 -s038863098.12 0.1508
012RKC 1.02-M912m -o29 -n- -ts39657698.08 0.1539
013ZPAQ 2.05max.cfg 240142998.05 0.1558
014HIPP 0.5819/so14 /o4440543798.03 0.1573
015CCM 1.30cCMMx -642867897.92 0.1663
016CMM4 0.2b5643054497.91 0.1671
017PIMPLE2(none)43575797.89 0.1691
018BIT 0.7-p=543681597.88 0.1695
019PPMY SSE (9A9)/o41 /m92044030497.86 0.1709
020ENC 0.15abe -o24 -d25644291697.85 0.1719
021BEE 0.7.9-m3 -d546238997.76 0.1794
022RK 1.04.1-c -mx1 -M52 -B2000047474497.70 0.1842
023PPMZ 8.3-c9 -l47488397.70 0.1843
024RINGS 1.61048841097.63 0.1895
025RKUC 1.04-x -o1650062897.57 0.1943
026FreeARC 0.666-m5t50241597.56 0.1950
027RZM 0.07h(none)50642797.54 0.1965
028COMPRESSIA 1.0bBS15 SE50691297.54 0.1967
029BSC 3.0.0-H1051153497.52 0.1985
0307-Zip 9.25a-m0=ppmd:o=26:mem=2951586597.50 0.2002
031SQUEEZ 5.63-PPM2 -PPMM1024 -PPMO2651614197.50 0.2003
032BWMonstr 0.02(none)51764297.49 0.2009
033Ultra7z Opt 0.05(none)51905297.48 0.2014
034BMA 1.35b-m32m52713897.44 0.2045
035UHBC 1.0-m3 -b128m -d53002297.43 0.2057
036HOOK 1.419253136497.42 0.2062
037SZIP 1.12-b41o2453631097.40 0.2081
038M03BS=32Mb53741897.39 0.2085
039TarsaLZP 21Aug2007(none)54392697.36 0.2111
040BBB ver1fm18054941697.34 0.2132
041DARK 0.51-b128m55190297.32 0.2142
042STUFFIT 14BWT L3 M2355339997.32 0.2147
043MNZIP955505397.31 0.2154
044GRZipII 0.2.4-l -b8m55516797.31 0.2154
045SCM 0.0.1b(none)55639197.30 0.2159
046PPMVC 1.2-o16 -m256 -b55752097.30 0.2163
047LZPX(J) 1.2h-955789397.29 0.2165
048BSSC 0.95a-b16383ze55795697.29 0.2165
049BCM 0.12(none)55832897.29 0.2166
050Blizzard 0.24b3000000055876097.29 0.2168
051PPMZ2 0.81-e -t56070997.28 0.2176
052ZZIP 0.36c-40m56594997.25 0.2196
053YBS 0.03f-m16m -v56817297.24 0.2205
054DC 0.99.307b-fb -b1628856882597.24 0.2207
055PPMN 1.00b1 km-O9 -M:10 -MT1 -RLd57117197.23 0.2216
056MARbwt -g57273697.22 0.2222
057BALZ 1.15ex57729197.20 0.2240
058PIM 2.90(none)57730697.20 0.2240
059BWTZIP6000000057938697.19 0.2248
060M99 2.2.1-m 120m57974197.19 0.2250
061WinRAR 4.1b3-ep -m5 -mdA -mc25:128t+57995197.19 0.2250
062FlashZIP 0.99b8-m3 -c758042097.18 0.2252
063PPMd rev J-o16 -m4258201897.18 0.2258
064ABC 2.4-cv58230997.18 0.2260
065WINZIP 14Best Method58573597.16 0.2273
066BA 1.01-m -5058860997.15 0.2284
067ACB 2.00cu58871797.14 0.2284
068CTXf 0.75 b1-me59444497.12 0.2307
069M1 0.3btext2 profile59621497.11 0.2313
070QLFC 6.6w059767697.10 0.2319
071CHILE 0.5b13107260366197.07 0.2342
072ICEOWS 4.20bMaximum60688897.06 0.2355
073Quark 0.95r-l13 -d2661363697.02 0.2381
074Quad 1.12x61970196.99 0.2405
075SBC 0.970 rev3-b42 -ad62375596.97 0.2420
076QAZAR 0.0pre5-d5 -x7 -l763064896.94 0.2447
077UHARC 0.6b-mx -md-63950296.90 0.2481
078QC 0.050-864143196.89 0.2489
079WinHKI 1.74HKI2 Max64175696.89 0.2490
080TC 5.2 dev2(none)64765996.86 0.2513
081SR3a(none)65576696.82 0.2545
082BioArc 1.9Hyper Auto66297996.78 0.2573
083GRZIP 0.7.3(none)66408296.78 0.2577
084RKIVE 1.92-p0 -mb3 -t16384 -b60000 -mm167014896.75 0.2600
085NNTCP667537996.72 0.2621
086PPMX 0.07(none)69564396.63 0.2699
087BZP 0.3(none)69950496.61 0.2714
088ShipInBottle 1.0 b17alg:ppm mem:116 len:3970159696.60 0.2722
089TRANSFORM 1.02Maximum70522896.58 0.2736
090CTW 0.1-n16M -f16M -d12 -s70585896.58 0.2739
091BOA 0.58b-m570601696.58 0.2740
092WINIMP 1.21M2, Block 1020,SUS 16 Mb, MM71202296.55 0.2763
093IMP 1.12-271497796.53 0.2774
094OrangeArchiver 1.05(none)71690096.52 0.2782
095BZIP 0.21(none)71779896.52 0.2785
096LZTurbo 0.95-5971813596.52 0.2787
097BZIP2 1.0.5-972341596.49 0.2807
098EXP 1.0(none)72348596.49 0.2807
099DACT 0.8.42-b2200000072348996.49 0.2807
100PAR 2.00(none)72357896.49 0.2808
101777 0.04b1-m573301396.44 0.2844
102UFA 0.04b1-m573301396.44 0.2844
103DGCA 1.10(none)73564896.43 0.2855
104WinTurtle 1.6.0(none)74122296.40 0.2876
105PAC 17apr2004comp375514496.34 0.2930
10612Ghosts 7.0(none)76159796.31 0.2955
107DLC 0.6.1(none)76244496.30 0.2958
108PSA 0.91a-o8 -m524288K76295296.30 0.2960
109ARHANGEL 1.40-mo676728896.28 0.2977
110TURTLE 0.07(none)76888796.27 0.2983
111X1 0.95aam6l378479296.19 0.3045
112BruteCM 0.1d(none)78674996.18 0.3053
113BAR 1.1.2(none)80813596.08 0.3136
114BICOM 1.01(none)82130396.02 0.3187
115XPv5c182688095.99 0.3209
116LZPM 0.16e82882095.98 0.3216
117DST 0.91b-mm -484036295.92 0.3261
118HuffComp 1.3(none)84313795.91 0.3272
119DCGA b8(none)84604895.90 0.3283
120LZXQ 0.4extreme85694595.84 0.3325
121GCA 0.9k(none)86411795.81 0.3353
122ACE 2.6-m5 -d409687667495.75 0.3402
123WinACE 2.69Max 4096Kb87667495.75 0.3402
124QUANTUM 0.97-c7 -t2188690095.70 0.3441
125DeepFreezer 1.06(none)90262695.62 0.3502
126CABARC 1.00.0106-m LZX:2190451395.61 0.3510
127PACKET 0.91a-m6 -s990478395.61 0.3511
128LZ2ASd104857690903495.59 0.3527
129YZX 0.04-m2 -c591372295.57 0.3545
130ASD 0.2.0-m6 -mdh91579295.56 0.3554
131LZDS v2.1-s1024 -m692452295.52 0.3587
132Tornado 0.4a-12 -l819293207395.48 0.3617
133JAR 1.02-m493408695.47 0.3625
134THOR 0.96e493501695.46 0.3628
135EXTREME 1.06-t894413595.42 0.3664
136RZIP 2.1-994759395.40 0.3677
137BIX 1.00b7-mdG97918995.25 0.3800
138Comprox 0.3.0e2098187895.24 0.3810
139PPMT 0.1(none)98516195.22 0.3823
140LGHA 1.1g-2100120995.14 0.3885
141HA 0.999ba21100120995.14 0.3885
142CODEC 3.21-c10102070395.05 0.3961
143SLUG X(none)102202595.04 0.3966
144BVI 1.70-m5103136595.00 0.4002
145Archiver 1.0Dict=16M105763294.87 0.4104
146CSC 3.2a6-m3 -d20 -fo110317994.65 0.4281
147RDMC 0.06c(none)112153994.56 0.4352
148ARI 2.2(none)113551394.49 0.4406
149YAC 1.02(none)114741594.43 0.4452
150Crush 0.01cf117057594.32 0.4542
151SEMONE 0.6(none)117063094.32 0.4542
152DCA 1.0.1bBest118078494.27 0.4582
153LCW 0.2-l9118878694.23 0.4613
154LZAP 0.20.0b(none)119519094.20 0.4638
155JCALG1 5.32-9120945494.13 0.4693
156LHA32 1.88.3.14-jm4 -je65536120960794.13 0.4694
157LHARK 0.4d-tob121343294.11 0.4708
158UC II v3.05b-tst121529294.11 0.4716
159ZAP32 0.15.0b(none)122126994.08 0.4739
160ULZ 0.0.2c6125510593.91 0.4870
161SYMBRA 0.2-c6 -p2125676793.90 0.4877
162KZIP 14-APR-2007/b400126126793.88 0.4894
163BJWFLATE 1.54(none)129244493.73 0.5015
164LCSSR 0.2-l9131041093.64 0.5085
165SQUEEZE 1.08.4/p1 /q4 /m2131990393.60 0.5122
166LIMIT 1.2-mx132939593.55 0.5158
167ZET 0.10b-eh132963793.55 0.5159
168AMG 2.2Max compression132982993.55 0.5160
169HiP beta 16133083793.54 0.5164
170PKZIP 2.50-exx133172493.54 0.5167
171DZIP 2.90-9133316493.53 0.5173
172vuZIP 1.8Maximum133321793.53 0.5173
173EAZEL 1.0(best)133324793.53 0.5173
174BSA 2.00-+0133581093.52 0.5183
175GZIP 1.3.5-9133791493.51 0.5191
176ZIP 2.2-9133799993.51 0.5192
177WINZIP 8.0(Max Compression)133799993.51 0.5192
178LHA 2.67(none)133806593.51 0.5192
179ESP 1.92(none)134159693.49 0.5206
180AIN 2.32/m1134169493.49 0.5206
181RAX 1.02-m7134281993.49 0.5211
182aPLib 0.43(none)134353393.48 0.5213
183HIT 2.10-x134370393.48 0.5214
184XPA 1.0.2(none)134993693.45 0.5238
185Etincelle RC2(none)135021793.45 0.5239
186AKT 1.00a3(none)135379993.43 0.5253
187HAP 3.06(none)136094993.40 0.5281
188HYPER 2.5(none)137117793.35 0.5321
189AI 1.1-mt137631093.32 0.5340
190ARJ 2.85-jm -e -jh65535137955293.31 0.5353
191WIN-GZ 1.2(None)144946792.97 0.5624
192File2Pack 2.0(none)144957892.97 0.5625
193 BCArchive 1.08.7(none)147353692.85 0.5718
194Chaos Comp 3.0(none)147911592.83 0.5739
195Windows XP built-in(none)151807592.64 0.5891
196LZA 1.01/C1159377692.27 0.6184
197ALZip 7.0Normal161446692.17 0.6265
198LZC 0.0810162716592.11 0.6314
199SAR 1.0(none)162739392.11 0.6315
200AR 1.0(none)162739392.11 0.6315
201ZOO 2.1ah162753092.11 0.6315
202QuickLZ 1.40b9mode2163256292.08 0.6335
203LZOP 1.02rc1-9168620491.82 0.6543
204Zhuff 0.2(none)185005591.03 0.7179
205Secura 1.7(none)200382890.28 0.7775
206CODER 1.1-ew 20971520200936890.25 0.7797
207ARX 1.0(none)210219289.80 0.8157
208CA-ZIP 3.4-a212611289.69 0.8250
209LZ 1.0(none)215528789.55 0.8363
210QPress 0.38b-L3216626789.49 0.8406
211LZP2 0.7d(none)216730389.49 0.8410
212BigCrunch 0.4a1(none)217246689.46 0.8430
213BriefLZ 1.04(none)222115089.23 0.8619
214LZBW1 0.8(none)241265288.30 0.9362
215SRANK 1.0c8280998986.37 1.0904
216LZRW1(none)491083776.18 1.9055
217ERI 5.1fre(none)533229674.14 2.0691
218SHcodec 1.0.1(none)1400703732.06 5.4351
219Shindlet(none)1403761531.91 5.4470
220fp.log206170710.00 8.0000

Show historic data

Lossless data compression ratio's of the best and some well know compression programs for a large LOG-file.

Next Test Home Previous Test


©2003-2011 MaximumCompression (lossless data compression software benchmarks)