Big log-file compression test


File type : Large traffic log file of a popular website
# of files to compress in this test : 1
Total File Size (bytes) : 20,617,071
Sample of data :

216.154.140.251 - - [03/Feb/2003:04:21:02 +0100] "GET /thumbn/cf104_r.jpg HTTP/1.1" 200 4559 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
216.154.140.251 - - [03/Feb/2003:04:21:07 +0100] "GET /full/mig17.jpg HTTP/1.1" 200 11756 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
68.113.117.251 - - [03/Feb/2003:04:21:20 +0100] "GET /info/f16.htm HTTP/1.1" 200 17687 "http://www.fighter-planes.com/data7080.htm" "Mozilla/4.0 (compatible; MSIE 5.5; MSN 2.5; Windows 98)"
68.113.117.251 - - [03/Feb/2003:04:21:21 +0100] "GET /thumbn/f16_r.jpg HTTP/1.1" 304 - "http://www.fighter-planes.com/info/f16.htm" "Mozilla/4.0 (compatible; MSIE 5.5; MSN 2.5; Windows 98)"
202.156.2.164 - - [03/Feb/2003:04:21:35 +0100] "GET /thumbn/f16_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /data9099.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /thumbn/yf23_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /thumbn/idf_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:41 +0100] "GET /thumbn/su35_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:43 +0100] "GET /info/f22.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
216.154.140.251 - - [03/Feb/2003:04:21:45 +0100] "GET /full/cf105.jpg HTTP/1.1" 200 26803 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
202.156.2.164 - - [03/Feb/2003:04:21:50 +0100] "GET /info/ef2000.htm HTTP/1.1" 200 9066 "http://www.fighter-planes.com/data99.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)"
202.156.2.164 - - [03/Feb/2003:04:22:04 +0100] "GET /info/f18e.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"

Conclusion: Because of the many repeating patterns the LOG-file is the best compressable file in the test-set. Although nearly all compressors are capable of compressing this file by at least 90%, the best program in this test, PAQ8, has a resulting filesize which is 5 times smaller then WinZip 8.0 and more then 3 times smaller then Microsofts CABARC. The resulting archive size with PAQ8 is only 1.25% of the log-files' original size (that's 80 times smaller!!). This test is again dominated by compressors using a PAQ(like) engine.

Number of different compressors/archivers listed in this test: 215


Pos. Name Compressor Best switches combination Compressed Compress Bits per
  Size Ratio Byte
  (bytes) (%) (b/B)
001PAQ8PX-725719398.75 0.0998
002WinRK 3.1.2MAX (PWCM)27162898.68 0.1054
003PAQAR 4.5-831518298.47 0.1223
004SLIM 0.23d-o41 -m400 -tt-33889498.36 0.1315
005DURILCA 0.5-o34 -t2(2)34136898.34 0.1325
006PPMonstr J rev.1-o64 -m91235572298.27 0.1380
007LPAQ8835726398.27 0.1386
008NanoZip 0.08a-cc -m640m36032698.25 0.1398
009EPM r9c04060129800708699197007 -m91236970798.21 0.1435
010ASH 07/o42 /m91238292898.14 0.1486
011Ocamyd 1.66test1-m9 -s038863098.12 0.1508
012RKC 1.02-M912m -o29 -n- -ts39657698.08 0.1539
013ZPAQ 1.10max.cfg 240142998.05 0.1558
014HIPP 0.5819/so14 /o4440543798.03 0.1573
015CCM 1.30cCMMx -642867897.92 0.1663
016CMM4 0.2b5643054497.91 0.1671
017PIMPLE2(none)43575797.89 0.1691
018BIT 0.7-p=543681597.88 0.1695
019PPMY SSE (9A9)/o41 /m92044030497.86 0.1709
020ENC 0.15abe -o24 -d25644291697.85 0.1719
021BEE 0.7.9-m3 -d546238997.76 0.1794
022RK 1.04.1-c -mx1 -M52 -B2000047474497.70 0.1842
023PPMZ 8.3-c9 -l47488397.70 0.1843
024RINGS 1.61048841097.63 0.1895
025RKUC 1.04-x -o1650062897.57 0.1943
026FreeARC 0.666-m5t50241597.56 0.1950
027RZM 0.07h(none)50642797.54 0.1965
028COMPRESSIA 1.0bBS15 SE50691297.54 0.1967
029BSC 2.2.5-H1051143997.52 0.1985
0307-Zip 9.15-m0=ppmd:o=26:mem=2951586597.50 0.2002
031SQUEEZ 5.63-PPM2 -PPMM1024 -PPMO2651614197.50 0.2003
032BWMonstr 0.02(none)51764297.49 0.2009
033Ultra7z Opt 0.05(none)51905297.48 0.2014
034BMA 1.35b-m32m52713897.44 0.2045
035UHBC 1.0-m3 -b128m -d53002297.43 0.2057
036HOOK 1.419253136497.42 0.2062
037SZIP 1.12-b41o2453631097.40 0.2081
038M03BS=32Mb53741897.39 0.2085
039TarsaLZP 21Aug2007(none)54392697.36 0.2111
040BBB ver1fm18054941697.34 0.2132
041DARK 0.51-b128m55190297.32 0.2142
042STUFFIT 14BWT L3 M2355339997.32 0.2147
043MNZIP955505397.31 0.2154
044GRZipII 0.2.4-l -b8m55516797.31 0.2154
045PPMVC 1.2-o16 -m256 -b55752097.30 0.2163
046LZPX(J) 1.2h-955789397.29 0.2165
047BSSC 0.95a-b16383ze55795697.29 0.2165
048Blizzard 0.24b3000000055876097.29 0.2168
049BCM 0.11(none)55965897.29 0.2172
050PPMZ2 0.81-e -t56070997.28 0.2176
051ZZIP 0.36c-40m56594997.25 0.2196
052YBS 0.03f-m16m -v56817297.24 0.2205
053DC 0.99.307b-fb -b1628856882597.24 0.2207
054PPMN 1.00b1 km-O9 -M:10 -MT1 -RLd57117197.23 0.2216
055MARbwt -g57273697.22 0.2222
056BALZ 1.15ex57729197.20 0.2240
057PIM 2.90(none)57730697.20 0.2240
058BWTZIP6000000057938697.19 0.2248
059M99 2.2.1-m 120m57974197.19 0.2250
060WinRAR 3.91-ep -m5 -mdA -mc25:128t+57995197.19 0.2250
061FlashZIP 0.99b8-m3 -c758042097.18 0.2252
062PPMd rev J-o16 -m4258201897.18 0.2258
063ABC 2.4-cv58230997.18 0.2260
064WINZIP 14Best Method58573597.16 0.2273
065BA 1.01-m -5058860997.15 0.2284
066ACB 2.00cu58871797.14 0.2284
067CTXf 0.75 b1-me59444497.12 0.2307
068M1 0.3btext2 profile59621497.11 0.2313
069QLFC 6.6w059767697.10 0.2319
070PPMX 0.05(none)59785597.10 0.2320
071CHILE 0.5b13107260366197.07 0.2342
072ICEOWS 4.20bMaximum60688897.06 0.2355
073Quark 0.95r-l13 -d2661363697.02 0.2381
074Quad 1.12x61970196.99 0.2405
075SBC 0.970 rev3-b42 -ad62375596.97 0.2420
076QAZAR 0.0pre5-d5 -x7 -l763064896.94 0.2447
077UHARC 0.6b-mx -md-63950296.90 0.2481
078QC 0.050-864143196.89 0.2489
079WinHKI 1.74HKI2 Max64175696.89 0.2490
080TC 5.2 dev2(none)64765996.86 0.2513
081SR3a(none)65576696.82 0.2545
082BioArc 1.9Hyper Auto66297996.78 0.2573
083GRZIP 0.7.3(none)66408296.78 0.2577
084RKIVE 1.92-p0 -mb3 -t16384 -b60000 -mm167014896.75 0.2600
085NNTCP667537996.72 0.2621
086BZP 0.3(none)69950496.61 0.2714
087ShipInBottle 1.0 b17alg:ppm mem:116 len:3970159696.60 0.2722
088TRANSFORM 1.02Maximum70522896.58 0.2736
089CTW 0.1-n16M -f16M -d12 -s70585896.58 0.2739
090BOA 0.58b-m570601696.58 0.2740
091WINIMP 1.21M2, Block 1020,SUS 16 Mb, MM71202296.55 0.2763
092IMP 1.12-271497796.53 0.2774
093OrangeArchiver 1.05(none)71690096.52 0.2782
094BZIP 0.21(none)71779896.52 0.2785
095LZTurbo 0.95-5971813596.52 0.2787
096BZIP2 1.0.5-972341596.49 0.2807
097EXP 1.0(none)72348596.49 0.2807
098DACT 0.8.41-b2200000072350496.49 0.2807
099PAR 2.00(none)72357896.49 0.2808
100777 0.04b1-m573301396.44 0.2844
101UFA 0.04b1-m573301396.44 0.2844
102DGCA 1.10(none)73564896.43 0.2855
103WinTurtle 1.6.0(none)74122296.40 0.2876
104PAC 17apr2004comp375514496.34 0.2930
10512Ghosts 7.0(none)76159796.31 0.2955
106DLC 0.6.1(none)76244496.30 0.2958
107PSA 0.91a-o8 -m524288K76295296.30 0.2960
108ARHANGEL 1.40-mo676728896.28 0.2977
109TURTLE 0.07(none)76888796.27 0.2983
110X1 0.95aam6l378479296.19 0.3045
111BruteCM 0.1d(none)78674996.18 0.3053
112BAR 1.1.2(none)80813596.08 0.3136
113BICOM 1.01(none)82130396.02 0.3187
114LZPM 0.16e82882095.98 0.3216
115DST 0.91b-mm -484036295.92 0.3261
116HuffComp 1.3(none)84313795.91 0.3272
117DCGA b8(none)84604895.90 0.3283
118LZXQ 0.4extreme85694595.84 0.3325
119GCA 0.9k(none)86411795.81 0.3353
120WinACE 2.69Max 4096Kb87667495.75 0.3402
121ACE 2.6-m5 -d409687667495.75 0.3402
122QUANTUM 0.97-c7 -t2188690095.70 0.3441
123DeepFreezer 1.06(none)90262695.62 0.3502
124CABARC 1.00.0106-m LZX:2190451395.61 0.3510
125PACKET 0.91a-m6 -s990478395.61 0.3511
126LZ2ASd104857690903495.59 0.3527
127YZX 0.04-m2 -c591372295.57 0.3545
128ASD 0.2.0-m6 -mdh91579295.56 0.3554
129LZDS v2.1-s1024 -m692452295.52 0.3587
130Tornado 0.4a-12 -l819293207395.48 0.3617
131JAR 1.02-m493408695.47 0.3625
132THOR 0.96e493501695.46 0.3628
133EXTREME 1.06-t894413595.42 0.3664
134RZIP 2.1-994759395.40 0.3677
135BIX 1.00b7-mdG97918995.25 0.3800
136PPMT 0.1(none)98516195.22 0.3823
137LGHA 1.1g-2100120995.14 0.3885
138HA 0.999ba21100120995.14 0.3885
139CODEC 3.21-c10102070395.05 0.3961
140SLUG X(none)102202595.04 0.3966
141BVI 1.70-m5103136595.00 0.4002
142Archiver 1.0Dict=16M105763294.87 0.4104
143CSC 3.2a6-m3 -d20 -fo110317994.65 0.4281
144RDMC 0.06c(none)112153994.56 0.4352
145ARI 2.2(none)113551394.49 0.4406
146YAC 1.02(none)114741594.43 0.4452
147SEMONE 0.6(none)117063094.32 0.4542
148DCA 1.0.1bBest118078494.27 0.4582
149LCW 0.2-l9118878694.23 0.4613
150LZAP 0.20.0b(none)119519094.20 0.4638
151JCALG1 5.32-9120945494.13 0.4693
152LHA32 1.88.3.14-jm4 -je65536120960794.13 0.4694
153LHARK 0.4d-tob121343294.11 0.4708
154UC II v3.05b-tst121529294.11 0.4716
155ZAP32 0.15.0b(none)122126994.08 0.4739
156ULZ 0.0.2c6125510593.91 0.4870
157SYMBRA 0.2-c6 -p2125676793.90 0.4877
158KZIP 14-APR-2007/b400126126793.88 0.4894
159BJWFLATE 1.54(none)129244493.73 0.5015
160LCSSR 0.2-l9131041093.64 0.5085
161SQUEEZE 1.08.4/p1 /q4 /m2131990393.60 0.5122
162LIMIT 1.2-mx132939593.55 0.5158
163ZET 0.10b-eh132963793.55 0.5159
164AMG 2.2Max compression132982993.55 0.5160
165HiP beta 16133083793.54 0.5164
166PKZIP 2.50-exx133172493.54 0.5167
167DZIP 2.90-9133316493.53 0.5173
168vuZIP 1.8Maximum133321793.53 0.5173
169EAZEL 1.0(best)133324793.53 0.5173
170BSA 2.00-+0133581093.52 0.5183
171GZIP 1.3.5-9133791493.51 0.5191
172WINZIP 8.0(Max Compression)133799993.51 0.5192
173ZIP 2.2-9133799993.51 0.5192
174LHA 2.67(none)133806593.51 0.5192
175ESP 1.92(none)134159693.49 0.5206
176AIN 2.32/m1134169493.49 0.5206
177RAX 1.02-m7134281993.49 0.5211
178aPLib 0.43(none)134353393.48 0.5213
179HIT 2.10-x134370393.48 0.5214
180XPA 1.0.2(none)134993693.45 0.5238
181Etincelle RC2(none)135021793.45 0.5239
182AKT 1.00a3(none)135379993.43 0.5253
183HAP 3.06(none)136094993.40 0.5281
184HYPER 2.5(none)137117793.35 0.5321
185AI 1.1-mt137631093.32 0.5340
186ARJ 2.85-jm -e -jh65535137955293.31 0.5353
187WIN-GZ 1.2(None)144946792.97 0.5624
188File2Pack 2.0(none)144957892.97 0.5625
189 BCArchive 1.08.7(none)147353692.85 0.5718
190Chaos Comp 3.0(none)147911592.83 0.5739
191Windows XP built-in(none)151807592.64 0.5891
192LZA 1.01/C1159377692.27 0.6184
193ALZip 7.0Normal161446692.17 0.6265
194LZC 0.0810162716592.11 0.6314
195AR 1.0(none)162739392.11 0.6315
196SAR 1.0(none)162739392.11 0.6315
197ZOO 2.1ah162753092.11 0.6315
198QuickLZ 1.40b9mode2163256292.08 0.6335
199LZOP 1.02rc1-9168620491.82 0.6543
200Zhuff 0.2(none)185005591.03 0.7179
201Secura 1.7(none)200382890.28 0.7775
202CODER 1.1-ew 20971520200936890.25 0.7797
203ARX 1.0(none)210219289.80 0.8157
204CA-ZIP 3.4-a212611289.69 0.8250
205LZ 1.0(none)215528789.55 0.8363
206QPress 0.38b-L3216626789.49 0.8406
207LZP2 0.7d(none)216730389.49 0.8410
208BigCrunch 0.4a1(none)217246689.46 0.8430
209BriefLZ 1.04(none)222115089.23 0.8619
210LZBW1 0.8(none)241265288.30 0.9362
211SRANK 1.0c8280998986.37 1.0904
212LZRW1(none)491083776.18 1.9055
213ERI 5.1fre(none)533229674.14 2.0691
214SHcodec 1.0.1(none)1400703732.06 5.4351
215Shindlet(none)1403761531.91 5.4470
216fp.log206170710.00 8.0000

Show historic data

Lossless data compression ratio's of the best and some well know compression programs for a large LOG-file.

Next Test Home Previous Test


©2003-2009 MaximumCompression (lossless data compression software benchmarks)