Big log-file compression test


File type : Large traffic log file of a popular website
# of files to compress in this test : 1
Total File Size (bytes) : 20,617,071
Sample of data :

216.154.140.251 - - [03/Feb/2003:04:21:02 +0100] "GET /thumbn/cf104_r.jpg HTTP/1.1" 200 4559 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
216.154.140.251 - - [03/Feb/2003:04:21:07 +0100] "GET /full/mig17.jpg HTTP/1.1" 200 11756 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
68.113.117.251 - - [03/Feb/2003:04:21:20 +0100] "GET /info/f16.htm HTTP/1.1" 200 17687 "http://www.fighter-planes.com/data7080.htm" "Mozilla/4.0 (compatible; MSIE 5.5; MSN 2.5; Windows 98)"
68.113.117.251 - - [03/Feb/2003:04:21:21 +0100] "GET /thumbn/f16_r.jpg HTTP/1.1" 304 - "http://www.fighter-planes.com/info/f16.htm" "Mozilla/4.0 (compatible; MSIE 5.5; MSN 2.5; Windows 98)"
202.156.2.164 - - [03/Feb/2003:04:21:35 +0100] "GET /thumbn/f16_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /data9099.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /thumbn/yf23_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:40 +0100] "GET /thumbn/idf_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:41 +0100] "GET /thumbn/su35_r.jpg HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
202.156.2.164 - - [03/Feb/2003:04:21:43 +0100] "GET /info/f22.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"
216.154.140.251 - - [03/Feb/2003:04:21:45 +0100] "GET /full/cf105.jpg HTTP/1.1" 200 26803 "http://www.fighter-planes.com/data5060.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95)"
202.156.2.164 - - [03/Feb/2003:04:21:50 +0100] "GET /info/ef2000.htm HTTP/1.1" 200 9066 "http://www.fighter-planes.com/data99.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)"
202.156.2.164 - - [03/Feb/2003:04:22:04 +0100] "GET /info/f18e.htm HTTP/1.1" 304 - "-" "Mozilla/3.01 (compatible;)"

Conclusion: Because of the many repeating patterns the LOG-file is the best compressable file in the test-set. Although nearly all compressors are capable of compressing this file by at least 90%, the best program in this test, PAQ8, has a resulting filesize which is 5 times smaller then WinZip 8.0 and more then 3 times smaller then Microsofts CABARC. The resulting archive size with PAQ8 is only 1.3% of the log-files' original size (that's 77 times smaller!). This test is again dominated by compressors using a PAQ(like) engine.

Number of different compressors/archivers listed in this test: 201


Pos. Name Compressor Best switches combination Compressed Compress Bits per
  Size Ratio Byte
  (bytes) (%) (b/B)
001PAQ8P-726498298.71 0.1028
002WinRK 3.0.3PWCM 912MB27122798.68 0.1052
003PAQAR 4.5-831518298.47 0.1223
004SLIM 0.23d-o41 -m400 -tt-33889498.36 0.1315
005DURILCA 0.5-o34 -t2(2)34136898.34 0.1325
006PPMonstr J rev.1-o64 -m91235572298.27 0.1380
007LPAQ8835726398.27 0.1386
008NanoZip 0.04a-cc -m800m36079698.25 0.1400
009EPM r9c04060129800708699197007 -m91236970798.21 0.1435
010ASH 04a/o43 /s205 /m91237289198.19 0.1447
011Ocamyd 1.66test1-m9 -s038863098.12 0.1508
012RKC 1.02-M912m -o29 -n- -ts39657698.08 0.1539
013HIPP 0.5819/so14 /o4440543798.03 0.1573
014CCM 1.30cCMMx -642867897.92 0.1663
015CMM4 0.2b5643054497.91 0.1671
016PIMPLE2(none)43575797.89 0.1691
017PPMY SSE (9A9)/o41 /m92044030497.86 0.1709
018ENC 0.15abe -o24 -d25644291697.85 0.1719
019BEE 0.7.9-m3 -d546238997.76 0.1794
020RK 1.04.1-c -mx1 -M52 -B2000047474497.70 0.1842
021PPMZ 8.3-c9 -l47488397.70 0.1843
022RINGS 1.5c748688697.64 0.1889
023FreeARC 0.50a-m5t48755997.64 0.1892
024RKUC 1.04-x -o1650062897.57 0.1943
025RZM 0.07h(none)50642797.54 0.1965
026COMPRESSIA 1.0bBS15 SE50691297.54 0.1967
0277-Zip 4.60b-t7z -m0=ppmd:o=26:mem=2951586597.50 0.2002
028SQUEEZ 5.5-PPM2 -PPMM1024 -PPMO2651614197.50 0.2003
029BMA 1.35b-m32m52713897.44 0.2045
030UHBC 1.0-m3 -b128m -d53002297.43 0.2057
031SZIP 1.12-b41o2453631097.40 0.2081
032M03BS=32Mb53741897.39 0.2085
033TarsaLZP 21Aug2007(none)54392697.36 0.2111
034HOOK 1.319254578197.35 0.2118
035BBB ver1fm18054941697.34 0.2132
036BALZ 1.13ex55105597.33 0.2138
037DARK 0.51-b128m55190297.32 0.2142
038STUFFIT 12.0Method 3, M=2555473797.31 0.2153
039MNZIP955505397.31 0.2154
040GRZipII 0.2.4-l -b8m55516797.31 0.2154
041PPMVC 1.2-o16 -m256 -b55752097.30 0.2163
042LZPX(J) 1.2h-955789397.29 0.2165
043BSSC 0.95a-b16383ze55795697.29 0.2165
044Blizzard 0.24b3000000055876097.29 0.2168
045PPMZ2 0.81-e -t56070997.28 0.2176
046FlashZIP 0.91b-m2 -s756553097.26 0.2194
047ZZIP 0.36c-40m56594997.25 0.2196
048YBS 0.03f-m16m -v56817297.24 0.2205
049DC 0.99.307b-fb -b1628856883297.24 0.2207
050PPMN 1.00b1 km-O9 -M:10 -MT1 -RLd57117197.23 0.2216
051MARbwt -g57273697.22 0.2222
052BIT 0.3-m lwcx -p best57698197.20 0.2239
053BWTZIP6000000057938697.19 0.2248
054M99 2.2.1-m 120m57974197.19 0.2250
055WinRAR 3.80b5-ep -m5 -mdA -mc25:128t+57995197.19 0.2250
056PPMd rev J-o16 -m4258201897.18 0.2258
057PIM 2.10(none)58202097.18 0.2258
058ABC 2.4-cv58230997.18 0.2260
059WINZIP 11.0PPMd58573597.16 0.2273
060BA 1.01-m -5058860997.15 0.2284
061ACB 2.00cu58871797.14 0.2284
062CTXf 0.75 b1-me59444497.12 0.2307
063QLFC 6.6w059767697.10 0.2319
064CHILE 0.5b13107260366197.07 0.2342
065ICEOWS 4.20bMaximum60688897.06 0.2355
066LZPM 0.15960810197.05 0.2360
067Quark 0.95r-l13 -d2661363697.02 0.2381
068Quad 1.12x61970196.99 0.2405
069SBC 0.970 rev3-b42 -ad62375596.97 0.2420
070QAZAR 0.0pre5-d5 -x7 -l763064896.94 0.2447
071UHARC 0.6b-mx -md-63950296.90 0.2481
072QC 0.050-864143196.89 0.2489
073WinHKI 1.74HKI2 Max64175696.89 0.2490
074TC 5.2 dev2(none)64765996.86 0.2513
075M1 0.1apaq65525696.82 0.2543
076SR3a(none)65576696.82 0.2545
077BioArc 1.9Hyper Auto66297996.78 0.2573
078GRZIP 0.7.3(none)66408296.78 0.2577
079RKIVE 1.92-p0 -mb3 -t16384 -b60000 -mm167014896.75 0.2600
080NNTCP667537996.72 0.2621
081BZP 0.3(none)69950496.61 0.2714
082ShipInBottle 1.0 b17alg:ppm mem:116 len:3970159696.60 0.2722
083TRANSFORM 1.02Maximum70522896.58 0.2736
084CTW 0.1-n16M -f16M -d12 -s70585896.58 0.2739
085BOA 0.58b-m570601696.58 0.2740
086WINIMP 1.21M2, Block 1020,SUS 16 Mb, MM71202296.55 0.2763
087IMP 1.12-271497796.53 0.2774
088OrangeArchiver 1.05(none)71690096.52 0.2782
089BZIP 0.21(none)71779896.52 0.2785
090BZIP2 1.0.5-972341596.49 0.2807
091EXP 1.0(none)72348596.49 0.2807
092DACT 0.8.41-b2200000072350496.49 0.2807
093PAR 2.00(none)72357896.49 0.2808
094UFA 0.04b1-m573301396.44 0.2844
095777 0.04b1-m573301396.44 0.2844
096DGCA 1.10(none)73564896.43 0.2855
097WinTurtle 1.6.0(none)74122296.40 0.2876
098PAC 17apr2004comp375514496.34 0.2930
09912Ghosts 7.0(none)76159796.31 0.2955
100DLC 0.6.1(none)76244496.30 0.2958
101PSA 0.91a-o8 -m524288K76295296.30 0.2960
102ARHANGEL 1.40-mo676728896.28 0.2977
103TURTLE 0.07(none)76888796.27 0.2983
104X1 0.95aam6l378479296.19 0.3045
105BruteCM 0.1d(none)78674996.18 0.3053
106LZTurbo 0.92-5978902196.17 0.3062
107BAR 1.1.2(none)80813596.08 0.3136
108BICOM 1.01(none)82130396.02 0.3187
109DST 0.91b-mm -484036295.92 0.3261
110HuffComp 1.3(none)84313795.91 0.3272
111DCGA b8(none)84604895.90 0.3283
112LZXQ 0.4extreme85694595.84 0.3325
113GCA 0.9k(none)86411795.81 0.3353
114ACE 2.04-m5 -d409687667495.75 0.3402
115WinACE 2.69Max 4096Kb87667495.75 0.3402
116QUANTUM 0.97-c7 -t2188690095.70 0.3441
117DeepFreezer 1.06(none)90262695.62 0.3502
118CABARC 1.00.0106-m LZX:2190451395.61 0.3510
119PACKET 0.90b-m4 -s990795795.60 0.3523
120LZ2ASd104857690903495.59 0.3527
121ASD 0.2.0-m6 -mdh91579295.56 0.3554
122LZDS v2.1-s1024 -m692452295.52 0.3587
123Tornado 0.4a-12 -l819293207395.48 0.3617
124JAR 1.02-m493408695.47 0.3625
125THOR 0.96e493501695.46 0.3628
126EXTREME 1.06-t894413595.42 0.3664
127RZIP 2.1-994759395.40 0.3677
128BIX 1.00b7-mdG97918995.25 0.3800
129PPMT 0.1(none)98516195.22 0.3823
130HA 0.999ba21100120995.14 0.3885
131LGHA 1.1g-2100120995.14 0.3885
132CODEC 3.21-c10102070395.05 0.3961
133BVI 1.70-m5103136595.00 0.4002
134Archiver 1.0Dict=16M105763294.87 0.4104
135SLUG 1.27b(none)107616894.78 0.4176
136RDMC 0.06c(none)112153994.56 0.4352
137ARI 2.2(none)113551394.49 0.4406
138YAC 1.02(none)114741594.43 0.4452
139SEMONE 0.6(none)117063094.32 0.4542
140DCA 1.0.1bBest118078494.27 0.4582
141LCW 0.2-l9118878694.23 0.4613
142LZAP 0.20.0b(none)119519094.20 0.4638
143JCALG1 5.32-9120945494.13 0.4693
144LHA32 1.88.3.14-jm4 -je65536120960794.13 0.4694
145LHARK 0.4d-tob121343294.11 0.4708
146UC II v3.05b-tst121529294.11 0.4716
147ZAP32 0.15.0b(none)122126994.08 0.4739
148SYMBRA 0.2-c6 -p2125676793.90 0.4877
149KZIP 14-APR-2007/b400126126793.88 0.4894
150BJWFLATE 1.54(none)129244493.73 0.5015
151LCSSR 0.2-l9131041093.64 0.5085
152SQUEEZE 1.08.4/p1 /q4 /m2131990393.60 0.5122
153LIMIT 1.2-mx132939593.55 0.5158
154ZET 0.10b-eh132963793.55 0.5159
155AMG 2.2Max compression132982993.55 0.5160
156HiP beta 16133083793.54 0.5164
157PKZIP 2.50-exx133172493.54 0.5167
158DZIP 2.90-9133316493.53 0.5173
159vuZIP 1.8Maximum133321793.53 0.5173
160EAZEL 1.0(best)133324793.53 0.5173
161BSA 2.00-+0133581093.52 0.5183
162GZIP 1.3.5-9133791493.51 0.5191
163ZIP 2.2-9133799993.51 0.5192
164WINZIP 8.0(Max Compression)133799993.51 0.5192
165LHA 2.67(none)133806593.51 0.5192
166ESP 1.92(none)134159693.49 0.5206
167AIN 2.32/m1134169493.49 0.5206
168RAX 1.02-m7134281993.49 0.5211
169aPLib 0.43(none)134353393.48 0.5213
170HIT 2.10-x134370393.48 0.5214
171XPA 1.0.2(none)134993693.45 0.5238
172AKT 1.00a3(none)135379993.43 0.5253
173HAP 3.06(none)136094993.40 0.5281
174HYPER 2.5(none)137117793.35 0.5321
175AI 1.1-mt137631093.32 0.5340
176ARJ 2.84-jm -e -jh65535137955793.31 0.5353
177WIN-GZ 1.2(None)144946792.97 0.5624
178File2Pack 2.0(none)144957892.97 0.5625
179 BCArchive 1.08.7(none)147353692.85 0.5718
180Chaos Comp 3.0(none)147911592.83 0.5739
181Windows XP built-in(none)151807592.64 0.5891
182LZA 1.01/C1159377692.27 0.6184
183ALZip 7.0Normal161446692.17 0.6265
184LZC 0.0810162716592.11 0.6314
185AR 1.0(none)162739392.11 0.6315
186SAR 1.0(none)162739392.11 0.6315
187ZOO 2.1ah162753092.11 0.6315
188QuickLZ 1.40b9mode2163256292.08 0.6335
189LZOP 1.02rc1-9168620491.82 0.6543
190Secura 1.7(none)200382890.28 0.7775
191CODER 1.1-ew 20971520200936890.25 0.7797
192ARX 1.0(none)210219289.80 0.8157
193CA-ZIP 3.4-a212611289.69 0.8250
194LZ 1.0(none)215528789.55 0.8363
195BigCrunch 0.4a1(none)217246689.46 0.8430
196BriefLZ 1.04(none)222115089.23 0.8619
197SRANK 1.0c8280998986.37 1.0904
198LZRW1(none)491083776.18 1.9055
199ERI 5.1fre(none)533229674.14 2.0691
200SHcodec 1.0.1(none)1400703732.06 5.4351
201Shindlet(none)1403761531.91 5.4470
202fp.log206170710.00 8.0000

Show historic data

Lossless data compression ratio's of the best and some well know compression programs for a large LOG-file.

Next Test Home Previous Test


©2003-2008 MaximumCompression (lossless data compression software benchmarks)