Big TXT high redundancy file - 692MB

http://static.wikipedia.org/downloads/2008-06/en/html.lst

Someone has the courage to try it?

http://static.wikipedia.org/downloads/2008-06/en/wikipedia-en-html.tar.7z

Hi,

I haven't been able to find results for the parallel gzip 'pigz' or the parallel 'bzip2' programs.

Now that you have a quad core processor would it be possible to run http://www.zlib.net/pigz/ and http://compression.ca/pbzip2/ ?

thanks

Hello Werner, please don't forget about http://www.maximumcompression.com/data/summary_sf_hist.php

yeah that's my fault, i just forgot that paq8h used a dictionary which later versions didn't. -_-!!!

> hey guys, there seems to be too many paq8 forks
> recently, this is getting confusing so maybe someone
> should make a version tree diagram, and then attempt to
> combine the improvements together.

As written before paq8q is a perfect combination of all improvements in functional direction.
The reason why there still are work on more then this branch are others. You want to continue your own releases and you may don't like the additional features paq8q has. I don't understand this but I respect it .

> i just looked at matt's wiki compression test, and
> noticed that the recent paq8p's text compression has
> degraded significantly than old paq8h. i know optimizing
> for some data formats would result in degrading in
> other formats, but this difference seems too much.

It looks so but if you look closer you are going to notice that also text compression is getting better with almost every new release.
Paq8h is much better then paq8px/paq8q yes but only because it uses a dictionary for compression and that's something you most likely never will see in a normal paq version that's not only optimized for text compression / the hutter prize. But still text compression can be improved in the future.

hey guys, there seems to be too many paq8 forks recently, this is getting confusing so maybe someone should make a version tree diagram, and then attempt to combine the improvements together.
and i just looked at matt's wiki compression test, and noticed that the recent paq8p's text compression has degraded significantly than old paq8h. i know optimizing for some data formats would result in degrading in other formats, but this difference seems too much.

Hi folks!

I've started rewriting TarsaLZP to Java (current snapshot is here - http://www.ii.uj.edu.pl/~tarsa/jTarsaLZP.7z - it's folder with NetBeans project, main .jar file is in /dist folder).

I'm a bit upset that encode.ru forums have disabled registration, because I don't have a working account there. Is anyone able to help me with that issue?

My email address is contained in .txt file in: http://asembler.republika.pl/bin/TarsaLZP.zip

I found the problem in paq8q. It's now 1 byte better then paq8px on all files in default mode and should now set a new record on MFC too.

BCM v0.08 - The ultimate BWT-based file compressor!

http://encode.ru/forum/showthread.php?t=382

Paq8q should be able to replace paq8p3 at the moment because it is as fast as or faster as paq8p3 with better compression in the maximum mode.

Jan Ondrus the main developer of paq8px wants to stick to paq8px so I do some improvements on this branch too because it's stupid to work against each other and do fixes on both versions.
It's really surprising that paq8q is sometimes worse then paq8px. This shouldn't be the case and I am going to investigate to find the reason. In the best compression it should be equal.
At the moment it is the best to test paq8px because of nice improvements and if you find time mode 5 or 3 of paq8q.
Hope this helps you. Maximum compression is your idea and interest and this is px .

paq8px improvements + paq8q improvements + paq8p3 == Paq10??

It looks like paq8px, paq8q, and paq8p3 are 3 separate forks. Most of the development seems to be focused on paq8px. I'm not involved in the development. I am also not testing on LTCB right now, as there are new versions coming out faster than I can test.

New version : lzturbo 0.95 parallel compressor.
download: http://lzturbo.web.officelive.com/documents/lzturbo.zip

compress mostly better (Mode -59) and decompress faster than lzma2 (with same or larger block size) on multicore systems.

Remember to re-test Nanozip -cd/cD/co and cO with the -m800m switch.

WinRK the best compression exe, dll, txt, pdf, that use large dictionaries.

http://info.elf.stuba.sk/packages/pub/pc/pack/winrk312.exe

Compacted Instalation file $R0 is ziped dictionaries, and all DLL, and EXE use attachments dictionaries, can be opened with 7zip.

Clearly, a program of 10MB, full of dictionaries and tunnig to the Maximum Compression test, will have more compression than others.

Test the compression of a 2,9MB text file, with a program filled with 10MB of dictionaries, the program file is greater than file tested, or is a fake compression and fake results.

"RAR 3.90 beta 1 compression speed is improved for multi-core and multi-CPU systems. This improvement is most noticeable in Windows Vista and Windows 7 operating systems."

and btw, you still don't tested nz in fast modes and winrk in best asymmetric mode

Thank you. Then most probably your TIFF files are compressed and it loses some bytes here and there (bmp, wav). If you ever going to test mode 5 and or 4 it at least would win some (many) efficient places.

I respect Werner's opinion but disagree, faster modes do have a place in PAQ, very nice work Simon.

Sorry for posting again. I meant PAQ8PX v19

I am aware that some people don't like those changes and maybe this version won't be a success but I don't pay attention to old structures. Paq8p3 was a step in this direction and I built a version which has more "speed techniques" without any loss in the maximum mode.
I admit that it's maybe better that the default mode is -m6 instead of -m5 so that there aren't any changes in the normal usage of PAQ.
If you really test this version you will see that modes 4 and 5 have a BIG time improvement with mostly only a few more kbs in size.
If you still don't like this version it's a pity (for me) but you are free to use PAQ8PX which should provide the same compression on most files.

In my tests there were bigger sizes only on 2-3 files not more. On most it was better. On some files like BMP it comes from a change in later PX versions which improves picture detection a lot and adds only some extra bytes.
That's the reason we now got TIFF, TGA and MOD support relatively easy.

I don't like to do changes and don't change the name. But you could name it only paq8q v2 or whatever .
No, the switch absolutely has to be separated from the memory switch. Why shouldn't it be possible to have faster modes with a high memory usage?

I have finished for now
There are big improvements over the last paq8px tested version on almost all SFC files. Cut the big gap of WinRk in the exe file down to less then 50%.

PAQ8Q has been released.

http://encode.ru/forum/showthread.php?goto=newpost&t=359

I am also very interested in the paq8p3 MFC results.

There is a new paq9px release. Currently paq8px_v15. It adds Tiff compression and should improve MFC if there are uncompressed Tiff files.

David Bryant, the WavPack developer, works for Livescribe, so they have a brilliant dev and an awesome guy in their ranks.