Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed and compression improvements #119

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

Conversation

fhanau
Copy link
Contributor

@fhanau fhanau commented Jul 18, 2016

This pull request adds some additional improvements from ECT. The first commit is a follow-up of #111 and makes huffman encoding even faster. The second one improves blocksplitting by only decreasing the lengthscore when the match is short.

@jibsen
Copy link
Contributor

jibsen commented Jul 21, 2016

Here are some results on the Silesia compression corpus:

file            original       fhanau
--------------------------------------
dickens        3.673.398    3.669.761*
mozilla       18.317.359   18.314.126*
mr             3.450.243*   3.452.055
nci            2.740.616*   2.740.719
ooffice        2.992.131*   2.992.545
osdb           3.607.235    3.607.233*
reymont        1.692.012    1.691.946*
samba          5.107.606*   5.108.959
sao            5.242.412    5.242.359*
webster       11.518.308   11.516.700*
x-ray          5.744.715*   5.744.863
xml              627.187*     627.231
--------------------------------------
              64.713.222   64.708.497
                1516 sec     1541 sec

It looks like it is an improvement in speed/ratio on some files, and not on others. On average I am getting a small compression gain on the files I tested so far though.

@fhanau
Copy link
Contributor Author

fhanau commented Jul 21, 2016

That's worse than I expected. On my benchmarks(png images and enwik8), it improved both speed and compression.

@fhanau
Copy link
Contributor Author

fhanau commented Aug 27, 2016

With all changes from this pull request, zopfli --i1 is 9% faster(tested on enwik8). d680428 will not influence the compression ratio. Note that the first iteration may be slightly slower.

@fhanau
Copy link
Contributor Author

fhanau commented Oct 18, 2016

15% faster than master.
Changes:

  • Faster cost model through inlining the cost model function and converting it to table lookups
  • Use std::sort instead of C's quicksort for sorting(Note that you need to compile katajainen.c as C++ for the speed improvement. When compiled as C the old sort is used.)

SuperFun99 and others added 3 commits March 15, 2017 14:28
When a small file is stored with a palette, a test is made to see if
removing the palette, and the overhead that goes with it, results in
an even smaller file.  That test was always attempting RGB or RGBA
mode even when the image in question was greyscale.  I added a check
for greyscale.
When checking if an image is smaller without a palette, use
the minimum bit depth for greyscale images instead of
defaulting to 8 bits.
Check for greyscale when small image uses palette
@googlebot
Copy link

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored by someone other than the pull request submitter. We need to confirm that they're okay with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this state. It's up to you to confirm consent of the commit author(s) and merge this pull request when appropriate.

@parasew
Copy link

parasew commented Apr 8, 2023

Are there any updates to this status, or do we have to live with the fact that this will never get merged?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants