Vertical banding with default JPEG DCT method

[electroCutie]

ImageMagick version

ImageMagick 6.9.11-60 Q16 x86_64 2021-01-25

Operating system

Linux

Operating system, version and so on

Xuybuntu 22.10

Description

I noticed this on mastodon first but traced it back to image-magick. The default-used DCT method (not to be confused with the "default"-named method, see below!) causes vertical banding aligned with the 8 pixel wide DCT block that is quite visible to the human eye on some photographs

Steps to Reproduce

Compress a large photograph that has some large smooth areas an a medium amount of ISO noise with quality of 90 or below using no other command line options. Observe vertical banding. Note that smaller images tend to not show this very well.

Not everyone can see it, I seem to be particularly sensitive to it but to be certain I've also written a quick and dirty program to take the FFT of every line of an image and then sum the complex values. It then prints out the wavelengths and magnitudes normalized by the invserse of square-root of wavelength

The reason the magnitudes are normalized like this is because when summing the complex FFT values each value will have a more or less random phase in situations where there is no correlation between lines and so will have an energy spectrum similar to that of thermal noise. By normalizing this way any correlated signal stands out.

And It really does stand out! When doing the FFT on the X axis and summing the rows we get a huge spike at the 8 pixel wavelength

In the other direction (transposed in the test program not by an image editor) we see no similar correlation

The spike goes away when specifying a different DCT method
Here is is worth pointing out that the "default"-named method seems NOT to the default-used method. Image Magick seems to use the "fastest" method by default

Anyway. If I specify the DCT method like this -define jpeg:dct-method=default the spike goes away

Using -define jpeg:dct-method=float gives similarly good results

These graphs were obtained by running this program on the test images https://github.com/electroCutie/vertical_banding and plotting the output with gnu-plot (be sure to zoom in on the early part of the graph)

I don't know if this is an error in the fast-integer method but GnuIMP also has this problem with the fast-integer method so there may be a common cause

I don't know why it is appearing only on one axis and not the other

Images

look in the out of focus areas of this picture, you should notice vertical banding

Again, I must stress that some people can see it more easily than others and that is why I wrote the software linked above to help pick out the signal

[urban-warrior]

Artifacts in JPEG images can be anticipated due to the default pixel quantization to integers. This default approach strikes a balance between speed and quality. Although we introduced the float method as the default option a while back, there was resistance due to its impact on performance. We suggest using -define jpeg:dct-method=default for users who are ready to endure the performance trade-off in exchange for improved results' quality. If you're certain the problem lies with ImageMagick, feel free to proceed, otherwise, for further deliberation, it's advisable to engage with the developers of the JPEG delegate library.

[electroCutie]

@urban-warrior I'm working on finding the root cause in he underlying JPEG library. I have a few theories as to the underlying bug but regardless of that I thought that this sort of visual artifact being so widespread in the default option would be unacceptable.

I actually have never seen the -define jpeg:dct-method=default until I was digging into this bug today so perhaps it needs to be communicated better if changing the default isn't an option, but that is up to the ImageMagick maintainers

I dunno, I see this constantly in the photography tag on mastodon and its really nasty. I reported it to them first but i thought that fixing the root cause would be better than playing whack a mole and that performance sensitive people can choose the faster method once they understand the consequences of doing so rather than making everyone have worse images because of those people from whom speed is more important

[electroCutie]

Artifacts in JPEG images can be anticipated due to the default pixel quantization to integers. This default approach strikes a balance between speed and quality.

and just to be very clear: I understand that JPEG is lossy. Its the long range vertical stripes that literally cover the entire image that are the problem