Using compare and fuzz mismatch between AE and image output

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Steve2.0
Posts: 10
Joined: 2020-02-21T08:29:11-07:00
Authentication code: 1152

Using compare and fuzz mismatch between AE and image output

Post by Steve2.0 »

I am not sure if this is a bug or an expected change in behavior – it’s also based on large delta between ImageMagick version hence posting to this forum first.

I have been using compare and fuzz to find a subset of worst pixels in an image.
In version 7.0.7-11 Q16 x64 2017-11-12 (Windows) the behavior is as expected whereas version 7.0.9-25 Q16 x64 2020-02-21 (Windows) produces a different AE result and doesn’t show all the pixel errors in the hard diff image output.

Compare 2 images with the following command and get a hard diff image with errored pixels highlighted as white against a black background:

Code: Select all

magick compare -verbose -metric ae -fuzz 3.5% -compose src -highlight-color white -lowlight-color black .\source.png .\reference.png fuzz_errors.png
Results from 7.0.7-11 Q16 x64 2017-11-12:
Channel distortion: AE
red: 1466
green: 3384
blue: 991
all: 5206
Using histogram check confirmed resultant image contains 5206 white pixels

Results from 7.0.9-25 Q16 x64 2020-02-21:
Channel distortion: AE
red: 0
green: 0
blue: 175
all: 175
Using histogram check there are 0 white pixels, adjusting the fuzz value lower eventually produces pixels in the image but the number does not match the AE result

Tried searching this issue but not found anything similar so far - perhaps this is an expected change in the fuzz behaviour but I can't quite see how that would be the case.

Thanks
Steve

User avatar
fmw42
Posts: 26383
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Using compare and fuzz mismatch between AE and image output

Post by fmw42 »

For IM 7.0.9.25 Q16 Mac OSX and IM 6.9.10.95, I get:

Code: Select all

magick compare -verbose -metric ae -fuzz 3.5% -compose src -highlight-color white -lowlight-color black lena.png lena.jpg diff.png
lena.png PNG 256x256 256x256+0+0 8-bit sRGB 181737B 0.010u 0:00.009
lena.jpg JPEG 256x256 256x256+0+0 8-bit sRGB 29728B 0.000u 0:00.004
Image: lena.png
Channel distortion: AE
red: 49
green: 82
blue: 429
all: 429
lena.png=>diff.png PNG 256x256 256x256+0+0 8-bit sRGB 181737B 0.100u 0:00.027


Seems reasonable to me.

Can you provide your images to some free hosting service and put the URLs here.

Steve2.0
Posts: 10
Joined: 2020-02-21T08:29:11-07:00
Authentication code: 1152

Re: Using compare and fuzz mismatch between AE and image output

Post by Steve2.0 »

Source
https://1drv.ms/u/s!An0gPH6tQBq7hKtP63I ... Q?e=hHgTbx

Ref
https://1drv.ms/u/s!An0gPH6tQBq7hKtOeWA ... Q?e=4uB3Ja

Code: Select all

magick.exe compare -verbose -metric AE -fuzz 2.5% .\source_crop_2489_1280.png .\reference_2489_1280.png -compose src -highlight-color white -lowlight-color black fuzz_errors.png
Results from 7.0.7-11 Q16 x64 2017-11-12:

.\source_crop_2489_1280.png PNG 508x508 11520x6480+9704+4868 8-bit sRGB 27618B 0.016u 0:00.007
.\reference_crop_2489_1280.png PNG 508x508 11520x6480+9704+4868 8-bit sRGB 27424B 0.000u 0:00.011
Image: .\gen1-1522220226_crop_2489_1280.png
Channel distortion: AE
red: 71120
green: 113008
blue: 83984
all: 149312

Results from 7.0.9-25 Q16 x64 2020-02-21:

.\source_crop_2489_1280.png PNG 508x508 11520x6480+9704+4868 8-bit sRGB 27618B 0.000u 0:00.009
.\reference_crop_2489_1280.png PNG 508x508 11520x6480+9704+4868 8-bit sRGB 27424B 0.016u 0:00.012
Image: .\gen1-1522220226_crop_2489_1280.png
Channel distortion: AE
red: 0
green: 336
blue: 72064
all: 72064

User avatar
fmw42
Posts: 26383
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Using compare and fuzz mismatch between AE and image output

Post by fmw42 »

This seems to work properly for me. I add a 5x5 black area to the top left corner of the logo: image and compare to the original. I expect 25 pixels for the AE count.

Code: Select all

magick logo: logo.png
magick logo: -size 5x5 xc:black -composite logo2.png
magick compare -verbose -metric ae -fuzz 3.5% -compose src -highlight-color white -lowlight-color black logo.png logo2.png diff.png
logo.png PNG 640x480 640x480+0+0 8-bit sRGB 256c 27398B 0.000u 0:00.006
logo2.png PNG 640x480 640x480+0+0 8-bit sRGB 67187B 0.010u 0:00.007
Image: logo.png
Channel distortion: AE
red: 25
green: 25
blue: 25
all: 25
logo.png=>diff.png PNG 640x480 640x480+0+0 8-bit sRGB 27398B 0.300u 0:00.076

Do you get something different from it?

User avatar
fmw42
Posts: 26383
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Using compare and fuzz mismatch between AE and image output

Post by fmw42 »

For the two images you posted, I get zero AE difference. Might you have posted the wrong images?

Steve2.0
Posts: 10
Joined: 2020-02-21T08:29:11-07:00
Authentication code: 1152

Re: Using compare and fuzz mismatch between AE and image output

Post by Steve2.0 »

First thank you for the super quick responses...

I've re-uploaded the two sample images, not shared like this from onedrive before seems I may have messed up the original links:

https://1drv.ms/u/s!An0gPH6tQBq7hKtOeWA5YRrdViuPAQ

https://1drv.ms/u/s!An0gPH6tQBq7hKtP63IwN2u2xit8HQ

I've also uploaded the resultant image from 7.0.7 version of the tool which is what I expect, however I get an all black result using 7.0.9 as well as the different AE results.

https://1drv.ms/u/s!An0gPH6tQBq7hKtQ-s-vjpp0tz15MA

Steve

User avatar
fmw42
Posts: 26383
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Using compare and fuzz mismatch between AE and image output

Post by fmw42 »

I still get zero difference using IM 7.0.9.25 and 6.9.10.95

Code: Select all

magick compare -verbose -metric AE -fuzz 3.5% -compose src -highlight-color white -lowlight-color black reference_crop_2489_1280.png source_crop_2489_1280.png diff.png
reference_crop_2489_1280.png PNG 508x508 11520x6480+9704+4868 8-bit sRGB 27424B 0.000u 0:00.010
source_crop_2489_1280.png PNG 508x508 11520x6480+9704+4868 8-bit sRGB 27618B 0.010u 0:00.006
Image: reference_crop_2489_1280.png
Channel distortion: AE
red: 0
green: 0
blue: 0
all: 0
reference_crop_2489_1280.png=>diff.png PNG 508x508 11520x6480+9704+4868 8-bit sRGB 27424B 0.240u 0:00.060


Did you try my test above with the logo: image?

snibgo
Posts: 13034
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Using compare and fuzz mismatch between AE and image output

Post by snibgo »

Steve2.0 wrote:magick compare -verbose -metric ae -fuzz 3.5% -compose src -highlight-color white -lowlight-color black .\source.png .\reference.png fuzz_errors.png
You are on Windows. Are these commands in BAT scripts? If so, you should double the percents %%.
snibgo's IM pages: im.snibgo.com

Steve2.0
Posts: 10
Joined: 2020-02-21T08:29:11-07:00
Authentication code: 1152

Re: Using compare and fuzz mismatch between AE and image output

Post by Steve2.0 »

It's clear I am failing to operate OneDrive sharing correctly.
I've created a zip containing the images directly from where I am testing using the command line.

https://1drv.ms/u/s!An0gPH6tQBq7hKtRhVa ... Q?e=27zIpn

I really am seeing errors as this is the output of my python tool that extracting crops around the worst errors, I've check and double checked and the results I reported above are accurate.

I am running on Windows via PowerShell - "%%" shouldn't be necessary but check and got the same results.
When I am running my Python analysis script, that calls ImageMagick via a subprocess.Popen call, I get the same differing results from the two version of the tool.

Hopefully I've supplied images that demonstrate it this time, as they are most definitely different.

Steve

Steve2.0
Posts: 10
Joined: 2020-02-21T08:29:11-07:00
Authentication code: 1152

Re: Using compare and fuzz mismatch between AE and image output

Post by Steve2.0 »

Forgot to add I have tested the logo.png example and it gets the expected result in both versions.
However if I just added some light noise in the range of +/- 8 delta on each channel I can recreate the issue
Sorry I am not an ImageMagick guru so added the noise in another program - learning now how do achieve that on the command line so you can replicate....

Steve

User avatar
fmw42
Posts: 26383
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Using compare and fuzz mismatch between AE and image output

Post by fmw42 »

I downloaded your zip file and processed the two images and still get 0 AE difference when using -fuzz 3.5%.

If I set the fuzz to 0%, then I get:

Code: Select all

magick compare -verbose -metric AE -fuzz 0% -compose src -highlight-color white -lowlight-color black reference_crop_2489_1280.png 
source_crop_2489_1280.png diff.png
reference_crop_2489_1280.png PNG 508x508 11520x6480+9704+4868 8-bit sRGB 27424B 0.000u 0:00.007
source_crop_2489_1280.png PNG 508x508 11520x6480+9704+4868 8-bit sRGB 27618B 0.010u 0:00.006
Image: reference_crop_2489_1280.png
Channel distortion: AE
red: 236512
green: 256208
blue: 258064
all: 258064


As user snibgo questioned, are you doing this in a bat file or in the CMD window? If the former, then -fuzz 3.5% should be -fuzz 3.5%%. If the latter, then I cannot explain why I get 0 difference at 3.5% and you get other values.

Steve2.0
Posts: 10
Joined: 2020-02-21T08:29:11-07:00
Authentication code: 1152

Re: Using compare and fuzz mismatch between AE and image output

Post by Steve2.0 »

Worked out an example based on the logo image:

Code: Select all

magick logo: logo.png
magick logo: -attenuate 1% +noise gaussian logo_noise.png
magick compare -verbose -metric ae -fuzz 15% -compose src -highlight-color white -lowlight-color black .\logo.png .\logo_noise.png diff_logo.png
For 7.0.9 this results in:

.\logo.png PNG 640x480 640x480+0+0 8-bit sRGB 256c 27398B 0.000u 0:00.013
.\logo_noise.png PNG 640x480 640x480+0+0 8-bit sRGB 659953B 0.016u 0:00.023
Image: .\logo.png
Channel distortion: AE
red: 144
green: 484
blue: 1256
all: 1256

Then checking with a histogram:

Code: Select all

magick convert .\diff_logo.png -fill black +opaque white -format %c histogram:info:
outputs:

306766: ( 0, 0, 0) #000000 black
434: (255,255,255) #FFFFFF white

I'm expecting 1256 white pixels and while 7.0.7 produces a different AE result with an 'all' pixel error of 27085 that's exactly the number of white pixels I get in the difference image using the older version.

Steve

User avatar
fmw42
Posts: 26383
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Using compare and fuzz mismatch between AE and image output

Post by fmw42 »

This is what I get with IM 7.0.9.25

Code: Select all

magick logo: logo.png
magick logo: -attenuate 1% +noise gaussian logo_noise.png
magick compare -verbose -metric ae -fuzz 15% -compose src -highlight-color white -lowlight-color black logo.png logo_noise.png diff_logo.png
logo.png PNG 640x480 640x480+0+0 8-bit sRGB 256c 27398B 0.010u 0:00.006
logo_noise.png PNG 640x480 640x480+0+0 8-bit sRGB 656714B 0.010u 0:00.013
Image: logo.png
Channel distortion: AE
red: 129
green: 477
blue: 1203
all: 1203
logo.png=>diff_logo.png PNG 640x480 640x480+0+0 8-bit sRGB 27398B 0.300u 0:00.091


The number of white pixels in the output is:

Code: Select all

magick diff_logo.png txt: | grep "white" | tail -n +2 | wc -l
406

Histogram shows:

Code: Select all

magick diff_logo.png -format "%c" histogram:info:
306793: (0,0,0) #000000 black
407: (255,255,255) #FFFFFF white

User avatar
fmw42
Posts: 26383
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Using compare and fuzz mismatch between AE and image output

Post by fmw42 »

If I use IM 7.0.8.14

Code: Select all

magick compare -verbose -metric ae -fuzz 15% -compose src -highlight-color white -lowlight-color black logo.png logo_noise.png diff_logo.png
logo.png PNG 640x480 640x480+0+0 8-bit sRGB 256c 27398B 0.010u 0:00.009
logo_noise.png PNG 640x480 640x480+0+0 8-bit sRGB 656714B 0.010u 0:00.019
Image: logo.png
Channel distortion: AE
red: 129
green: 477
blue: 1203
all: 1203

Code: Select all

magick diff_logo.png -format "%c" histogram:info:
281112: ( 0, 0, 0) #000000 black
26088: (255,255,255) #FFFFFF white


I do not understand why older versions show a different result for the histogram, nor why we get different results on IM 7.0.9.25 between Windows and Mac, nor why the histogram does not match the AE count.

I suspect the older versions are not creating the output image using the fuzz value, even though the metric values are computed using the fuzz value.

Steve2.0
Posts: 10
Joined: 2020-02-21T08:29:11-07:00
Authentication code: 1152

Re: Using compare and fuzz mismatch between AE and image output

Post by Steve2.0 »

Well at least now I know I am not going mad.
Is this something I should post to the bug forum?

I can deal with differences in the AE results between versions but not getting the expected number of pixels in the output image is a problem.

Steve

Locked