compare of identical tiffs returns all points are different

Post any defects you find in the released or beta versions of the ImageMagick software here. Include the ImageMagick version, OS, and any command-line required to reproduce the problem. Got a patch for a bug? Post it here.
Post Reply
alex-kas
Posts: 11
Joined: 2016-05-01T05:17:47-07:00
Authentication code: 1151

compare of identical tiffs returns all points are different

Post by alex-kas » 2019-06-07T13:46:04-07:00

Dear all,
as simple as the subject. The command and the result are as follows:

Code: Select all

alex@novo /mnt/storage/income.pre/20190521.d600 $ compare -metric AE  79_2.tif 79_2.tif /dev/null
2.41603e+07
As you see, I compare a file with itself. It is roughly 6000x4000 pixels, so roughly all pixels are claimed different. I checked 16bit and 8bit tiffs and the outcome is the same weird one ... I tried different metrics and no luck.
If I convert to, say, .png or .bmp, then all is in order, i.e. 0 different pixels.

For your info, here is the version:

Code: Select all

alex@novo /mnt/storage/income.pre/20190521.d600 $ compare -verbose
Version: ImageMagick 7.0.8-46 Q16 x86_64 2019-05-30 https://imagemagick.org
I sit on linux 64-bit.

Any thoughts about this mess are welcome.
Thanks, Alex

snibgo
Posts: 12306
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: compare of identical tiffs returns all points are different

Post by snibgo » 2019-06-07T15:19:22-07:00

For v7, I suggest you use "magick compare" instead of "compare".

How many images are in your tif? (I suppose only one, but it is worth checking.)

Does your version of IM have HDRI? What is the complete text output of "magick -version"?

Perhaps there is a floating-point problem. What does "magick 79_2.tif -verbose info:" say about your file? Can you provide a link to 79_2.tif?
snibgo's IM pages: im.snibgo.com

alex-kas
Posts: 11
Joined: 2016-05-01T05:17:47-07:00
Authentication code: 1151

Re: compare of identical tiffs returns all points are different

Post by alex-kas » 2019-06-08T00:57:35-07:00

GREAT!!!
You are crazily right! The image has itself and the thumbnail ... specifying 79_2.tif[0] has resolved the issue.
How come I could miss this simple point? But I have a feeling it was not like that in previous versions of IM.
compare vs magick compare has no difference in behaviour.

Why not to warn about such a subtlety from the IM side?

Greatest thanks go to you, dude!

snibgo
Posts: 12306
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: compare of identical tiffs returns all points are different

Post by snibgo » 2019-06-08T03:16:36-07:00

alex-kas wrote:Why not to warn about such a subtlety from the IM side?
IM doesn't second-guess what the user really wants. If IM can't do what the user asks, it raises an error. If there is a problem such as something minor wrong with an image file but the work can still be done, IM raises a warning.

But IM doesn't create warnings such as, "Yes, I can do that, but it may not be what you intended."
snibgo's IM pages: im.snibgo.com

User avatar
fmw42
Posts: 25664
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: compare of identical tiffs returns all points are different

Post by fmw42 » 2019-06-08T10:14:03-07:00

The thumbnail is not likely the issue. Please post your TIFF file and we can take a look at what is happening.

alex-kas
Posts: 11
Joined: 2016-05-01T05:17:47-07:00
Authentication code: 1151

Re: compare of identical tiffs returns all points are different

Post by alex-kas » 2019-06-08T10:51:49-07:00

The full output of magick -version is:

Code: Select all

alex@novo ~ $ magick -version
Version: ImageMagick 7.0.8-46 Q16 x86_64 2019-05-30 https://imagemagick.org
Copyright: © 1999-2019 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenCL OpenMP(4.5) 
Delegates (built-in): bzlib cairo djvu fontconfig freetype gslib jng jp2 jpeg lcms ltdl openexr pangocairo png ps raw rsvg tiff webp x xml zlib
Full outputs for strange and expected comparisons are:

Code: Select all

alex@novo /mnt/storage/income.pre/20190521.d600 $ magick compare -metric AE 79_2.tif 79_2.tif /dev/null
2.41603e+07compare: Unknown field with tag 34864 (0x8830) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/995.
compare: Incompatible type for "FileSource"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/995.
compare: Incompatible type for "SceneType"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/995.
compare: Wrong data type 3 for "GainControl"; tag ignored. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/995.
compare: Unknown field with tag 42240 (0xa500) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/995.
compare: Incompatible type for "FileSource"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/995.
compare: Incompatible type for "SceneType"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/995.
compare: Wrong data type 3 for "GainControl"; tag ignored. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/995.
compare: Unknown field with tag 42240 (0xa500) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/995.
compare: IO error writing tag data. `TIFFWriteDirectoryTagData' @ error/tiff.c/TIFFErrors/656.

alex@novo /mnt/storage/income.pre/20190521.d600 $ magick compare -metric AE 79_2.tif[0] 79_2.tif[0] /dev/null
0compare: Unknown field with tag 34864 (0x8830) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/995.
compare: Incompatible type for "FileSource"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/995.
compare: Incompatible type for "SceneType"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/995.
compare: Wrong data type 3 for "GainControl"; tag ignored. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/995.
compare: Unknown field with tag 42240 (0xa500) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/995.
compare: Incompatible type for "FileSource"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/995.
compare: Incompatible type for "SceneType"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/995.
compare: Wrong data type 3 for "GainControl"; tag ignored. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/995.
compare: Unknown field with tag 42240 (0xa500) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/995.
compare: IO error writing tag data. `TIFFWriteDirectoryTagData' @ error/tiff.c/TIFFErrors/656.
And the link to this exact file is as follows:
http://altaray.net/altaray.net/79_2.tif

Thanks for taking care of this,
Alex

User avatar
fmw42
Posts: 25664
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: compare of identical tiffs returns all points are different

Post by fmw42 » 2019-06-08T12:02:04-07:00

I get the same results as you using -metric rmse -- not 0. I suspect your TIFF is malformed in some way causing all the warnings and especially the error at the end

Code: Select all

compare: IO error writing tag data. `TIFFWriteDirectoryTagData' @ error/tiff.c/TIFFErrors/656.
Does this happen with other TIFFs?

User avatar
fmw42
Posts: 25664
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: compare of identical tiffs returns all points are different

Post by fmw42 » 2019-06-08T12:04:54-07:00

Also Mac PREVIEW shows two images -- the main image and a smaller one. Photoshop does not show the smaller one. This may the issue that the smaller one is not written in the file in the correct location for a thumbnail? I am not an expert on TIFF. But snibgo solution of adding [0] seems to resolve it.

alex-kas
Posts: 11
Joined: 2016-05-01T05:17:47-07:00
Authentication code: 1151

Re: compare of identical tiffs returns all points are different

Post by alex-kas » 2019-06-08T14:53:53-07:00

The tiffs are created by Nikon Capture NX2 (i.e. Nikon native, old though, NEF (aka RAW) converter). I will check about tiffs of other origin. For the moment I know two facts:
- The newer RAW converter from Nikon (Capture NX-D) creates tiffs which result in the same weird behaviour of IM.
- GraphicsMagick (GM) works flawlessly on my tiffs without throwing any warning/error. The issue with GM that it does not have AE metric and this is the one I badly need.
- On all common metrics results of IM and GM coincide as long as I add [0] for IM.

Technically I'm fine adding [0] and also technically I cannot avoid working with the tiffs I have.

User avatar
fmw42
Posts: 25664
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: compare of identical tiffs returns all points are different

Post by fmw42 » 2019-06-08T16:36:43-07:00

Is your TIFF jpg compressed? If so and that is an old jpg compression format, that could be an issue.

alex-kas
Posts: 11
Joined: 2016-05-01T05:17:47-07:00
Authentication code: 1151

Re: compare of identical tiffs returns all points are different

Post by alex-kas » 2019-06-09T03:27:00-07:00

I'm not sure what is jpg compression inside of a tiff but what is named LZW compression is turned OFF in the tiff output dialog of Nikon programs which I use.

snibgo
Posts: 12306
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: compare of identical tiffs returns all points are different

Post by snibgo » 2019-06-09T04:08:23-07:00

I'm confident the problem is that the two images in the first TIF are being compared, rather than the first image in the first TIF being compared with the first image in the second TIF. The solution is to use [0].

Compare, like the other IM utilities, accepts any number of input images, but will operate on only the first two.
snibgo's IM pages: im.snibgo.com

alex-kas
Posts: 11
Joined: 2016-05-01T05:17:47-07:00
Authentication code: 1151

Re: compare of identical tiffs returns all points are different

Post by alex-kas » 2019-06-13T01:52:58-07:00

I agree with placing [0] but one point is unclear. Thumbnail and the complete image are of different sizes. At best they share the same aspect ratio, but most likely not exactly in decimal floats either. What is the meaning of comparing them? Maybe my question looks stupid but what would be your idea on this?

snibgo
Posts: 12306
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: compare of identical tiffs returns all points are different

Post by snibgo » 2019-06-13T03:56:19-07:00

"Compare" of two input images actually compares WW*HH pixels, where WW is the largest of the two widths and HH is the largest of the two heights.

If either image is less than WW*HH, virtual pixels are used.

For example:

Code: Select all

magick toes.png -background red -extent 400x400 x.png

magick compare -background Red -virtual-pixel Background -metric AE toes.png x.png NULL:

0
x.png is toes.png extended with red pixels, so they are different sizes. But we can compare them with "red" for virtual pixels, and this correctly says they are identical.
snibgo's IM pages: im.snibgo.com

Post Reply