Color information lost during resize operation

Post any defects you find in the released or beta versions of the ImageMagick software here. Include the ImageMagick version, OS, and any command-line required to reproduce the problem. Got a patch for a bug? Post it here.
Post Reply
jwoelper
Posts: 7
Joined: 2012-04-25T09:14:34-07:00
Authentication code: 13

Color information lost during resize operation

Post by jwoelper » 2012-04-25T09:29:17-07:00

I am using convert to scale images (actually normal map video game textures containing the specularity in the alpha channel).

if i do a convert in.tga out.tif the out.tif looks as expected (alpha and color channels intact).
if i do a convert -resize 1024x1024 in.tga out.tif the color information is lost (turned to black) where alpha=0.

I've read through the manual but have not found a flag to prevent this. I would guess that this may be due to some optimization in the resize process or a premultiplication issue. Regardless, format conversion should not produce other results then scaling the image in this case.


Version: ImageMagick 6.7.6-4 2012-04-10 Q16

User avatar
fmw42
Posts: 25764
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Color information lost during resize operation

Post by fmw42 » 2012-04-25T09:33:43-07:00

I am not sure about tga alpha information in IM

try

convert in.tga out.tga

does it retain your alpha

if so, then move the resize after the input image

convert in.tga -resize 1024x1024 out.tif

or

convert in.tga -channel rgba -alpha on -resize 1024x1024 out.tif

jwoelper
Posts: 7
Joined: 2012-04-25T09:14:34-07:00
Authentication code: 13

Re: Color information lost during resize operation

Post by jwoelper » 2012-04-25T10:24:12-07:00

convert in.tga out.tga does retain the colors, as convert in.tga out.tif does.

convert in.tga -resize 1024x1024 out.tif also does not retain the colors where a=0.

Just for clarification, the alpha is always retained, just the rgb colors are missing where alpha is absolutely zero.

fmw42 wrote:I am not sure about tga alpha information in IM

try

convert in.tga out.tga

does it retain your alpha

if so, then move the resize after the input image

convert in.tga -resize 1024x1024 out.tif

or

convert in.tga -channel rgba -alpha on -resize 1024x1024 out.tif

User avatar
fmw42
Posts: 25764
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Color information lost during resize operation

Post by fmw42 » 2012-04-25T13:53:45-07:00

can you post a link to an example tga file that has this probem and your resized result also?

jwoelper
Posts: 7
Joined: 2012-04-25T09:14:34-07:00
Authentication code: 13

Re: Color information lost during resize operation

Post by jwoelper » 2012-04-26T00:43:41-07:00

Sure!

http://madprocessor.de/files/in.tga
the first, unaltered file. You should see a red image when viewed with display -alpha off in.tga.

http://madprocessor.de/files/out.tga
the same file, still tga, just resized by one pixel in x/y. You should see text.

You have to display the images using:

Code: Select all

display -alpha off out.tga
Unfortunately, 99,9% of all image viewers seem to multiply images with its alpha channel and have no option to turn it off, so this problem is only visible with an image viewer capable of displaying the color channels alone. Otherwise both images look the same.
fmw42 wrote:can you post a link to an example tga file that has this probem and your resized result also?

User avatar
fmw42
Posts: 25764
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Color information lost during resize operation

Post by fmw42 » 2012-04-26T10:08:11-07:00

I can confirm that this is a bug on IM 6.7.6.7 Q16 Mac OSX Snow Leopard.


Input:

Image

convert in.tga -alpha off in_aoff.png

Image


convert in.tga -alpha extract in_alpha.png

Image


Now resize:

convert in.tga -resize 255x255! in_rs.png

Image


convert in_rs.png -alpha off in_rs_aoff.png

Image


convert in_rs.png -alpha extract in_rs_alpha.png

Image


But there is a workaround

convert in.tga \
\( -clone 0 -alpha off \) \
\( -clone 0 -alpha extract \) \
-delete 0 -resize 255x255! \
-alpha off -compose copy_opacity -composite \
in_rs3.png

Image


convert in_rs3.png -alpha off in_rs3_aoff.png

Image


convert in_rs3.png -alpha extract in_rs3_alpha.png

Image




This is a simpler workaround, but you need to know ahead of time what the background color is or what you would like it to be.

convert in.tga -resize 255x255! -background red -alpha background in_rs4.png

User avatar
anthony
Posts: 8883
Joined: 2004-05-31T19:27:03-07:00
Authentication code: 8675308
Location: Brisbane, Australia

Re: Color information lost during resize operation

Post by anthony » 2012-04-26T22:03:12-07:00

THIS IS NOT A BUG

The reason for the loss of color when resizing, is that resize (and many other operators) treat alpha as a 'blending' channel.

That is a fully-transparent pixel is transparent and thus the values in the color channels has no (or lesser) meaning, and should not contribute (or contribute less) to the overall resulting color. As a consequence alpha is used to weight alpha during convolutions (like resize) to ensure fully-transparent pixels do not take part in the resize.

This is to prevent halo effects that existed early in IMv6 history
http://www.imagemagick.org/Usage/bugs/resize_halo/

IMv7 will see the ability to turn off 'alpha blending' completely, and thus just treat all channels, including alpha, as a normal image data channel, without any special meaning. In IMv6 it is necessary to separate alpha from the image and handle the channels as separate entities.

Simple way, just separate ALL channels!

Code: Select all

   convert image  -channel RGBA -separate  -resize 100x100  -combine  result
Anthony Thyssen -- Webmaster for ImageMagick Example Pages
https://imagemagick.org/Usage/

jwoelper
Posts: 7
Joined: 2012-04-25T09:14:34-07:00
Authentication code: 13

Re: Color information lost during resize operation

Post by jwoelper » 2012-04-27T01:14:08-07:00

Fantastic, thanks a lot for clearing this up!

I agree that in most cases colored pixels are less important where alpha is zero.
On the other hand, in many scientifical and highend (although im is pretty highend, too) programs like houdini / shake / nuke every image channel is treated equally during pixel operations per default, maybe this is what had me puzzled.
Maybe operators that discard color information could be identified as such in the documentation or the operators' reference.

Thanks,

Johann

anthony wrote:THIS IS NOT A BUG

The reason for the loss of color when resizing, is that resize (and many other operators) treat alpha as a 'blending' channel.

That is a fully-transparent pixel is transparent and thus the values in the color channels has no (or lesser) meaning, and should not contribute (or contribute less) to the overall resulting color. As a consequence alpha is used to weight alpha during convolutions (like resize) to ensure fully-transparent pixels do not take part in the resize.

This is to prevent halo effects that existed early in IMv6 history
http://www.imagemagick.org/Usage/bugs/resize_halo/

IMv7 will see the ability to turn off 'alpha blending' completely, and thus just treat all channels, including alpha, as a normal image data channel, without any special meaning. In IMv6 it is necessary to separate alpha from the image and handle the channels as separate entities.

Simple way, just separate ALL channels!

Code: Select all

   convert image  -channel RGBA -separate  -resize 100x100  -combine  result

User avatar
anthony
Posts: 8883
Joined: 2004-05-31T19:27:03-07:00
Authentication code: 8675308
Location: Brisbane, Australia

Re: Color information lost during resize operation

Post by anthony » 2012-04-27T03:52:35-07:00

May I ask, what is this data. It does not actually sond like 'alpha' values, but a fudge to use an 'extra channel' for 'extra data'.


Note that this isn't so must as 'discarding information', as mathematically finding the information 'unimportant'. Though I can see how it could be seen as 'loosing' information. Keeping the information can however be just as bad when alpha blending is present. It can lead to a 'false-sense of security', as pixels that are semi-transparent or fully-opaque pixels involved with such translucent pixels, are also effects by alpha blending.

Better to loose it an KNOW alpha blending happened, than not loose it and wonder why pixels are not quite right.

This view may be a cop-out, but it is ceratinaly easier to discover than trying to keep transparent pixel values, which IM considered 'irrelevent' in the calculation.


Generally if the operation involve a pixel neighbourhood (convolution), or multiple pixels (composition), then some type of alpha blending will be need. That is any time you get colors being generated by mixing colors.

That is typically all operations except per-pixel operations (level, thresholding, etc).

IM has to do alpha blending by default as it it designed for user images in mind. However We do understand that in many cases alpha blending is not wanted, and while not all operations can do this, we did include options for specific cases.

One case in point was the mathematical composition method. Setting -channel to RGBA (away from the default setting) told IM to apply the maths to the alpha channel, and not just the SVG defined alpha blending.
Morphology methods also did this.

IMv7 not only provides more than the default 5 channels, but also has channel traits that allow you to specify alpha blending in specific ways. (also write masks, and eventually read masks). This is a low level implementation that opperators can make use of, so it will be a much more universal in nature.

This means you can simply avoid 'alpha' and use extra color channels, when the data isn't actually 'alpha'.

However IMv7 is still in alpha. With Magick doing much of the low level work, while I am working on the higher level API.



Things are progressing.
Anthony Thyssen -- Webmaster for ImageMagick Example Pages
https://imagemagick.org/Usage/

User avatar
fmw42
Posts: 25764
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Color information lost during resize operation

Post by fmw42 » 2012-04-27T09:13:27-07:00

Anthony,

Thanks for clearing that up. Learned something new today. Also thanks for the better solution.

Fred

jwoelper
Posts: 7
Joined: 2012-04-25T09:14:34-07:00
Authentication code: 13

Re: Color information lost during resize operation

Post by jwoelper » 2012-05-02T03:53:03-07:00

As for the data, it is indeed a bit of a special case. In the 3d/games industry some programs or game engines use the alpha channel in normal maps to define the specularity of a material/shader. This saves memory as one does not need an extra texture to hold a greyscale image only. That way, using only one map for color with transparency and a second one with normal map as RGB / specular map as alpha one only needs two textures for about 99% of possible shader/material combinations.

But with your workaround this works perfectly and is easy to handle! Good to know! I already look forward to IM 7, too!

Thanks again for shedding light on this!

Johann
anthony wrote:May I ask, what is this data. It does not actually sond like 'alpha' values, but a fudge to use an 'extra channel' for 'extra data'.


Note that this isn't so must as 'discarding information', as mathematically finding the information 'unimportant'. Though I can see how it could be seen as 'loosing' information. Keeping the information can however be just as bad when alpha blending is present. It can lead to a 'false-sense of security', as pixels that are semi-transparent or fully-opaque pixels involved with such translucent pixels, are also effects by alpha blending.

Better to loose it an KNOW alpha blending happened, than not loose it and wonder why pixels are not quite right.

This view may be a cop-out, but it is ceratinaly easier to discover than trying to keep transparent pixel values, which IM considered 'irrelevent' in the calculation.


Generally if the operation involve a pixel neighbourhood (convolution), or multiple pixels (composition), then some type of alpha blending will be need. That is any time you get colors being generated by mixing colors.

That is typically all operations except per-pixel operations (level, thresholding, etc).

IM has to do alpha blending by default as it it designed for user images in mind. However We do understand that in many cases alpha blending is not wanted, and while not all operations can do this, we did include options for specific cases.

One case in point was the mathematical composition method. Setting -channel to RGBA (away from the default setting) told IM to apply the maths to the alpha channel, and not just the SVG defined alpha blending.
Morphology methods also did this.

IMv7 not only provides more than the default 5 channels, but also has channel traits that allow you to specify alpha blending in specific ways. (also write masks, and eventually read masks). This is a low level implementation that opperators can make use of, so it will be a much more universal in nature.

This means you can simply avoid 'alpha' and use extra color channels, when the data isn't actually 'alpha'.

However IMv7 is still in alpha. With Magick doing much of the low level work, while I am working on the higher level API.



Things are progressing.

Post Reply