Stream zero output on palette (and grayscale) PNG

Post any defects you find in the released or beta versions of the ImageMagick software here. Include the ImageMagick version, OS, and any command-line required to reproduce the problem. Got a patch for a bug? Post it here.
Post Reply
abukaj
Posts: 4
Joined: 2014-06-17T02:51:11-07:00
Authentication code: 6789

Stream zero output on palette (and grayscale) PNG

Post by abukaj »

When I am trying to stream file Ambermoon-lyramion.6400x6400.png my output is a stream of zeros.

Code: Select all

$ stream Ambermoon-lyramion.6400x6400.png - | hexdump
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
7530000
However other IM tools seem to open it properly (display, convert etc.).

I think the problem is connected with palette mode of the PNG.

Code: Select all

$ identify Ambermoon-lyramion.6400x6400.png            
Ambermoon-lyramion.6400x6400.png PNG 6400x6400 6400x6400+0+0 8-bit PseudoClass 256c 6.562MB 0.000u 0:00.009
However when the image is converted to GIF - then stream streams it correctly, so it is not general palette mode issue.

My version of IM is:

Code: Select all

$ stream -version
Version: ImageMagick 6.6.9-7 2014-03-06 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2011 ImageMagick Studio LLC
Features: OpenMP
The file is available at: https://dl.dropboxusercontent.com/u/223 ... 0x6400.png

I know I can convert image and then stream it, but I need a lightweight tool for automated streaming of large images and convert is not a lightweight tool.
Last edited by abukaj on 2014-06-27T02:39:23-07:00, edited 2 times in total.
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: Stream zero output on palette PNG

Post by magick »

We can reproduce this problem. If you first convert the PNG to PNM, streaming works. We'll investigate the problem and hopefully have a patch soon.
abukaj
Posts: 4
Joined: 2014-06-17T02:51:11-07:00
Authentication code: 6789

Re: Stream zero output on palette PNG

Post by abukaj »

Thanks for the piece of advice, however what I need is to stream the image without loading it into the memory (AFAIK convert does it that way).

I am developing a server for hosting images of brain tissue of high resolution, so the solution should be also format-independent. Luckily it is very unlikely to have a palette PNG uploaded by our users.
User avatar
glennrp
Posts: 1147
Joined: 2006-04-01T08:16:32-07:00
Location: Maryland 39.26.30N 76.16.01W

Re: Stream zero output on palette PNG

Post by glennrp »

We understand your requirement, but I don't think the fix is trivial. PNG is designed to be streamable, and so is libpng, but applications must use a different part of libpng (the progressive reader, found in pngpread.c, instead of the sequential reader, found in pngread.c), to achieve streaming of a PNG datastream. By streamable, I mean that the encoder can receive a buffer containing only a part of the image, process and emit the buffer, and wait for another buffer. Note that in this context, the term "progressive" has nothing to do with PNG interlacing.

There are demonstrations of progressive reading in the libpng distribution. "example.c" is a commented example that does not actually run, and "contrib/gregbook" contains "rpng*.c" examples that do work. I think the licensing is OK (dual-licensed GPLV2 and BSD-like).

Certainly there is a bug that is causing "convert" to write zeroes instead of the correct image pixels, though.
Last edited by glennrp on 2014-06-18T11:06:53-07:00, edited 2 times in total.
User avatar
glennrp
Posts: 1147
Joined: 2006-04-01T08:16:32-07:00
Location: Maryland 39.26.30N 76.16.01W

Re: Stream zero output on palette PNG

Post by glennrp »

magick wrote: If you first convert the PNG to PNM, streaming works.
By this I assume you mean that "convert" is able to stream in the PNM, not that the conversion from PNG to PNM works in a stream.
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: Stream zero output on palette PNG

Post by magick »

Right, lots of other image formats stream correctly, however, PNG does not. Depending on how the coder handles the image format, streaming may not be possible for certain image formats.
User avatar
glennrp
Posts: 1147
Joined: 2006-04-01T08:16:32-07:00
Location: Maryland 39.26.30N 76.16.01W

Re: Stream zero output on palette PNG

Post by glennrp »

magick wrote:Right, lots of other image formats stream correctly, however, PNG does not. Depending on how the coder handles the image format, streaming may not be possible for certain image formats.
It's not the PNG format that is the problem; it's the fact that we are using the sequential decoder in libpng instead of the progressive decoder.
abukaj
Posts: 4
Joined: 2014-06-17T02:51:11-07:00
Authentication code: 6789

Re: Stream zero output on palette PNG

Post by abukaj »

I found that the same problem is with grayscale-PNGs.
glennrp wrote:applications must use a different part of libpng (the progressive reader, found in pngpread.c, instead of the sequential reader, found in pngread.c), to achieve streaming of a PNG datastream.
Would it not move the memory usage of stream out of control? However at the moment it seems that in case of interlaced PNG stream reads whole image into memory. Same is with grayscale non-interlaced PNG. :(

Possibly use of png_set_palette_to_rgb and png_set_gray_to_rgb functions of libpng would be a kind of a walkaround?
glennrp wrote:Certainly there is a bug that is causing "convert" to write zeroes instead of the correct image pixels, though.
Convert works for me with that image.
magick wrote:Right, lots of other image formats stream correctly, however, PNG does not. Depending on how the coder handles the image format, streaming may not be possible for certain image formats.
How can I simply check, if the image is suitable for lightweight streaming it with stream? By "lightweight streaming" I mean not loading a whole image into memory before the RGB stream can be generated.
Post Reply