Page 1 of 1

magick translate PAM to PNG ignores compression option

Posted: 2020-02-07T12:58:29-07:00
by PatrickPhotog
magick appears to apply compression to .PNG files even when told not to: ... p#compress
Use pixel compression specified by type when writing the image.
Specify +compress to store the binary image in an uncompressed format.
-compress type ; Choices are: None, ...
Tried both.

Code: Select all

pat@nova:~/exp/pam> magick  +compress;  
pat@nova:~/exp/pam> magick  -compress None;  
pat@nova:~/exp/pam> lsr -s*.png         145317167       98943404   << Way too small to hold uncompressed bitmap      98943404

Code: Select all

pat@nova:~/exp/pam> head -n7
WIDTH 7378
MAXVAL 65535
pat@nova:~/exp/pam> head -n7 | wc
      7      12      79
With a 79 byte header, the PAM uncompressed bitmap size is 145317088 Bytes
Checking: m 145317088 /7378 /4924 -> 4 bytes per pixel. No compression.

I noticed a similar thread 5 years ago in which the Linux always compressed but the windoz version worked.

I can't find a Linux viewer for PAM files so I need to convert them for viewing.
For temporary local viewing and a 10TB drive, compression is unnecessary.

pat@nova:~/bin> magick -version
Version: ImageMagick 7.0.8-20 Q16 x86_64 2018-12-25
Copyright: © 1999-2019 ImageMagick Studio LLC
Features: Cipher DPC HDRI Modules OpenMP
Delegates (built-in): bzlib fontconfig freetype jng jpeg lcms ltdl png tiff webp x zlib

uname -a (OpenSuse Leap 15)
Linux nova 4.12.14-lp150.12.82-default #1 SMP Tue Nov 12 16:32:38 UTC 2019 (c939e24) x86_64 x86_64 x86_64 GNU/Linux

Re: magick translate PAM to PNG ignores compression option

Posted: 2020-02-07T13:51:06-07:00
by fmw42
PNG compression is lossless. So why does it matter if it is compressed or not?

See ... hp#quality

Re: magick translate PAM to PNG ignores compression option

Posted: 2020-02-07T16:00:39-07:00
by PatrickPhotog
Time and data quality are both important.
If you don't lose any data, time does not matter? I don't follow your logic.

It takes 13 seconds to compress a 145MB PAM to PNG. And, I have hundreds to do per job, thousands per day some days.

Writing a PAM file takes 122ms:
WBHB: Wrote 145.317167MB (79 + 145317088) in 121.728ms, 1193.786 MB/s to file

timeit magick -compress None;
TI: Elapsed time = 13.222 sec -> 7.42 MB/sec

mult 13.333 /0.121728 -> 109.5 times as long to write. A fraction of a second is not bad. Over a dozen seconds adds up over a day / week.

You are saving 46 MB of disk space at a cost of 12 seconds of your time. A 2TB drive costs $84. Do the math...
$84 / 2E12B * 46 E6 B = $0.001932 worth of disk space saved if you never use it again.

Let's calculate your hourly value:
0.001921$ / 12 sec * 60 sec/min * 60 min/hr = 57.96 CENTS per hour. And, you can rewrite that disk space a million times.

My time is worth more than that.

Compression is a useful option. I was hoping that somebody could spot my command line error.

Re: magick translate PAM to PNG ignores compression option

Posted: 2020-02-07T16:15:54-07:00
by fmw42
OK. If time is an issue, I see your point. Have you tried the commands for setting the quality and compression via the defines in the second link I sent? I believe they give better control and the first link suggests that.

What about lossless TIFF?

Re: magick translate PAM to PNG ignores compression option

Posted: 2020-02-07T16:36:06-07:00
by magick
Try this command:

Code: Select all

magick  -define png:compression-level=0 -define png:compression-filter=0 -define png:color-type=2
This command took less than 2 seconds to complete for us for a mocked up 145MB PAM image file.

Re: magick translate PAM to PNG ignores compression option

Posted: 2020-02-07T18:24:10-07:00
by fmw42
I would look into saving as TIFF rather than PNG. The file size is smaller and it might run faster. Test:

Code: Select all

convert logo: logo.png
convert logo.png  -define png:compression-level=0 -define png:compression-filter=0 -define png:color-type=2 x1.png
convert logo.png -compress none x1.tif
identify logo.png x1.png x1.tif
logo.png PNG 640x480 640x480+0+0 8-bit sRGB 256c 27398B 0.000u 0:00.000
x1.png PNG 640x480 640x480+0+0 8-bit sRGB 922814B 0.000u 0:00.000
x1.tif TIFF 640x480 640x480+0+0 8-bit sRGB 256c 309006B 0.000u 0:00.000

It is strange (BUG?) that using -quality 00 does not achieve the same results.

Code: Select all

convert logo.png -quality 00 PNG24:x2.png
identify x2.png
x2.png PNG 640x480 640x480+0+0 8-bit sRGB 67178B 0.000u 0:00.000

This produces a much smaller (more compressed) result.

Re: magick translate PAM to PNG ignores compression option

Posted: 2020-02-07T20:21:37-07:00
by PatrickPhotog
I had to change the PNG color type to 4 for Grayscale - Alpha:

Code: Select all

timeit magick  -define png:compression-level=0 -define png:compression-filter=0 -define png:color-type=4
TI: Elapsed time = 4.183 sec
The size is right and the channels are right and it is 3x faster. Much better.

I have a Samsung 2TB 970 EVO+ PCIe 3, a 4 GHz Skylake and 64GB of 3200 MHz CL16 DDR4. There is no hardware I know of anywhere near twice this fast but you are seeing < 2seconds and I am getting > 4 sec. Impressive!

Are you using a custom magick build fully optimized for your chipset with -march $CPU, -Ofast, etc.? I may be running on Suse code de-optimized for least common denominator hardware like NAHALEM or core2. Pthreads?

You are seeing great performance! I may attempt a source build! I get a 6x speed improvement compiling dcraw from source vs binary!

Re: magick translate PAM to PNG ignores compression option

Posted: 2020-02-07T20:54:14-07:00
by PatrickPhotog

Code: Select all

timeit magick  +compress
TI: Elapsed time = 443.617 ms
Darn fast! The file size is right at ~145MB

Code: Select all

Channel depth:
    Gray: 16-bit
    Alpha: 16-bit
      min: 0  (0)
      max: 65535 (1)
      mean: 6948.25 (0.106023)
      standard deviation: 12953.1 (0.197652)
      kurtosis: 7.53538
      skewness: 2.75987
      entropy: 0.634877
      min: 0  (0)
      max: 42693 (0.651453)
      mean: 4807.5 (0.0733578)
      standard deviation: 9931.49 (0.151545)
      kurtosis: 5.08598
      skewness: 2.46357
      entropy: 0.640839
The Alpha looks right in `identify -verbose` but is displays with opacity === 100%, zero transparency anywhere in multiple viewers.
It is a PGM, not a transparent file!

Gwenview, Okular 100% opaque. Crashes in XV. Black in ShowFoto. Xnview shows it solid.

TIFF is such a byzantine format (IRS invention?). That's why I like the 79 byte, fully readable PAM header and simple bitmap.

XNview actually decodes the PAM file, but it shows Acid Trip colors instead of boring grayscale. Gruesome color mapping.

How hard could it be to write a viewer for a PAM file? Essentially a PGM with Opacity?!