Converting a tiff file to jpeg without increasing filesize

[twn]

Hello,

I'm trying to convert a lot (millions) of tiff files to jpegs without increasing the filesize too much. Here is an example conversion:

Code: Select all

$ convert img.tif img.jpeg
$ du -b img*
214377  img.jpeg
9655    img.tif

So I'm seeing more than a 20x increase in filesize. Ideally I'd keep the filesize roughly the same. I'm very new to any of this stuff, but I have figured out that the tif image has "group 4" compression. I don't really see anything comparable for jpeg. What I'm mainly wondering is the following:

1) Are there standard methods to doing the conversion and roughly keeping the filesizes the same or do most people just fiddle with the quality values until they find something the like? I guess I could just convert random samples with different qualities set and look at blowup in size compared to visual image quality, but hopefully I can avoid that.
2) Is there a better image format (i.e. not jpeg) for this sort of thing? I want to display the images on a webpage which is why I'm converting them, but I honestly don't care much which image format as long as it will render on major browsers.

Thanks a lot for any wisdom!

[fmw42]

I believe that group 4 compression is for bitonal (bilevel b/w) images. JPG does not do well with solid colors. It adds new colors due to the compression. It is also a lossy compression. So you probably should convert to PNG or perhaps better GIF with only two colors, black and white

Code: Select all

convert img.tif -depth 2 -type bilevel PNG8:img.png

or

Code: Select all

convert img.tif -depth 2 -type bilevel img.gif

See http://www.imagemagick.org/script/comma ... s.php#type. For PNG, you can also control the compression and quality. See http://www.imagemagick.org/Usage/formats/#png_quality and http://www.imagemagick.org/script/comma ... hp#quality for PNG or use other tools such as pngcrush to optimize the file size. See http://www.imagemagick.org/Usage/formats/#png_non-im. The GIF may actually be smaller. I have not checked, since PNG may not support bilevel, but is limited to 8-bit images, though compressed with lzw.


Here is a test that I did:

Code: Select all

convert test.tiff -depth 2 -type bilevel PNG8:test.png
convert test.tif -depth 2 -type bilevel PNG8:test.png
convert test.tif -depth 2 -type bilevel test.gif
identify test.tif test.png test.gif
test.tif TIFF 200x200 200x200+0+0 1-bit Bilevel Gray 344B 0.000u 0:00.009
test.png PNG 200x200 200x200+0+0 8-bit sRGB 2c 530B 0.000u 0:00.000
test.gif GIF 200x200 200x200+0+0 8-bit sRGB 2c 560B 0.000u 0:00.000

So it looks like the tiff is smallest followed by the png and gif, which are still slightly larger than the tif



P.S. Please always provide your IM version and platform, since syntax may differ.

[twn]

Thank you very much for all the help! Sorry about the versions, here it is:

Code: Select all

$ convert --version
Version: ImageMagick 6.7.7-10 2014-03-06 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP

I'm running this on ubuntu and imagemagick was installed from the repos:

Code: Select all

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.2 LTS
Release:        14.04
Codename:       trusty

I did some experimenting with the following results:

Code: Select all

$ convert img.tif -depth 2 -type bilevel PNG8:img.png
$ du -b img*
32990   img.png
9655    img.tif
$ convert img.tif -quality 100 -depth 2 -type bilevel img.png
$ du -b img*
22201   img.png
9655    img.tif
$ convert img.tif -quality 0 -depth 2 -type bilevel img.png
$ du -b img*
24264   img.png
9655    img.tif
$ convert img.tif -depth 2 -type bilevel img.gif
$ du -b img*
32528   img.gif
9655    img.tif
$ convert img.tif -quality 100 -depth 2 -type bilevel PNG8:img.png
$ du -b img*
28883   img.png
9655    img.tif

It seems like this gives the best results:

Code: Select all

convert img.tif -quality 100 -depth 2 -type bilevel img.png

I tried pngcrush and it didn't really improve on that setting. I'll keep experimenting, but this blowup in filesize is much more manageable. I'll update the thread for others if I find something better.

Once again thanks so much!

[fmw42]

Be careful. -quality is different for PNG than for JPG. See http://www.imagemagick.org/Usage/formats/#png_quality

[twn]

Thank you very much for the caution!

Though this is a bit unrelated to this thread (though very tied to how I'm using it), do you know of a way to do a batch convert of files in imagemagick? I don't mean in a for loop or using something like gnu parallel, but I mean internal to imagemagick. I'm thinking about the possibility that starting up the convert process takes a significant amount of time compared to the conversion of the image. In that case, I'd certainly only like to start it up once and do many conversions. I kind of doubt that this would be true for something like this, but I have little intuition for the computation costs of such a job.

I feel bad steering the thread somewhere else, but you've been so helpful so far so I'm tempted...

[snibgo]

With IM v6, you can run scripts like this:

Code: Select all

convert @rosewiz.scr NULL:

... where rosewiz.scr is a text file like this:

Code: Select all

rose: -resize 50% +write r.png +delete
wizard: -resize 50% +write w.png

Scripts can be as long as you want. I've tried it with 100,000 lines as a test. Always ensure the script leaves an image in the list, and use NULL: or another output name at the end of the command line.

IM v6 wasn't designed to be run this way, but it works.

IM v7 has a more sophisticated script mechanism.

[twn]

Great thanks so much for the info!

[GeeMack]

Though this is a bit unrelated to this thread (though very tied to how I'm using it), do you know of a way to do a batch convert of files in imagemagick? I don't mean in a for loop or using something like gnu parallel, but I mean internal to imagemagick.

If you're going to run a particular command on all the TIFs in a directory, for example, you can use wildcards in the command something like this...

Code: Select all

convert *.tif -set filename:f "%t" -quality 100 "%[filename:f].jpg"

That will process all the TIF files, converting them to JPG files with 100% quality, and keeping the same file name only with a .jpg extension instead of the .tif of the original.

Also, similar to snibgo's suggestion about running many commands from a single file, you can process tens of thousands of images from a text file list if the process will be the same for all of them. The command would look something like this...

Code: Select all

convert @listofimages.txt -set filename:f "%t" -quality 100 "%[filename:f].jpg"

... where "listofimages.txt" is a simple text list of images, one file per line. You will need to put quote marks around the file names if there are spaces in them. The "@" at the beginning of the text file name in the command tells IM to read from that file. You can even use a list of files from many directories and create the output files in those directories if you include full paths in the file names and "-set" the output names using the appropriate formatting escapes you can find at this link.

Please note, I'm using Windows, so you may need to escape percent signs, brackets, slashes, etc., and/or change double quotes to single quotes to make these work from a *nix command line or script.

[fmw42]

You can do one command line of processing containing only one input and one output for every image in a directory using IM mogrify. So if you just want to convert one image to another format and keep the same name, mogrify is what you want. I always use the -path argument to put the output images into an empty existing directory to avoid overwriting my input images.

cd to existing directory of images

Code: Select all

mogrify -path path2/emptydirectory -format png -quality XX -depth 2 -type bilevel *.tif

where XX is whatever quality value you want. You can add -resize WxH if you want to resize the images.

See
http://www.imagemagick.org/Usage/basics/#mogrify
http://www.imagemagick.org/script/mogrify.php

[twn]

Though this is a bit unrelated to this thread (though very tied to how I'm using it), do you know of a way to do a batch convert of files in imagemagick? I don't mean in a for loop or using something like gnu parallel, but I mean internal to imagemagick.

If you're going to run a particular command on all the TIFs in a directory, for example, you can use wildcards in the command something like this...

Code: Select all

convert *.tif -set filename:f "%t" -quality 100 "%[filename:f].jpg"

That will process all the TIF files, converting them to JPG files with 100% quality, and keeping the same file name only with a .jpg extension instead of the .tif of the original.

Also, similar to snibgo's suggestion about running many commands from a single file, you can process tens of thousands of images from a text file list if the process will be the same for all of them. The command would look something like this...

Code: Select all

convert @listofimages.txt -set filename:f "%t" -quality 100 "%[filename:f].jpg"

... where "listofimages.txt" is a simple text list of images, one file per line. You will need to put quote marks around the file names if there are spaces in them. The "@" at the beginning of the text file name in the command tells IM to read from that file. You can even use a list of files from many directories and create the output files in those directories if you include full paths in the file names and "-set" the output names using the appropriate formatting escapes you can find at this link.

Please note, I'm using Windows, so you may need to escape percent signs, brackets, slashes, etc., and/or change double quotes to single quotes to make these work from a *nix command line or script.

I tried a basic variation of this method and it's not giving me the results I want. What I tried was to act on different groups of files. Say my folder contains 0001.tiff, ..., 1000.tiff. Then what I would do is generate different batches of commands to do the conversion. For example, if the batches sizes were 1, I'd use the following commands:

Code: Select all

convert 0001.tiff -set filename:f "%t" -quality 100 {1}"%[filename:f].png"
...
convert 1000.tiff -set filename:f "%t" -quality 100 {1}"%[filename:f].png"

If I used a batch size of 3, the commands would be:

Code: Select all

convert 0001.tiff 0002.tiff 0003.tiff -set filename:f "%t" -quality 100 {1}"%[filename:f].png"
convert 0004.tiff 0005.tiff 0006.tiff -set filename:f "%t" -quality 100 {1}"%[filename:f].png"
...
convert 9997.tiff 9998.tiff 9999.tiff -set filename:f "%t" -quality 100 {1}"%[filename:f].png"
convert 1000.tiff -set filename:f "%t" -quality 100 {1}"%[filename:f].png"

I would execute these commands in parallel. The end result is that the operation is actually _slower_ in increasing batch size numbers which kind of should make no sense since if anything it should be a little faster due to startup time in the process. It's a bit odd. Given these results, I see no reason to use this scripting mechanism. (Of course I could try to other scripting mechanisms mentioned in this thread as well, but I might be running of time for this kind of experimentation and may just need to move on.)

Thanks for all the help everyone!