Resizing huge images failing

Questions and postings pertaining to the development of ImageMagick, feature enhancements, and ImageMagick internals. ImageMagick source code and algorithms are discussed here. Usage questions which are too arcane for the normal user list should also be posted here.
Post Reply
matt1008
Posts: 5
Joined: 2016-09-23T10:53:54-07:00
Authentication code: 1151

Resizing huge images failing

Post by matt1008 » 2016-09-23T11:21:45-07:00

Hi - I'm trying to reduce a tif image from 94095x54316 to 9410x5432 by using the -resize 10% option, but getting an "unable to extend cache" error:

C:\Users\Matt>convert -monitor -limit memory 2GiB -limit map 4GiB -define registry:temporary-path=c:\temp Huge.tif -resize 10% Haircut.tif

Error message:
convert: unable to extend cache 'Huge.tif': No space left on device @ error/cache.c/OpenPixelCache/3607.
convert: no images defined `Haircut.tif' @ error/convert.c/ConvertImageCommand/3253.

I tried all the suggestions for increasing cache, but still get errors. Any ideas?

I have 32GB of RAM / Windows 10

Thank you!

snibgo
Posts: 9718
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Resizing huge images failing

Post by snibgo » 2016-09-23T11:33:48-07:00

What have you run out of, disk space or memory? "No space left on device" sounds like disk.
snibgo's IM pages: im.snibgo.com

matt1008
Posts: 5
Joined: 2016-09-23T10:53:54-07:00
Authentication code: 1151

Re: Resizing huge images failing

Post by matt1008 » 2016-09-23T11:41:07-07:00

I'm assuming memory. I have 28GB free on my C:\ drive. A little low but should be enough. Let me free up some space to see if that makes a difference.
Thanks

snibgo
Posts: 9718
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Resizing huge images failing

Post by snibgo » 2016-09-23T12:32:31-07:00

Just to store the image in memory needs 94095*54316 * 8 / 1e9 = 40.88 GB. So you ran out of disk (virtual memory).
snibgo's IM pages: im.snibgo.com

matt1008
Posts: 5
Joined: 2016-09-23T10:53:54-07:00
Authentication code: 1151

Re: Resizing huge images failing

Post by matt1008 » 2016-09-23T14:05:39-07:00

Looks like you were dead on with that tip. It's processing...I'll update when it's done. Thank you!

The "8" represents 4 channels at 2 bytes each? If I used Q8 it would take half the space?

[UPDATE]
It got farther but still ended up with a "unable to extend cache"

User avatar
fmw42
Posts: 22651
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Resizing huge images failing

Post by fmw42 » 2016-09-23T14:52:25-07:00

Could you be running out of temp space?

matt1008
Posts: 5
Joined: 2016-09-23T10:53:54-07:00
Authentication code: 1151

Re: Resizing huge images failing

Post by matt1008 » 2016-09-23T15:26:12-07:00

I don't think so. My D drive has 400GB free and that's where I'm loading and saving.

I took a look at LibTiff and was able to extract a smaller page from the 8 paged Tif. I wasn't sure what approach would work best, but it looks like I'll be using LibTiff for this solution.

Thanks for all your help. I really appreciate it.
Matt

User avatar
fmw42
Posts: 22651
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Resizing huge images failing

Post by fmw42 » 2016-09-23T18:08:20-07:00

Is your temp directory already too full to allow processing of your file or have some limit set for it? I am not an expert on this, but thought you might check your temp directory to see that it is not overly full or getting too full while processing.

User avatar
fmw42
Posts: 22651
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Resizing huge images failing

Post by fmw42 » 2016-09-23T18:08:49-07:00

What version of Libtiff is IM using on your system? Do you need to upgrade it?

snibgo
Posts: 9718
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Resizing huge images failing

Post by snibgo » 2016-09-23T18:13:30-07:00

matt1008 wrote:My D drive has 400GB free and that's where I'm loading and saving.
But you had:
matt1008 wrote:-define registry:temporary-path=c:\temp
If there isn't space in real memory for pixels, they are put in a large file there.
matt1008 wrote:The "8" represents 4 channels at 2 bytes each? If I used Q8 it would take half the space?
Yes and yes.
snibgo's IM pages: im.snibgo.com

matt1008
Posts: 5
Joined: 2016-09-23T10:53:54-07:00
Authentication code: 1151

Re: Resizing huge images failing

Post by matt1008 » 2016-09-24T06:39:00-07:00

I changed the script to D:\Temp folder and did my load and save from there. It got farther but eventually error-ed out with "unable to extend cache". The main difference between the two approaches is I'm not loading the highest resolution, I'm cycling thru them and loading a much smaller one. The image has 8 resolutions (called directories). I found some code to save a directory to a tif, so I was able to extract the resolution I wanted instead of resizing to get it.

Can ImageMagick extract a specific resolution from a mulit-resolution tif? Now that would be cool. Thanks for your help!

snibgo
Posts: 9718
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Resizing huge images failing

Post by snibgo » 2016-09-24T08:16:11-07:00

IM (may or may not) see the multiple resolutions as different images.
Try:

Code: Select all

identify Huge.tif
If there is only one line of output, then IM sees only one image. But there may be multiple lines, one per resolution, so you can convert the one you want, eg:

Code: Select all

convert Huge.tif[3] out.tif
The LIBTIFF toolset, available for Windows from Cygwin, generally read tiff files serially rather than attempting to read the entire image into memory. For example, tiffcrop might successfully break a huge tifff file into pieces. And those tools may understand multi-resolution data that IM doesn't understand.
snibgo's IM pages: im.snibgo.com

Post Reply