No space for output buffer

Post any defects you find in the released or beta versions of the ImageMagick software here. Include the ImageMagick version, OS, and any command-line required to reproduce the problem. Got a patch for a bug? Post it here.
Locked
snibgo
Posts: 13034
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

No space for output buffer

Post by snibgo »

Windows 7, IM v6.8.7-0. Trying to create a large TIFF file (12 G pixels) fails after 8.5 hours with an error message.

h:\temp and the directory I am running the command in has 1 TB free.

Code: Select all

convert -verbose -monitor -define tiff:rows-per-strip=1 -define registry:temporary-path=h:\temp -size 111394x111394 gradient:red-blue -compress zip h:\huge\huge.tiff
The command successfully creates a 99 GB file in h:\temp, then goes quiet until saying:

Code: Select all

gradient:red-blue=>red-blue GRADIENT 111394x111394 111394x111394+0+0 16-bit sRGB 3324.038u 515:01.471
gradient:red-blue=>h:\huge\huge.tiff GRADIENT 111394x111394 111394x111394+0+0 16-bit sRGB 306B 0.047u 0:01.909
convert.exe: No space for output buffer. `TIFFWriteBufferSetup' @ error/tiff.c/TIFFErrors/563.
What can I do to successfully create the file?

(I have also tried without the "-define tiff:rows-per-strip=1", with the same result.)
snibgo's IM pages: im.snibgo.com

User avatar
magick
Site Admin
Posts: 11254
Joined: 2003-05-31T11:32:55-07:00

Re: No space for output buffer

Post by magick »

Add -limit memory 0 -limit map to your command line. That reduces the memory requirement of the ImageMagick pixel cache, however, we have no control over the resource demands of the TIFF delegate library. Also try saving to another image format such as PNG or JPEG. Does that work for you? If so, the problem is likely in the TIFF delegate library.

snibgo
Posts: 13034
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: No space for output buffer

Post by snibgo »

Thanks. A one-tenth size (1.2 G pixels) has now worked with mpc and "-limit memory 32 -limit map 32", so I'm hopeful about a full-sized version.

I'm currently trying png at full size.
snibgo's IM pages: im.snibgo.com

snibgo
Posts: 13034
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: No space for output buffer

Post by snibgo »

Creating a large file (14.4 G pixels) as png worked. It took about one day.

Code: Select all

convert -monitor -define registry:temporary-path=h:\temp -size 120000x120000 xc: large.png

identify large.png
l.png PNG 120000x120000 120000x120000+0+0 8-bit sRGB 1c 1.748MB 0.000u 0:00.027
So the problem seems to be in writing tiffs. The workaround is to use png, not tiff, for very large images.

I'll try to do more experiments to narrow down the tiff problem. Is there somewhere I can escalate it to? Links at Bugs, Bugzilla, and the TIFF Mailing List are broken.

EDIT: I have uploaded my large png file to https://dl.dropboxusercontent.com/s/1pf ... w40crREcCw in case anyone wants to play with it. Being entirely white it is very boring but compresses well to 1,747,854 bytes.
snibgo's IM pages: im.snibgo.com

User avatar
magick
Site Admin
Posts: 11254
Joined: 2003-05-31T11:32:55-07:00

Re: No space for output buffer

Post by magick »

You might be able to reduce the memory footprint with tiles. Add-define tiff:tile-geometry=128x128, for example, to your command-line.

In most cases, ImageMagick attempts to save a TIFF scanline at a time to ensure a modest memory footprint. In some cases it falls through to a generic TIFF API call that stores the entire image in memory.

Locked