Appending Large Images Using Streams

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
kevinswarner
Posts: 2
Joined: 2018-07-07T09:28:20-07:00
Authentication code: 1152

Appending Large Images Using Streams

Post by kevinswarner » 2018-07-07T09:40:16-07:00

I am working on a project where I need to append two very large TIF images vertically. Each image is several hundred MBs. We have very limited resources as we are using Amazon Lambda function compute service. So, we have memory and disk limitations.

I have been using the following to do the work.

Code: Select all

magick convert image1.tif image2.tif -append result.tif
Because of the size of the images, we are running into resource problems, both disk and memory.

I have tried using -limit, but this only partially helps.

I would like to know if I can use the "stream" option (https://www.imagemagick.org/script/stream.php) in ImageMagick to perform this operation using less memory and disk space. I understand the basic concepts of streaming, but from all the examples I have seen on using "stream", I cannot figure out how to do this. Again, just looking for example usage.

Thanks for your help.

User avatar
fmw42
Posts: 24125
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Appending Large Images Using Streams

Post by fmw42 » 2018-07-07T10:19:48-07:00

I cannot answer your question about streaming. But in Imagemagick 7, you should use just magick and not magick convert, nor just convert.

User avatar
magick
Site Admin
Posts: 10865
Joined: 2003-05-31T11:32:55-07:00

Re: Appending Large Images Using Streams

Post by magick » 2018-07-07T13:20:41-07:00

Streaming won't help. Have you reviewed https://www.imagemagick.org/script/arch ... tera-pixel? You should be able to convert any sized image as long as you point the pixel cache to a disk partition that has plenty of free space. To track resource requirements, add -debug cache to your command-line.

kevinswarner
Posts: 2
Joined: 2018-07-07T09:28:20-07:00
Authentication code: 1152

Re: Appending Large Images Using Streams

Post by kevinswarner » 2018-07-07T15:26:00-07:00

Thanks, but we are also limited on disk space. Our limits are 1 GB temporary disk space and 3GB memory. The two files we are appending are around 300 MB each. Even with -limits in place, we are exceeding one resource or the other. What is the reason streaming will not work? I guess I am just looking for more detail.

I am moving to using a custom build of IM with quantum depth of 8 to see if that helps. I have read in a few places that this will "halve" the memory requirements / usage.

Thanks.

Post Reply