memory exhaust - how to process images individually?

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
lada04

memory exhaust - how to process images individually?

Post by lada04 » 2009-02-12T12:05:51-07:00

I try to combine 373 greyscale jpg-images into a single pdf-doc,
each images is around 2300x3600 pixels.
The following command

Code: Select all

convert -density 200  -compress JPEG */*.jpg zzz.pdf
exhaust all available memory, physical (2GB), swap (4GB) and tmp-file
(18GB free space on the filesystem, where /tmp resides), but cannot finish the job.
I guess, convert tries to read all 373 images into memory before writing the pdf.
Could anybody tell me a way to force convert, to process the images sequentially?


--
Version: ImageMagick 6.2.3 10/12/07 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2005 ImageMagick Studio LLC

User avatar
magick
Site Admin
Posts: 10690
Joined: 2003-05-31T11:32:55-07:00

Re: memory exhaust - how to process images individually?

Post by magick » 2009-02-12T12:12:07-07:00

Its always best to have the latest ImageMagick release, current is 6.4.9-4. To reduce resource requirements you can install the Q8 version rather than the default Q16 version (has 1/2 the resource requirement of Q16). Otherwise if you have lots of disk space, your command will run. Assume there is 300GB available at /data/tmp. Try these command:
  • export MAGICK_TMPDIR=/data/tmp
    convert -limit memory 1mb -limit map 1mb -density 200 -compress JPEG */*.jpg zzz.pdf
Otherwise you would need to convert 1 page at a time and use a PDF script that collates individual pages into one PDF.

lada04

Re: memory exhaust - how to process images individually?

Post by lada04 » 2009-02-12T12:35:05-07:00

thank you for your answer.

disk space is not available as needed, if I understood right, the Q16 version uses 16bit/per channel, and I have to expect 4 channels for the pixel-cache, which result in
373*2300*3600*16*4/8/1024/1024/1024
=23GB memory usage?

Can anybody recommend a tool (script) to combine single pdf-files into one?

lada04

Re: memory exhaust - how to process images individually?

Post by lada04 » 2009-02-13T01:23:10-07:00

problem solved:
convert each file individually and use pdfjoin from the PDFjam-package
http://www.warwick.ac.uk/go/pdfjam
gives perfect result!

User avatar
anthony
Posts: 8869
Joined: 2004-05-31T19:27:03-07:00
Authentication code: 8675308
Location: Brisbane, Australia

Re: memory exhaust - how to process images individually?

Post by anthony » 2009-02-14T04:04:33-07:00

Other suggestions made previously...

Using perl module PDF::Reuse...

Code: Select all

#!/usr/bin/perl
#  Script   pdf-combiner.pl
use strict;
use warnings;
use PDF::Reuse;

prFile('combo.pdf'); # Output.
for (qw/a b c d/) # Inputs.
{
  prImage("result_$_.pdf");
  prPage();
}
prEnd();
Or a JAVA toolkit

Code: Select all

#!/bin/bash
for x in ./*.jpeg
do
    echo $x to ${x}.pdf
    convert $x -quality 75 ${x}.pdf
done

echo Merging...
java tool.pdf.Merge *.pdf
NOTE I have not tried these out so I do not know how good they are.

I have added your 'find' to IM Examples, Common Formats, Postscript and PDF
http://www.imagemagick.org/Usage/formats/#pdf
Anthony Thyssen -- Webmaster for ImageMagick Example Pages
http://www.imagemagick.org/Usage/

Post Reply