Too much memory usage

Post any defects you find in the released or beta versions of the ImageMagick software here. Include the ImageMagick version, OS, and any command-line required to reproduce the problem. Got a patch for a bug? Post it here.
Locked
naoliv
Posts: 110
Joined: 2007-12-10T18:54:27-07:00
Location: Brazil

Too much memory usage

Post by naoliv »

Hi!

These two (http://bugs.debian.org/389604 and http://bugs.debian.org/349200) bugs on Debian are related about memory usage with ImageMagick.

The first one is about display using too much memory to display images.
Using this image, for example: http://naoliv.googlepages.com/image.png, with 1.7 MB of size.

Doing a simple "display image.png" I see it using 248 MB of RAM here, while when using feh, it uses only 53 MB.

The second problem (bug 349200) is also related. The user is using montage with some JPEG images (summing 140 MB on his first test and 188 MB on his last one).
Memory usage explodes when he does this:

Code: Select all

montage -label %f -background '#fff' -geometry +4+4 -resize 10% *-*.jpg index.jpg
Thank you!

User avatar
magick
Site Admin
Posts: 11254
Joined: 2003-05-31T11:32:55-07:00

Re: Too much memory usage

Post by magick »

Add this to your command line to trade off memory usage for I/O usage: -limit memory 16mb -limit map 32mb. Another option is to build the 8-bit version of ImageMagick (add --with-quantum-depth=8 to the configure command line). The 8-bit version uses 100% less memory than the default 16-bit version.

broucaries
Posts: 467
Joined: 2008-12-21T11:51:10-07:00

Re: Too much memory usage

Post by broucaries »

Hi,

User use this command as recommended:
$ montage -limit memory 16000000 -limit map 160000000 -label %f -background '#fff' -geometry +4+4 -resize 10% '*_*.jpg' index.jpg

But get
...
montage: unable to extend cache `20080216_150344.jpg': No space left on device.
montage: unable to extend cache `20080217_015805.jpg': No space left on device.
montage: unable to extend cache `20080217_015859.jpg': No space left on device.

top said
8247 flip 20 0 203m 156m 152m D 17.3 16.6 0:04.74 montage

And it does not work :-(

Any test to do?

User avatar
magick
Site Admin
Posts: 11254
Joined: 2003-05-31T11:32:55-07:00

Re: Too much memory usage

Post by magick »

The exception is telling you that there is not enough free space on your disk to create the image. Try freeing space on your disk or point ImageMagick to another disk with plenty of free space (with the MAGICK_TMPDIR environment variable).

broucaries
Posts: 467
Joined: 2008-12-21T11:51:10-07:00

Re: Too much memory usage

Post by broucaries »

Thank you.

broucaries
Posts: 467
Joined: 2008-12-21T11:51:10-07:00

Re: Too much memory usage

Post by broucaries »

The user checked if they are neouth space on the temporary dir and check quota, It seems ok but th job need more than 2Go of memory :(

We asked for a strace (http://bugs.debian.org/cgi-bin/bugrepor ... bug=349200)

Could you check if something goes wrong, and perhaps give us some debugging tips? Next step will be to ask for a ltrace, using debugging symbol I suppose ?

Regards

Bastien

User avatar
magick
Site Admin
Posts: 11254
Joined: 2003-05-31T11:32:55-07:00

Re: Too much memory usage

Post by magick »

Try this command
  • montage -limit memory 0 -limit map 0 -label %f -background '#fff' -geometry +4+4 -resize 10% '*_*.jpg' index.jpg
That forces the entire pixel cache to disk. See http://www.imagemagick.org/script/architecture.php for a discussion of the pixel cache and resource limits.

pmarek

Re: Too much memory usage

Post by pmarek »

Hello everybody,

I'm the user with the montage problem.

I just started

Code: Select all

MAGICK_TMPDIR=. montage -limit memory 0 -limit map 0 -label %f -background '#fff' -geometry +4+4 -resize 10% '*_*.jpg' index.jpg
as requested, in a directory with 68 jpgs (200MB).

After 10 minutes real time I get 66MB memory usage of montage, and the directory is at 4GB used.

Why is that so? I had estimated 68 files, with destination size about 350x250, gives for RGB (24bit) uncompressed about 18MB memory usage - which should very easily fit into main memory.

I now stopped montage, after about 2min user/16 min real time.
A short test shows that this should run much faster:

Code: Select all

$ time convert -label %f -background '#fff' -geometry +4+4 -resize 10%   20080921_142956.jpg aaa.jpg
real    0m1.897s
user    0m1.588s
sys     0m0.128s
$ identify aaa.jpg
aaa.jpg JPEG 326x245 326x245+0+0 DirectClass 8-bit 77.3867kb
flip@heim:/home/allgemein/pictures/2008/2008-09$ ls -la aaa.jpg
-rw-r--r-- 1 flip users 79244 13. Jän 20:46 aaa.jpg
Why does convert/montage of some files take so much more memory and time?

Thank you for your time and effort!

User avatar
magick
Site Admin
Posts: 11254
Joined: 2003-05-31T11:32:55-07:00

Re: Too much memory usage

Post by magick »

Try this command (assumes thumbnails at 128 pixels):
  • montage -label %f -background '#fff' -size 128x128 -geometry 128x128+4+4 '*_*.jpg' index.jpg

pmarek

Re: Too much memory usage

Post by pmarek »

Thank you, that seems to work nicely.

Is there some explanation that I can understand?

Thank you very much!


Regards,

Phil

User avatar
magick
Site Admin
Posts: 11254
Joined: 2003-05-31T11:32:55-07:00

Re: Too much memory usage

Post by magick »

The size option causes ImageMagick to only store JPEG thumbnails in memory while its generating the montage layout. Without this option, the entire image is temporarily stored in memory until the montage page is generated.

broucaries
Posts: 467
Joined: 2008-12-21T11:51:10-07:00

Re: Too much memory usage

Post by broucaries »

Could this usage documented ? It is really a user trap !

Regards

This command should be included in the exemple section of theman page with a note explaining why dropping -size could fail.

Bastien

User avatar
magick
Site Admin
Posts: 11254
Joined: 2003-05-31T11:32:55-07:00

Re: Too much memory usage

Post by magick »

This feature is documented here: http://www.imagemagick.org/Usage/formats/#jpg_read. In addition we have someone (Rick Mabry) editting the documentation now. It may take several weeks because of our other responsibilities.

broucaries
Posts: 467
Joined: 2008-12-21T11:51:10-07:00

Re: Too much memory usage

Post by broucaries »

thank you for your answer.

Regards

User avatar
rmabry
Posts: 148
Joined: 2004-04-13T11:25:27-07:00

Re: Too much memory usage

Post by rmabry »

magick wrote:In addition we have someone (Rick Mabry) editing the documentation now. It may take several weeks because of our other responsibilities.
It will take many, many weeks. Maybe years. Also, I am presently not editing anything beyond the command-line-options and command-line-processing pages.

Rick

Locked