malloc sizes question

Questions and postings pertaining to the development of ImageMagick, feature enhancements, and ImageMagick internals. ImageMagick source code and algorithms are discussed here. Usage questions which are too arcane for the normal user list should also be posted here.
Post Reply
cloud
Posts: 26
Joined: 2009-06-26T07:09:31-07:00
Authentication code: 8675309
Location: Zürich, Switzerland | Como, Italy

malloc sizes question

Post by cloud » 2010-06-15T12:07:57-07:00

MagickGetImagesBlob and MagickWriteImages (i.e. when dealing with multiple-images images, animated GIFs, for instance) callstack looks something like this:

Code: Select all

MagickGetImagesBlob
ImagesToBlob
WriteImages
WriteImage
WriteGIFImage
SetImageType
QuantizeImage
GetCubeInfo
libSystem.B.dylib malloc
Malloc appears to be asking for 64.0MB - that's with --with-quantum-depth=8 (if it's dependent on this).

Now, on the iPhone applications can only allocate at max 25MB of memory or they get shut-down automatically by the OS. Now, is there a workaround around this? I'm testing it with a 4KB image, not with impossibly big ones.

Thank you in advance to the developers for some knowledge/workaround for this issue.

Best regards,

~Claudio
IM compilation script and test project: https://github.com/marforic/imagemagick_lib_iphone (always up-to-date)
IM latest iOS binaries: ftp://ftp.imagemagick.org/pub/ImageMagick/iOS/

User avatar
magick
Site Admin
Posts: 11097
Joined: 2003-05-31T11:32:55-07:00

Re: malloc sizes question

Post by magick » 2010-06-15T14:48:29-07:00

Try adding -limit memory 25mb on your command line. If that fixes the problem, you can make it permanent by setting this limit in your policy.xml file. You can try other limits as well. For example, set -limit memory 10mb or -limit area 4kb. Add -debug cache to see how the pixel cache is allocated and add -monitor to monitor the convert command progress.

If you come up with a set of limits that work well, post them here and we will set these as the default for the iPhone build.

cloud
Posts: 26
Joined: 2009-06-26T07:09:31-07:00
Authentication code: 8675309
Location: Zürich, Switzerland | Como, Italy

Re: malloc sizes question

Post by cloud » 2010-06-16T08:14:55-07:00

Hi,
I tried setting MAGICK_MEMORY_LIMIT like

Code: Select all

setenv("MAGICK_MEMORY_LIMIT", "10mb", 1);
or in the policy.xml file like:

Code: Select all

<policymap>
    <policy domain="resource" name="memory" value="10mb"/>
</policymap>
Nonetheless the function MagickWriteImages will eventually call

Code: Select all

MagickWriteImages
   2 AnimateGif AcquireQuantumMemory
   1 AnimateGif AcquireMagickMemory
passing to the latter as argument 67108864 (the 64.0 MB shown as asked by malloc).

This gets called 3 times (note, I'm trying to write a 4-frames animated gif).

Am I doing something wrong?

Regards,

~Claudio
IM compilation script and test project: https://github.com/marforic/imagemagick_lib_iphone (always up-to-date)
IM latest iOS binaries: ftp://ftp.imagemagick.org/pub/ImageMagick/iOS/

User avatar
magick
Site Admin
Posts: 11097
Joined: 2003-05-31T11:32:55-07:00

Re: malloc sizes question

Post by magick » 2010-06-16T09:26:01-07:00

Perhaps the color reduction algorithm is consuming the memory. Try writing to PNM (e.g. image.png) rather than GIF (e.g. image.gif). Does that work? If so, we can try to reduce the memory requirements of the color reduction algorithm.

cloud
Posts: 26
Joined: 2009-06-26T07:09:31-07:00
Authentication code: 8675309
Location: Zürich, Switzerland | Como, Italy

Re: malloc sizes question

Post by cloud » 2010-06-19T09:22:03-07:00

I tried as you suggested but IM doesn't let me create "animated PNGs". The code used is:

Code: Select all

MagickSetFormat(magick_wand, "png");
if (MagickWriteImages(magick_wand, "/path/to/img/img.png", true) == MagickFalse) {
	ThrowWandException(magick_wand);
}
But rather than creating a single .png image it creates *-0.png, *-1.png, ...

If I use the same code but setting the format to gif (MagickSetFormat(magick_wand, "gif");) it creates a single image *.gif with inside 4 frames.

Anyway, writing a non-animated gif doesn't malloc those 64MB. Neither does writing multiple pngs as explained above. Only writing out animated .gifs uses that big allocation.

Regards,

~C
IM compilation script and test project: https://github.com/marforic/imagemagick_lib_iphone (always up-to-date)
IM latest iOS binaries: ftp://ftp.imagemagick.org/pub/ImageMagick/iOS/

User avatar
magick
Site Admin
Posts: 11097
Joined: 2003-05-31T11:32:55-07:00

Re: malloc sizes question

Post by magick » 2010-06-19T09:44:51-07:00

We should have suggested MNG rather than PNG. MNG accepts an image sequence, whereas PNG accepts one frame per image.

We reduced the memory requirements for dithering on the iPhone from 64MB to 4MB. In the mean-time, add +dither to your command line:
  • convert *.jpg +dither image.gif

cloud
Posts: 26
Joined: 2009-06-26T07:09:31-07:00
Authentication code: 8675309
Location: Zürich, Switzerland | Como, Italy

Re: malloc sizes question

Post by cloud » 2010-06-19T11:35:31-07:00

Thank you for the clarification.
Using MNG is using again the 64MB malloc.
magick wrote: We reduced the memory requirements for dithering on the iPhone from 64MB to 4MB. In the mean-time, add +dither to your command line:
  • convert *.jpg +dither image.gif
I will look forward to an API enabled version for this to test it on the device.

Regards,

~C
IM compilation script and test project: https://github.com/marforic/imagemagick_lib_iphone (always up-to-date)
IM latest iOS binaries: ftp://ftp.imagemagick.org/pub/ImageMagick/iOS/

cloud
Posts: 26
Joined: 2009-06-26T07:09:31-07:00
Authentication code: 8675309
Location: Zürich, Switzerland | Como, Italy

Re: malloc sizes question

Post by cloud » 2010-06-20T07:02:32-07:00

magick wrote:We should have suggested MNG rather than PNG. MNG accepts an image sequence, whereas PNG accepts one frame per image.

We reduced the memory requirements for dithering on the iPhone from 64MB to 4MB. In the mean-time, add +dither to your command line:
  • convert *.jpg +dither image.gif
compiled and tested SVN trunk. Works like a charm.

~C
IM compilation script and test project: https://github.com/marforic/imagemagick_lib_iphone (always up-to-date)
IM latest iOS binaries: ftp://ftp.imagemagick.org/pub/ImageMagick/iOS/

Post Reply