Any way to speed up many almost identical conversions?

The MagickWand interface is a new high-level C API interface to ImageMagick core methods. We discourage the use of the core methods and encourage the use of this API instead. Post MagickWand questions, bug reports, and suggestions to this forum.
Post Reply
TheSHEEEP
Posts: 3
Joined: 2016-07-13T02:34:39-07:00
Authentication code: 1151

Any way to speed up many almost identical conversions?

Post by TheSHEEEP » 2016-07-13T02:41:50-07:00

Hello there,

I have the following code and want to improve its performance:

Code: Select all

// Reset the wand
ClearMagickWand(_wand);

// Read the data
MagickConstituteImage(_wand, width, height, format, MagickCore::StorageType::CharPixel, rawData);

// Convert to target format
MagickSetImageFormat(_wand, targetFormat);

// Store in output
(*outData) = MagickGetImageBlob(_wand, &outSize);
The performance isn't bad, don't get me wrong. On my rather weak laptop, it is 80-100ms for an image sized 1366x768 to convert from bgra to jpeg.
However, I'd like to improve the performance as much as possible, since MANY of these conversions are taking place (the images become a video in the end - and yes, I HAVE to convert them to something before the video step to save memory).

Since I have (almost) full control over the app, any suggestion would be welcome.

I'd guess that the fact that all the parameters (except the rawData pointer) never change could help somehow, but I am very new to ImageMagick, so maybe someone has an idea.
Multithreading is a possibility, of course, but I may be limited to two threads at once here.

snibgo
Posts: 9399
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Any way to speed up many almost identical conversions?

Post by snibgo » 2016-07-13T03:29:08-07:00

I assume your images are like photos rather than graphics.

In my experience, the time required to write jpeg is related to filesize, which is related to the "-quality" setting. The default is 92, but you may find that something lower (eg 85) works just as well but is twice as fast.
snibgo's IM pages: im.snibgo.com

TheSHEEEP
Posts: 3
Joined: 2016-07-13T02:34:39-07:00
Authentication code: 1151

Re: Any way to speed up many almost identical conversions?

Post by TheSHEEEP » 2016-07-18T22:29:03-07:00

Thank you for that.
The quality setting indeed seems to have lots of influence on performance, which is good.

Is there maybe a way not having to reset the Wand every time and not having to do MagickConstituteImage afterwards?
Maybe some way to simply change the internal data pointer?

snibgo
Posts: 9399
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Any way to speed up many almost identical conversions?

Post by snibgo » 2016-07-19T00:15:45-07:00

I don't use the Wand interface much. One reason is that it adds an extra layer of code between my own code and the useful work that IM actually does.

At the MagickCore level, I deal with images (and lists of images). A wand contains a list of images. At the MagickCore level, I can create an image, then populate it and write it, then re-populate it and write it, and eventually destroy it. I don't know if you can do that with MagickWand.

MagickConstituteImage() creates an image then reads your pixel data, from the array that you supply, into IM's internal format. You have to do that sometime, for each of your images. I don't know if there is a quicker supported method.

Where does your bgra data come from? Jpeg can't store alpha, so that might be a 25% saving.

ConstituteImage works indirectly pixel by pixel, and channel by channel. If your data is already in the format expected by IM (channel order, data type, bits/channel, channels/pixel) then a simple memcpy should be much faster, but you are then more vulnerable to changes in IM (eg between v6 and v7). Even IM itself doesn't do that, but calls SetPixelRed(), SetPixelGreen() etc for each pixel.

ClearMagickWand() removes all the images in the wand, and releases the memory the wand has used. Again, this has to be done. It should be fast already.
snibgo's IM pages: im.snibgo.com

TheSHEEEP
Posts: 3
Joined: 2016-07-13T02:34:39-07:00
Authentication code: 1151

Re: Any way to speed up many almost identical conversions?

Post by TheSHEEEP » 2016-07-19T00:43:01-07:00

Thanks for your help again.
snibgo wrote:I don't use the Wand interface much. One reason is that it adds an extra layer of code between my own code and the useful work that IM actually does.

At the MagickCore level, I deal with images (and lists of images). A wand contains a list of images. At the MagickCore level, I can create an image, then populate it and write it, then re-populate it and write it, and eventually destroy it. I don't know if you can do that with MagickWand.
Interesting, I though the Wand already was the lowest level interface.
I used Magick++ before, but found it far too limiting.

How would my code from above be when written in the lowest level actually possible (and btw, just how many levels are there? :D )
We could also split this thread, or move it to another forum, if that would be more fitting.
snibgo wrote:Where does your bgra data come from? Jpeg can't store alpha, so that might be a 25% saving.
Render target capture from DirectX, but I cannot influence the layout - at least not without losing performance, which I won't do. I certainly don't need the alpha channel, but it is part of the memory block I get.
snibgo wrote:ConstituteImage works indirectly pixel by pixel, and channel by channel. If your data is already in the format expected by IM (channel order, data type, bits/channel, channels/pixel) then a simple memcpy should be much faster, but you are then more vulnerable to changes in IM (eg between v6 and v7).
But that would only work when using the MagickCore directly, correct? Could you give a code example for it?
Version changes won't trouble me, since I can always decide to stick with a version once it works.

snibgo
Posts: 9399
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Any way to speed up many almost identical conversions?

Post by snibgo » 2016-07-19T01:56:32-07:00

For Core versus Wand, see http://www.imagemagick.org/script/magick-core.php and http://www.imagemagick.org/script/magick-wand.php. There isn't much that either can do but the other can't, and I don't suppose there is much difference in performance. For my work, I needed to know what IM did under the hood, and that's at the Core level.

To see the Core code equivalent to what your code does, read the source code for the functions you use. For example, MagickConstituteImage() is in magick-image.c. It does essentially two things: calls the Core function ConstituteImage() to create an image from your array, then a Wand function to insert that image into the wand.

Code: Select all

Could you give a code example for it?
Not for memcpy, no. I don't use that. There are some examples by some hackers on the internet.

I have published load of examples of MagickCore code, mostly on my "Process modules" page. I recently upgraded them to work with both v6 and v7.
snibgo's IM pages: im.snibgo.com

Post Reply