Page 1 of 1

Need for speed!

Posted: 2011-10-21T08:49:12-07:00
by mcaramb
Hi all,

We're a small observatory which hunts for meteors using all-sky cameras.

I've written perl-based software which combines hundreds of still frames using a "lighten" overlay. This gives us a single image which contains the hundreds of images created by our camera every five minutes. As the cameras run at night, and the image is predominately dark, fireballs will appear as streaks in these 5-minute composite images since only the lightest pixels are combined in the final image.

The problem is that the process is so infernally slow. Can anyone think of a faster way to accomplish this task?

We are using PerlMagick on a 4 processor (Xenon 1.9Ghz), 2Gbyte RAM, Ubuntu 11.10 server.
ImageMagick version: 6.6.0-4

Image processing is taking about 1second per frame when all four cameras are processing at once. This might seem fast, but when you're processing four cameras at once, with over a hundred thousand frames each, generating 60+ composites, it can take upwards of 5 hours to process 8 hours of 3-8fps footage.

Besides the OVERLAY function, we are also CROPPING off the time stamp (except for the first image), so it doesn't get overlaid in the composite. Also, we check for corruption in images by comparing the frame file size to that of the first image file size (which is never corrupted). If it's less than half the first image size, we don't process it. Here's a sample of the code for processing just one composite image from a batch of 1500.

Code: Select all

$final=new Image::Magick;
$final->Read('image0.jpg'); # LOAD FIRST FRAME OF SET
$tobeatfilesize=$final->Get('filesize'); # FIRST IMAGE IS NEVER CORRUPTED, STORE FILE SIZE TO COMPARE
for ($x=1; $x<1500; $x++) {
    undef $over;
    $over->Read('image'.$x.'jpg']); # LOAD NEXT FRAME
    $filesize=$over->Get('filesize'); # FILESIZE CHECK TO SEE IF IMAGE IS CORRUPTED
    if ($filesize>($tobeatfilesize/2)) {
    $geo=$width.' '.($height-16);
    $over->Crop(geometry=>$geo,gravity=>'South'); # CROP OUT THE TIME STAMP
    $final->Composite(image=>$over,compose=>'Lighten',gravity=>'South'); # COMBINE INTO THE FINAL COMPOSITE
Any help would be greatly appreciated!


Re: Need for speed!

Posted: 2011-10-21T12:31:48-07:00
by magick
Ensure you don't have a memory leak. Use undef $over:
  • for ($x=1; $x<1500; $x++) {
    my ($over);
    $over->Read('image'.$x.'jpg']); # LOAD NEXT FRAME

Re: Need for speed!

Posted: 2011-10-21T13:42:15-07:00
by mcaramb
look right under the for loop declaration :)

undef $over

is present already :) However, when I typed in the sample above, Initially, I did forget to put


I've just corrected that above...

I'm not using strict variables


Re: Need for speed!

Posted: 2011-10-27T05:52:17-07:00
by magick
Most likely the slow down is due to oversubscribing the threads. When using multiple instances of PerlMagick on a single machine, set MAGICK_THREAD_LIMIT to 1 to reduce contention.