MAGICK_THREAD_LIMIT = 0 doesn't disable OpenMP

[langer]

Hi --

I develop and distribute a program that uses pthreads and Magick++. Since pthreads aren't compatible with OpenMP on many platforms (including OS X) users of our software have to build Magick++ with the --disable-openmp option. (If it's omitted, the program crashes on the first Magick++ call from a subthread.) Two of the OS X package managers, fink and macports, distribute ImageMagick with openMP enabled, so our users can't use the package managers and have to build ImageMagick manually. This is an enormous burden for some of them. The maintainers of the fink and macports ImageMagick packages don't want to disable OpenMP by default, for obvious reasons.

So, the question is, is it possible to completely disable OpenMP at run-time? Setting MAGICK_THREAD_LIMIT to 1 (or even 0) doesn't do the trick. The program still crashes in gomp_resolve_num_threads. Here's part of the trace:

• Thread 2 Crashed:
  0 libMagickCore.3.dylib 0x0000000102c9e12c gomp_resolve_num_threads + 28
  1 libMagickCore.3.dylib 0x0000000102b34c95 AcquirePixelCache + 325
  2 libMagickCore.3.dylib 0x0000000102be9257 AcquireImage + 375
  3 libMagick++.3.dylib 0x000000010298a0b6 Magick::ImageRef::ImageRef() + 86
  4 libMagick++.3.dylib 0x0000000102980632 Magick::Image::Image() + 50

Thanks.

-- Steve

[langer]

I forgot to mention that I'm using IM 6.6.1 and OS X 10.6.4.

[magick]

OpenMP support is compiled in at build time. Setting MAGICK_THREAD_LIMIT simply calls omp_set_num_threads(). OpenMP and pthreads should interoperate. Its possible that libgomp under Mac OS X is buggy.

[langer]

My understanding, from discussions at OpenMP.org, is that the OpenMP standard doesn't say anything about how it interacts with threads, and so the behavior is undefined. See http://openmp.org/forum/viewtopic.php?f ... read#p2556, for example. This means that libMagick++ can't be used safely in any threaded program, unless ImageMagick is built with --disable-openmp. It would be useful if convert, et al. could use openmp without requiring it for all programs linking to libMagick.

-- Steve

[magick]

Dang, we had a Magick++ pthread regression test that works fine with OpenMP but we only tested it under Fedora Core 13.

We wonder if you could wrap your Magick++ around a OpenMP pragma:

• #pragma omp single
  {
  InitializeMagick()...
  Image image( geometry, canvasColor);
  ...
  }

OpenMP is enabled at the scanline level. Each algorithm is divided into scanline chunks that are passed to however many processors are available on your system. It can speed up algorithms significantly for dual and quad core processor technology. One of our systems has 20 processors and ImageMagick is blazingly fast even for huge images (100000x100000). There are hundreds of internal algorithms that work in this fashion, farming out an image into chunks of scanlines for processing in parallel.

Otherwise you'll need to build ImageMagick with the --disable-openmp option to disable OpenMP.

[langer]

No, using "#pragma omp single" on the ImageMagick calls doesn't help. It just moves the location of the seg fault from the OpenMP calls in ImageMagick to the OpenMP code generated in the calling routine by the pragma.

I don't doubt that ImageMagick is much faster with OpenMP. But it's also much slower when it seg faults...

-- Steve

[magick]

But it's also much slower when it seg faults...

Its libgomp that is seg faulting, not the ImageMagick API.

Ok, here's our recommendation to the user community:

Note that the OpenMP committee has not defined the behavior of mixing OpenMP with other threading models such as Posix threads. If you want to use Posix threads from a program module that calls one of the ImageMagick application programming interfaces (e.g. MagickCore, MagickWand, Magick++, etc.), you may need to disable OpenMP support when you build ImageMagick. Under Linux / Mac OS X/ Solaris, you can disable OpenMP with the --disable-openmp configure script option.

[langer]

Its libgomp that is seg faulting, not the ImageMagick API.

Yes, I know that. I wasn't implying that ImageMagick was seg faulting. But ImageMagick is using OpenMP in a way that causes OpenMP to seg fault if ImageMagick is used within a posix thread. This severely limits the ways in which ImageMagick can be used.

Would it be possible for ImageMagick's build scripts to generate two coexisting versions of the libraries, one with OpenMP and one without? Then programs linking to them could choose the appropriate one, and you wouldn't have to change anything in the source code.

-- Steve

[magick]

Can you post a URL to a small threaded Magick++ application that faults for you. We have two threaded regression tests that uses pthreads and they complete without complaint under Fedora.

Would it be possible for ImageMagick's build scripts to generate two coexisting versions of the libraries, one with OpenMP and one without?

You can do that now. Just change your installation prefix on the configure script command line and build. We know of users that, for example, create two versions of ImageMagick. One HDRI-enabled, and one that is not.

[langer]

It's short, I'll just post it here. All it does is try to open an image file on a subthread.

Code: Select all

// -*- C++ -*-
// compile with 
//   g++ -o magicktest magicktest.C `Magick++-config --cppflags --ldflags --libs`
// 
#include <Magick++.h>
#include <pthread.h>
#include <iostream>


void *subthread(void *threadid) {
  try {
    Magick::Image image = Magick::Image("dummyfilename");
  }
  catch (Magick::ErrorBlob) {
    ;
  }
  std::cerr << "OK" << std::endl;
  return 0;
}

int main(int, char**) {
 pthread_t subth;
 void *subth_status;
 int res =  pthread_create(&subth,NULL,&subthread,NULL); 
 pthread_join(subth,&subth_status); 
}

This crashes on OS X 10.6.4, gcc 4.2.1. It works on Debian Linux, but they have a different pthread implementation. It works on OS X if ImageMagick is built with --disable-openmp.

Would it be possible for ImageMagick's build scripts to generate two coexisting versions of the libraries, one with OpenMP and one without?

You can do that now. Just change your installation prefix on the configure script command line and build. We know of users that, for example, create two versions of ImageMagick.

I'll see if the macports maintainers are open to that idea.

-- Steve

[magick]

Your programs works fine under Fedora Core 13. We ran it over 1000 times and it returned OK in each case.

[langer]

Yes, I would expect it to, because it works with Linux pthreads. But the whole point is ImageMagick is relying on OpenMP behavior that is not defined in the OpenMP or pthreads standards, does not work with OS X pthreads, and is not guaranteed to work with future versions of Linux pthreads.

By not supplying a run-time way of disabling OpenMP (ie, making no calls at all to any OpenMP routines or pragmas) you are forcing anyone who wants to use ImageMagick in a threaded program either to install two versions of ImageMagick or to do without parallel processing in the ImageMagick command line tools. A run-time function call to disable OpenMP calls in ImageMagick would allow programmers to work around the problem without requiring system administrators and (possibly clueless) users to make system-wide changes.

-- Steve

[magick]

We know of no run-time method to disable OpenMP. If you discover a method, let us know.

[langer]

We know of no run-time method to disable OpenMP. If you discover a method, let us know.

Code: Select all

if(using_openmp)
  process_image_with_openmp();
else
  process_image_without_openmp();

[magick]

In the mean-time, does your threaded program work if your set the environment variable, OMP_NUM_THREADS to 1?