Page 1 of 1

Windows: Postscript Delegate: Concurrent PS/EPS read operations fail

Posted: 2018-08-02T08:38:49-07:00
by horstr
ReadImage fails on Windows with ImageMagick 6.9.10 and GhostScript 9.06 if multiple threads execute these operation using dedicated ImageInfo instances.

The output (1 thread):

Code: Select all

C:\>main 1 circle.eps test.jpg
Start READ
Start WRITE
END
The output (2 threads):

Code: Select all

C:\>main 2 circle.eps test.jpg
Start READ
Start READ
Failed READ: PostscriptDelegateFailed `The system cannot find the file specified
.
' @ error/ps.c/ReadPSImage/888
Start WRITE
END
The code:

Code: Select all

#include <cstring>
#include <thread>
#include <list>
#include <string>
#include <iostream>

#include "magick/api.h"

class MagickException
{
  ExceptionInfo exception_info;

public:
  MagickException()
  {
    GetExceptionInfo(&exception_info);
  }
  ~MagickException()
  {
    DestroyExceptionInfo(&exception_info);
  }

  operator ExceptionInfo * ()
  {
    return &exception_info;
  }
  ExceptionInfo * operator->()
  {
    return &exception_info;
  }
};

class MagickImage
{
  ImageInfo * imageinfo;
  Image * image;

public:
  MagickImage()
  {
    imageinfo = CloneImageInfo(nullptr);
    image = nullptr;
  }
  ~MagickImage()
  {
    DestroyImageList(image);
    DestroyImageInfo(imageinfo);
  }

  std::string Read(char const * file)
  {
    MagickException magick_exception;
    std::strcpy(imageinfo->filename, file);
    image = ReadImage(imageinfo, magick_exception);
    if (magick_exception->severity >= ErrorException) {
      return magick_exception->reason;
    }
    return std::string();
  }
  std::string Write(char const * file)
  {
    std::strcpy(imageinfo->filename, file);
    std::strcpy(image->filename, file);
    WriteImage(imageinfo, image);
    if (image->exception.severity >= ErrorException) {
      return image->exception.reason;
    }
    return std::string();
  }
};


static void Execute(char const * inputimage, char const * outputimage, std::string &errorstr)
{
  MagickImage magick_image;
  errorstr = magick_image.Read(inputimage);
  if (!errorstr.empty()) {
    return;
  }
  errorstr = magick_image.Write(outputimage);
  if (!errorstr.empty()) {
    return;
  }
}

class MagickContext
{
public:
  MagickContext()
  {
    MagickCoreGenesis(nullptr, MagickFalse);
  }
  ~MagickContext()
  {
    MagickCoreTerminus();
  }
};

int main(int argv, char **argc)
{
  MagickContext magick_context;

  unsigned threadcount = atoi(argc[1]);
  char const * inputimage = argc[2];
  char const * outputimage = argc[3];

  std::list<std::thread> threads;
  std::list<std::string> errorstrs;
  for (unsigned i = 0; i < threadcount; ++i) {
    errorstrs.emplace_back();
    threads.emplace_back(Execute, inputimage, (std::to_string(i)+outputimage).c_str(), std::ref(errorstrs.back()));
  }
  {
    decltype(threads)::iterator threadit = threads.begin();
    decltype(errorstrs)::iterator errorstrit = errorstrs.begin();
    for (unsigned i = 0; threadit != threads.end() && errorstrit != errorstrs.end(); ++i, ++threadit, ++errorstrit) {
      threadit->join();
      if (!errorstrit->empty()) {
        std::cerr << i << " " << *errorstrit << std::endl;
      }
    }
  }
  std::cout << "END" << std::endl;
  return 0;
}
This issues is independent from the actual PS/EPS input and does not occur on Linux. Please look into it.

Cheers,

Horst.

Re: Windows: Postscript Delegate: Concurrent PS/EPS read operation fail

Posted: 2018-08-02T11:09:35-07:00
by fmw42
Ghostscript 9.06 is ancient. Try upgrading to the current version which I think is 9.23. I have heard that 9.22 may also have problems. So be sure to get at least 9.23.

Re: Windows: Postscript Delegate: Concurrent PS/EPS read operation fail

Posted: 2018-08-02T11:22:26-07:00
by horstr
Thanks. I retried with 9.23, the issue still exists.

Re: Windows: Postscript Delegate: Concurrent PS/EPS read operation fail

Posted: 2018-08-02T11:31:39-07:00
by fmw42
I would suggest you post your EPS file so others can test with it.

Also you do not say what version of libjpeg you are using? You can find it from

convert -list format

The current version would say 90.

I am not sure if Ghostscript is multi-threaded. I do not know that it should fail, but expect it would just use one thread.

What happens if you disable mult-threading either by disabling OpenMP or setting the thread limit to 1.

MAGICK_THREAD_LIMIT=1 convert ....

or

convert -limit thread 1 ....

Re: Windows: Postscript Delegate: Concurrent PS/EPS read operation fail

Posted: 2018-08-02T12:10:29-07:00
by horstr
Thanks for the reply. ReadImage fails so my assumption is that the output format does not matter for the issue.
The EPS I used: https://people.sc.fsu.edu/~jburkardt/da ... circle.eps
Actually, I tried multiple files and always had the problem.

I found the following ImageMagick changelog entry:

2015-11-24 6.9.2-7 Dirk Lemstra <dirk@lem.....org>
* The DICOM reader now handles the rescale intercept and slope.
* Added 'bmp3:alpha' option for including the alpha channel when writing
an image in the BMP3 format. (reference
viewtopic.php?f=1&t=28733).
* Only one thread permitted when delegating to the Ghostscript library
under Windows.


I'm not sure if a lock was introduced with that change. Maybe the lock was removed again in a later version.

Re: Windows: Postscript Delegate: Concurrent PS/EPS read operation fail

Posted: 2018-08-02T13:06:06-07:00
by fmw42
I can tell you that on my INTEL Mac Mini OSX Sierra with 2 cores, I can convert your file fine using ImageMagick 6.9.10.8 Q16 with OpenMP enabled and libjpeg 90.

Code: Select all

convert circle.eps circle.jpg
I do not have a Windows system to test, so I will leave that to any Windows user who want to check or to the developers.

My resources show the possibility of using 4 threads

Code: Select all

convert -list resource
Resource limits:
  Width: 214.7MP
  Height: 214.7MP
  List length: 18.446744EP
  Area: 17.18GP
  Memory: 8GiB
  Map: 16GiB
  Disk: unlimited
  File: 192
  Thread: 4
  Throttle: 0
  Time: unlimited

Re: Windows: Postscript Delegate: Concurrent PS/EPS read operation fail

Posted: 2018-08-02T13:09:55-07:00
by horstr
Thanks. However, I don't think that this can be reproduced using convert alone because it does not execute multiple conversions concurrently - in a single process. My understanding is that the thread limit of convert is a limit for a single conversion. That's why I provided the testcase.

Re: Windows: Postscript Delegate: Concurrent PS/EPS read operation fail

Posted: 2018-08-02T13:10:32-07:00
by fmw42
What exact version of 6.9.10.x are you using? You might consider upgrading or reinstalling if at the current version.

Re: Windows: Postscript Delegate: Concurrent PS/EPS read operation fail

Posted: 2018-08-02T13:11:37-07:00
by fmw42
OK. I will defer now to the IM developers as I am limited to the command line interface.

Re: Windows: Postscript Delegate: Concurrent PS/EPS read operations fail

Posted: 2019-03-18T01:11:31-07:00
by horstr
Any news on this issue? We worked around this by adding a lock in case of Windows. On Windows, ImageMagick loads the GhostScript DLL and uses GhostSCript in-process - the DLL must not be used concurrently. On Linux, ImageMagick uses GhostScript out-of-process, so the issue cannot occur.

Thanks!

Re: Windows: Postscript Delegate: Concurrent PS/EPS read operations fail

Posted: 2019-03-21T17:42:21-07:00
by magick
Looks like Ghostscript 9.06 is not thread safe. We're using ImageMagick 9.25 and 9.26 and your program completes as expected without complaint. We tried 1 to 30 threads.