identify and cr2 raw takes ages

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
pauledd
Posts: 11
Joined: 2017-06-09T00:46:07-07:00
Authentication code: 1151

identify and cr2 raw takes ages

Post by pauledd »

Hi
I just noticed that using identify on an Canon Raw (EOS 700, *.cr2) takes 50 seconds when called:

Code: Select all

time identify image.cr2 
image.cr2=>/tmp/magick-15167CDydMTSdLK39.png CR2 5208x3476 5208x3476+0+0 16-bit sRGB 63.12MiB 0.000u 0:00.000

real    0m50.499s
user    0m58.773s
sys     0m0.129s
Is this normal?

Htop shows me one (of eight) thread going to 100% while identify is running.
(i7-6700, gentoo)

identify -version
Version: ImageMagick 6.9.8-6 Q32 x86_64 2017-06-08 http://www.imagemagick.org
Copyright: © 1999-2017 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Features: Cipher DPC Modules
Delegates (built-in): bzlib cairo fftw freetype jng jp2 jpeg lcms ltdl png rsvg tiff x xml zlib
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: identify and cr2 raw takes ages

Post by snibgo »

All the time will be in whatever delegate is used: dcraw or ufraw. "-verbose" will tell you which is used.

For a while, IM included a multi-threaded version of dcraw, but it wasn't reliable.
snibgo's IM pages: im.snibgo.com
Bonzo
Posts: 2971
Joined: 2006-05-20T08:08:19-07:00
Location: Cambridge, England

Re: identify and cr2 raw takes ages

Post by Bonzo »

On my Windows PC with a Imagemagick 7 version it take roughly 5 seconds on a 500D cr2 file.
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: identify and cr2 raw takes ages

Post by magick »

It took 15 seconds for a 5202x3469 CR2 image under Fedora with IMv7. It takes considerably less time if you edit delegates.xml and change the --out-type from PNG to PPM and --out-depth to 8. However, you do lose image information if you make this change, but identify works much faster-- the command completed in 3 seconds rather than 16.
pauledd
Posts: 11
Joined: 2017-06-09T00:46:07-07:00
Authentication code: 1151

Re: identify and cr2 raw takes ages

Post by pauledd »

Thank you very much for your anwers! I just found out there seems to be a difference in files acquired via libgphoto and those directly from the camera. My raws where read with kstars/ekos (astronomy) via USB from my camera and it uses libgphoto to do that.
My thought was gphoto reads the raw as it is on the camera and handles it out unchanged but that seems not true.

It seems gphoto or something else in the chain does something change with the raw that makes imagemagick read so long.
I just tested some raws directly from the sdcard and those take just 12 seconds to read.
pauledd
Posts: 11
Joined: 2017-06-09T00:46:07-07:00
Authentication code: 1151

Re: identify and cr2 raw takes ages

Post by pauledd »

snibgo wrote: 2017-06-09T02:03:06-07:00 All the time will be in whatever delegate is used: dcraw or ufraw. "-verbose" will tell you which is used.

For a while, IM included a multi-threaded version of dcraw, but it wasn't reliable.
I got

Code: Select all

Software: UFRaw
    Source: CanonEOS 700D
    unknown: UFRaw 0.22
pauledd
Posts: 11
Joined: 2017-06-09T00:46:07-07:00
Authentication code: 1151

Re: identify and cr2 raw takes ages

Post by pauledd »

magick wrote: 2017-06-09T07:27:17-07:00 It takes considerably less time if you edit delegates.xml and change the --out-type from PNG to PPM and --out-depth to 8. However, you do lose image information if you make this change, but identify works much faster-- the command completed in 3 seconds rather than 16.
Sorry I need exact values from 16bit files (astronomy raw's)
Bonzo
Posts: 2971
Joined: 2006-05-20T08:08:19-07:00
Location: Cambridge, England

Re: identify and cr2 raw takes ages

Post by Bonzo »

If you post a link to a problem image somebody may take a look but I would contact the person who maintains libgphoto and see what they say. Looking at the site there is a libgphoto2 version now; if you are not using that it may be worth giving it a go?
pauledd
Posts: 11
Joined: 2017-06-09T00:46:07-07:00
Authentication code: 1151

Re: identify and cr2 raw takes ages

Post by pauledd »

Sure, here is an image that came directly from kstars.
https://drive.google.com/file/d/0B-Xegf ... sp=sharing
And yes, sorry, when I was writing about libgphoto I actually meant libgphoto2.
I will first ask the kstars people since they implemented the library and maybe there is
somewhere a piece of code that changes the raw.
User avatar
glennrp
Posts: 1147
Joined: 2006-04-01T08:16:32-07:00
Location: Maryland 39.26.30N 76.16.01W

Re: identify and cr2 raw takes ages

Post by glennrp »

Have you tried using a Q16 build instead of Q32?
pauledd
Posts: 11
Joined: 2017-06-09T00:46:07-07:00
Authentication code: 1151

Re: identify and cr2 raw takes ages

Post by pauledd »

hmm, I am on gentoo-linux and it seems there is no q16 build-flag, they only have q8,q32,q64 in their package manager.

I will try to compile manualy
pauledd
Posts: 11
Joined: 2017-06-09T00:46:07-07:00
Authentication code: 1151

Re: identify and cr2 raw takes ages

Post by pauledd »

ok, q16 is default in gentoo ... q16 makes no difference, still 52 seconds to identify the file. I am quite sure its a kstars/libgphoto2 issue.
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: identify and cr2 raw takes ages

Post by snibgo »

I don't know how IM works with ufraw. With dcraw, IM tells dcraw to convert the raw file, and IM gets any required information from that. This takes some time, even if IM only wants to know the dimensions of the image.

I assume IM and ufraw works in the same way.

So if your workflow involves any other processing of the image, it is more efficient to read the raw file just once, converting it to a regular format such as tiff or png, then work with that.

On the other hand, if you want the dimensions and nothing else, exiftool is a more obvious solution as it doesn't read or convert the image.
snibgo's IM pages: im.snibgo.com
Post Reply