Smart crop function in IM?

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
maqe
Posts: 17
Joined: 2017-06-18T02:21:18-07:00
Authentication code: 1151

Smart crop function in IM?

Post by maqe »

I use IM to generate a bunch of thumbnails of different sizes and formats. Does IM have a function for smart crop like https://imagga.com/solutions/cropping-api.html ? If not, have any of you used any third party library for this?
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Smart crop function in IM?

Post by fmw42 »

There is no "smart" crop.

But ImageMagick has all kinds of crop options. You can crop to the largest size and pad. You can crop to the smallest size and trim off the extra. You can crop so that it does not respect aspect and thus get exactly the coordinates you want. see all the sections of
http://www.imagemagick.org/Usage/crop/#crop
and all the sections of
http://www.imagemagick.org/Usage/thumbnails/#creation

Also there is liquid rescale that has smart trimming of parts of the image. See http://www.imagemagick.org/script/comma ... id-rescale and http://www.fmwconcepts.com/misc_tests/l ... index.html
maqe
Posts: 17
Joined: 2017-06-18T02:21:18-07:00
Authentication code: 1151

Re: Smart crop function in IM?

Post by maqe »

I see. So I guess I need to look for a third party service do analyze the image so I know the coordinates for the best crop result.
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Smart crop function in IM?

Post by snibgo »

IM contains low-level objective operations for processing raster images. It doesn't contain higher-level subjective operations such as "crop to the interesting area".

IM tools can be used to build those high-level operations, once someone has defined "interesting".
snibgo's IM pages: im.snibgo.com
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Smart crop function in IM?

Post by snibgo »

For example, suppose we want to crop an image to 200 pixels wide. We want to find the most "interesting" sub-image that is 200 columns wide.

Image

We might say that a column of pixels that is different to its neighbour column is "interesting". When they are the same, it is not interesting. Hence, the "-compose Difference" reduced to a single channel and scaled to a single pixel high is a measure of interestingness.

Then, the mean of a rolling window 200 pixels wide will have one or more peaks that show the central position of the most interesting windows.

Windows BAT script:

Code: Select all

set SRC=barbara_test.jpg

%IM%convert ^
  %SRC% ^
  ( +clone ) ^
  -geometry +1+0 -compose Difference -composite ^
  -grayscale RMS -scale "x1^!" ^
  -statistic mean 200x1 -auto-level ^
  -fill Black +opaque White ^
  -format %%@ ^
  info:
The result is "2x1+558+0". So we want a crop 200 pixels wide, starting at x=559-100=458. [EDIT: I meant x, not y.]

Code: Select all

%IM%convert ^
  %SRC% ^
  -crop 200x+458+0 +repage ^
  -quality 40 barbara200.jpg
Image

This is fast and quite effective. Clearly, the definition of "interesting" can be more complex. It might include face-detection, which is very complex.
snibgo's IM pages: im.snibgo.com
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Smart crop function in IM?

Post by fmw42 »

Another way to do smart cropping is to find the subsection with the most edge content using canny edge detector. In unix syntax:

Image

Code: Select all

infile="white_cat1.jpg"
ww=250
hh=250
inname=`convert "$infile" -format "%t" info:`
convert "$infile" -canny 0x1+10%+30% tmp1.gif
coords=`compare -metric rmse -subimage-search -dissimilarity-threshold 1 tmp1.gif \( -size ${ww}x${hh} xc:white \) null: 2>&1 | cut -d\  -f4`
xoff=`echo "$coords" | cut -d, -f1`
yoff=`echo "$coords" | cut -d, -f2`
convert "$infile" -crop ${ww}x${hh}+${xoff}+${yoff} +repage ${inname}_crop.jpg
Image


Image

Code: Select all

infile="car_lady_dog_1.jpg"
ww=250
hh=300
inname=`convert "$infile" -format "%t" info:`
convert "$infile" -canny 0x1+10%+30% tmp1.gif
coords=`compare -metric rmse -subimage-search -dissimilarity-threshold 1 tmp1.gif \( -size ${ww}x${hh} xc:white \) null: 2>&1 | cut -d\  -f4`
xoff=`echo "$coords" | cut -d, -f1`
yoff=`echo "$coords" | cut -d, -f2`
convert "$infile" -crop ${ww}x${hh}+${xoff}+${yoff} +repage ${inname}_crop.jpg
Image

Speed-up can be had by doing the compare on a pyramid of resolutions -- two often is enough. Or by using my rmsecorr or normcrosscorr to do the subimage search.
Bonzo
Posts: 2971
Joined: 2006-05-20T08:08:19-07:00
Location: Cambridge, England

Re: Smart crop function in IM?

Post by Bonzo »

Out of interest I gave your code a try snibgo but get an error on the first piece of code:
convert: geometry does not contain image `' @ warning/attribute.c/GetImageBoundingBox/240.
Version 7.0.5-Q16 on a Windows 10 PC ( same error with magick and convert ).

I have a feeling I have seen a similar error before using magick if the operators are in the wrong order; but it should have worked with the convert as it is still using V6?
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Smart crop function in IM?

Post by fmw42 »

You get that error when the crop has zero size for one or both of Width and Height
Bonzo
Posts: 2971
Joined: 2006-05-20T08:08:19-07:00
Location: Cambridge, England

Re: Smart crop function in IM?

Post by Bonzo »

Thank you fmw42 as my crop dimensions from the command ( found using a php version of the code ) are 0x0+1+0

I get the same error using snibgo's image as well.

I also had a go at your codes but I am not sure how to implement the cut in Windows or php
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Smart crop function in IM?

Post by snibgo »

My command isn't robust, because "-trim" or "-format %@" can result in a 0x0 image, which IM then objects to.

A cure is to insert "-compose Over -bordercolor Black -border 1" before the "-format". Then parse out the x-offset, and subtract 1 for the crop. (This doesn't totally solve the problem: a totally black input image is totally uninteresting, and trims down to 0x0.)

Here is a more robust version (Windows BAT syntax):

Code: Select all

set SRC=\prose\pictures\barbara_test.png

rem REQ_W is required width of "interesting" region.
rem REQ_W should be less than actual width of image.

set REQ_W=200

for /F "usebackq tokens=3 delims=x+" %%X in (`%IM%convert ^
  %SRC% ^
  ^( +clone ^) ^
  -geometry +1+0 -compose Difference -composite ^
  -grayscale RMS -scale "x1^!" ^
  -statistic mean %REQ_W%x1 -auto-level ^
  -fill Black +opaque White ^
  -compose Over ^
  -bordercolor Black -border 1 ^
  -format %%@ ^
  info:`) do set /A CX=%%X-1-(%REQ_W%/2)

%IM%convert ^
  %SRC% ^
  -crop %REQ_W%x+%CX%+0 +repage ^
  -quality 40 out.jpg
I expect IM v7 can do this in a single command.
snibgo's IM pages: im.snibgo.com
Bonzo
Posts: 2971
Joined: 2006-05-20T08:08:19-07:00
Location: Cambridge, England

Re: Smart crop function in IM?

Post by Bonzo »

Thank you for updating the code snibgo; it works well on your example but for some reason fails on my test image. I get a crop of -crop 200x+-100+0 in my command display. I understand it will not work on every image but I was just interested in seeing how well it worked.

Image
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Smart crop function in IM?

Post by fmw42 »

I have written a bash unix shell script to do a minimally intelligent (or semi-smart) version of a smart crop. See http://www.fmwconcepts.com/imagemagick/ ... /index.php for examples.
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Smart crop function in IM?

Post by snibgo »

@Bonzo: The "-geometry +1+0 -compose Difference -composite" gives a spurious result at the left edge, so the cure is to then remove the left edge: "-crop x+1+0 +repage".

@Fred: Good stuff. Your page http://www.fmwconcepts.com/imagemagick/ ... /index.php has "Animation" in places I think you intended "Arguments:".


A method to find a 300x200 area that is "interesting" (ie contains the most detail), with my published Windows BAT scripts:

Code: Select all

set SRC=_MG_5357S.jpg

%IM%convert -size 300x200 xc:white w.png

call %PICTBAT%slopemag %SRC% s.png

call %PICTBAT%srchImg s.png w.png

%IM%convert %SRC% -crop %siCOMP_CROP% +repage -quality 40 insect300x200.jpg
Image

Similarly:

Image

Image

Image
snibgo's IM pages: im.snibgo.com
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Smart crop function in IM?

Post by snibgo »

Command and scripts above, from Fred and me, answer the question: "Where in this image is the NxM rectangle with the most detail?"

But how do we choose N and M?

My commands create a slope magnitude image, s.png, which is bright where this is detail, and dark where there is no detail. So each pixel has a lightness, which is a score. We can process this (or any similar image) to find the smallest rectangle that contains the T% top-scoring pixels.

s.png is noisy, so we blur it first.

Code: Select all

for /F "usebackq" %%L in (`%IM%convert ^
  s.png ^
  -blur 0x20 ^
  -equalize ^
  ^( +clone -threshold 99%% -format "t99=%%@\n" +write info: +delete ^) ^
  ^( +clone -threshold 97%% -format "t97=%%@\n" +write info: +delete ^) ^
  ^( +clone -threshold 95%% -format "t95=%%@\n" +write info: +delete ^) ^
  ^( +clone -threshold 90%% -format "t90=%%@\n" +write info: +delete ^) ^
  ^( +clone -threshold 85%% -format "t85=%%@\n" +write info: +delete ^) ^
  b.png`) do set %%L

%IM%convert ^
  %SRC% ^
  ( +clone -crop %t99% +write c99.png +delete ) ^
  ( +clone -crop %t97% +write c97.png +delete ) ^
  ( +clone -crop %t95% +write c95.png +delete ) ^
  ( +clone -crop %t90% +write c90.png +delete ) ^
  ( +clone -crop %t85% +write c85.png +delete ) ^
  NULL:

%IM%convert ^
  c99.png c97.png c95.png c90.png c85.png ^
  -bordercolor gray(50%%) -border 3 ^
  +append ^
  a.png
Image

Image
snibgo's IM pages: im.snibgo.com
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Smart crop function in IM?

Post by fmw42 »

snibgo wrote:But how do we choose N and M?

My commands create a slope magnitude image
In my script, I must admit, that I do not get good results for all sizes of N and M. So that is a great question.

Snibgo, what is a slope magnitude image? If you just pick an arbitrary N and M, do you always get good results on these images?
Post Reply