Compare (subimage-search) rotated image without intermediary files

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
snibgo
Posts: 9379
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Compare (subimage-search) rotated image without intermediary files

Post by snibgo » 2017-05-16T07:58:53-07:00

You show "An example of a cropped subimage". But the background should be transparent, correct? And you want your script to identify the last two as matches?

Then forget my optimization step 3. It won't work.

Could both the matching icons be cropped?

Your comparison method needs to treat transparent pixels as "wildcard", so they match any colour in the other image. Either that, or mask both images to the non-transparent pixels of the other before comparing.
snibgo's IM pages: im.snibgo.com

cowboyzed
Posts: 17
Joined: 2017-05-14T22:54:30-07:00
Authentication code: 1151

Re: Compare (subimage-search) rotated image without intermediary files

Post by cowboyzed » 2017-05-16T08:30:41-07:00

snibgo wrote:
2017-05-16T07:58:53-07:00
Could both the matching icons be cropped?
The first match may be cropped, but not more than 5px. The second match could be cropped as I show in the image above.
snibgo wrote:
2017-05-16T07:58:53-07:00
Your comparison method needs to treat transparent pixels as "wildcard", so they match any colour in the other image. Either that, or mask both images to the non-transparent pixels of the other before comparing.
I don't understand what this will change in the command to use.

User avatar
fmw42
Posts: 22081
Joined: 2007-07-02T17:14:51-07:00
Location: Sunnyvale, California, USA

Re: Compare (subimage-search) rotated image without intermediary files

Post by fmw42 » 2017-05-16T09:53:02-07:00

In IM 7, you can do a compare and make it insensitive to transparent pixels, so it only matches opaque pixels. see viewtopic.php?f=4&t=31053

Can you show one example of your smaller reference image? I am still confused about what you are using to compare against the larger image.

cowboyzed
Posts: 17
Joined: 2017-05-14T22:54:30-07:00
Authentication code: 1151

Re: Compare (subimage-search) rotated image without intermediary files

Post by cowboyzed » 2017-05-16T18:41:42-07:00

The last example gives :
Image then Image

untill
Image
(this last is the matching icon)

User avatar
fmw42
Posts: 22081
Joined: 2007-07-02T17:14:51-07:00
Location: Sunnyvale, California, USA

Re: Compare (subimage-search) rotated image without intermediary files

Post by fmw42 » 2017-05-16T18:58:29-07:00

Please be more specific with your replies. What last example? Can you share your exact command line and the two input images? That way we can try to reproduce your results.

P.S. Note my request at viewtopic.php?f=2&t=31938

cowboyzed
Posts: 17
Joined: 2017-05-14T22:54:30-07:00
Authentication code: 1151

Re: Compare (subimage-search) rotated image without intermediary files

Post by cowboyzed » 2017-05-17T04:56:07-07:00

Please note the algorithm proposed by snigbo : https://www.imagemagick.org/discourse-s ... 28#p146096

With the big image :

Image

We have to detect this subimage which is repeated :

Image

PS: the unique input is the big image.

User avatar
fmw42
Posts: 22081
Joined: 2007-07-02T17:14:51-07:00
Location: Sunnyvale, California, USA

Re: Compare (subimage-search) rotated image without intermediary files

Post by fmw42 » 2017-05-17T10:27:11-07:00

You have potentially 3 issues that you need to fix with your images.

1) You have cropped the small image without +repage, so that it has a virtual canvas. This messes up the rotation.

2) You have texture under the transparent areas. This needs to made into a constant background color

3) You may want to pad your large image with more transparency to account for rotations of your small image at angles other than increments of 90 deg. For example at 45 deg rotation, the small image will increase in size by a factor of 1.414. Without padding the large image, it may prevent the rotated small image from being moved to a location that properly matches its corresponding object in the larger image. For the example below, I have only rotated the small image 180 deg, so its size does not change.


The following works fine for me on IM 6.9.8.5 Q16 Mac OSX

Code: Select all

convert \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background \
    -background none -virtual-pixel background -distort SRT 180 -trim +repage \) miff:- |\
compare -metric rmse -subimage-search -dissimilarity-threshold 1 - diff.png
4734.52 (0.0722441) @ 269,40

Image



Likewise using -rotate rather than -distort SRT

Code: Select all

convert \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background \
    -background none -rotate 180 -trim +repage \) miff:- |\
compare -metric rmse -subimage-search -dissimilarity-threshold 1 - diff.png
4808.31 (0.0733701) @ 269,40


Likewise using +distort SRT

Code: Select all

convert \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background \
    -background none -virtual-pixel background +distort SRT 180 -trim +repage \) miff:- |\
compare -metric rmse -subimage-search -dissimilarity-threshold 1 - diff.png
4893.59 (0.0746713) @ 268,39

-distort SRT will make the output the same size as the input, potentially cropping off some of it at angles other than 90 increments. -rotate and +distort SRT will expand the size of the output so that nothing is cropped off. We add a -trim to be sure we do not pad too much during the rotation, though it likely does not matter since your reference (small) image seems to be cropped to its bounding box anyway.

cowboyzed
Posts: 17
Joined: 2017-05-14T22:54:30-07:00
Authentication code: 1151

Re: Compare (subimage-search) rotated image without intermediary files

Post by cowboyzed » 2017-05-18T23:37:31-07:00

Thank you very much. Your command is useful : I can rotate the small image and use subimage-search without using intermediate files.
I want to get the score matching and I will use the following :

Code: Select all

convert \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background \
    -background none -virtual-pixel background -distort SRT 180 -trim +repage \) miff:- |\
compare -metric rmse -subimage-search -dissimilarity-threshold 1 - null: 2>&1 
Since I have to loop the command over 10 to 350 degrees, I'm wondering if there is a way to improve the solution you proposed here (https://www.imagemagick.org/discourse-s ... 28#p146091). I think it may increase performance, according to the algorithm proposed by snigbo.

User avatar
fmw42
Posts: 22081
Joined: 2007-07-02T17:14:51-07:00
Location: Sunnyvale, California, USA

Re: Compare (subimage-search) rotated image without intermediary files

Post by fmw42 » 2017-05-18T23:43:24-07:00

I do not know if it can be optimized further. But see viewtopic.php?f=2&t=31938#p146190. Once this is fixed, you can do it all in-line in the compare command without any pipes or saved files.

So this should be available in the next release.

cowboyzed
Posts: 17
Joined: 2017-05-14T22:54:30-07:00
Authentication code: 1151

Re: Compare (subimage-search) rotated image without intermediary files

Post by cowboyzed » 2017-05-19T00:03:50-07:00

Ok. I want to say there is now two differents algorithms :

1. Chop into 6 images, looping over these 6 images, using subimage-search (with rotation) over the big image.

2. Chop into 6 images, comparing each of the 6 images (with rotation) against the others and take the best score.

Now we got the solution of the method 1, but I want to know if the method 2 is better (probably with the appropriate command).

snibgo
Posts: 9379
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Compare (subimage-search) rotated image without intermediary files

Post by snibgo » 2017-05-19T00:35:21-07:00

Of those two algorithms, #1 will take far longer than #2, and most of the effort will be wasted.

When doing the rotations, you could go in 1 degree increments: 0, 1, 2, ... 359 degrees. But there is is nothing magical about that increment. Instead, you might try 5 or 10 degrees. Find the best match, then try angles on both sides of that match.

Or you can use a multiscale approach: when comparing images, first "-resize" them by 25%. This takes 1/16th of the time. Again, this gives an approximate solution, so narrows the search space for a non-resized comparison.
snibgo's IM pages: im.snibgo.com

User avatar
fmw42
Posts: 22081
Joined: 2007-07-02T17:14:51-07:00
Location: Sunnyvale, California, USA

Re: Compare (subimage-search) rotated image without intermediary files

Post by fmw42 » 2017-05-19T14:09:02-07:00

See viewtopic.php?f=2&t=31938&p=146215#p146215. My command now works in the latest beta so that all commands are inline in the one compare and not external processing is needed to rotate the smaller reference image.

Code: Select all

compare -metric rmse -subimage-search -dissimilarity-threshold 1 \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background -rotate 180 -trim +repage \) \
diff.png
4808.31 (0.0733701) @ 269,40

cowboyzed
Posts: 17
Joined: 2017-05-14T22:54:30-07:00
Authentication code: 1151

Re: Compare (subimage-search) rotated image without intermediary files

Post by cowboyzed » 2017-05-20T01:27:24-07:00

fmw42 and snigbo,

thank you very much. Now my script run faster (from 2400s to 500s)

:)

PS : I will install the beta version later. I hope the new command will increase performance.

User avatar
fmw42
Posts: 22081
Joined: 2007-07-02T17:14:51-07:00
Location: Sunnyvale, California, USA

Re: Compare (subimage-search) rotated image without intermediary files

Post by fmw42 » 2017-05-20T17:06:12-07:00

The following now works in the latest release of IM 6.9.8.6.

Code: Select all

compare -metric rmse -subimage-search -dissimilarity-threshold 1 \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background +distort SRT 180 -trim +repage \) \
diff.png


In IM 7.0.5.7, you need to turn alpha off at the end or the diff-1.png image will be transparent.

Code: Select all

im7 magick compare -metric rmse -subimage-search -dissimilarity-threshold 1 \
\( 370669patch.png +repage -background black -alpha background \) \
\( 224977object4.png +repage -background black -alpha background +distort SRT 180 -trim +repage \) \
-alpha off diff.png

cowboyzed
Posts: 17
Joined: 2017-05-14T22:54:30-07:00
Authentication code: 1151

Re: Compare (subimage-search) rotated image without intermediary files

Post by cowboyzed » 2017-05-21T13:05:31-07:00

I have ImageMagick 7.0.5-5 Q16 x86_64.

I want to know : is this last command get the job faster than previous "convert" ?

Post Reply