Auto gravity to minimize background?

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Locked
jmspaggi
Posts: 29
Joined: 2020-02-09T15:15:17-07:00
Authentication code: 1152

Auto gravity to minimize background?

Post by jmspaggi »

Hi,

Is there an option to extent a picture and set the gravity to something like "auto" to minimize the background?

Consider this picture:
Image

if I run convert file.jpg -background transparent -gravity center -extent 100x100 output.jpg I get this picture:
Image

What I want is to have a 100x100 extract of the source picture that minimize the amount of background (transparency)

This will be a better option for me:
Image

I can brute-force that and from the 500x500 picture extract all the 100x100 pictures and then get the one with the less transparency, but I'm hoping for something smarter than me :-/

Thanks,

JMS

User avatar
fmw42
Posts: 26383
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Auto gravity to minimize background?

Post by fmw42 »

No such option. You would have to script something.

jmspaggi
Posts: 29
Joined: 2020-02-09T15:15:17-07:00
Authentication code: 1152

Re: Auto gravity to minimize background?

Post by jmspaggi »

Thanks. That was my feeling too. I'm not even sure scripting is the best way. Because I will have to generate 160k extract for each picture. That's a lot of files. I might just build a small tool to do just that, to avoid going back to disk.

snibgo
Posts: 13034
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Auto gravity to minimize background?

Post by snibgo »

As Fred says. However, a reasonable first guess is that the 100x100 crop should be centred at the point that is furthest from any border.

Code: Select all

magick 2020-02-22-102908-jpg-part-44.png -strip -alpha extract -morphology Distance Euclidean:4 -define identify:locate=maximum info:

Channel maximum locations:
  Red: 6635.46 (0.101251) 127,238
  Green: 6635.46 (0.101251) 127,238
  Blue: 6635.46 (0.101251) 127,238
So the crop would be centred at (127,238). We subtract 100/2 to get the top-left corner::

Code: Select all

magick 2020-02-22-102908-jpg-part-44.png -strip -crop 100x100+77+188 +repage out.png
snibgo's IM pages: im.snibgo.com

jmspaggi
Posts: 29
Joined: 2020-02-09T15:15:17-07:00
Authentication code: 1152

Re: Auto gravity to minimize background?

Post by jmspaggi »

Wow! That looks that a very good solution so far! I will apply that on all the pictures and see, but at least on that one the result is very good! Thanks!

jmspaggi
Posts: 29
Joined: 2020-02-09T15:15:17-07:00
Authentication code: 1152

Re: Auto gravity to minimize background?

Post by jmspaggi »

Any idea why I get the output at Gray instead of the 3 channels like you?

Channel maximum locations:
Gray: 6627 (0.101122) 127,238

jmspaggi
Posts: 29
Joined: 2020-02-09T15:15:17-07:00
Authentication code: 1152

Re: Auto gravity to minimize background?

Post by jmspaggi »

Overall result is pretty good, but for some pieces I have more white around and less background and he doesn't target correctly.

Image
Image

One solution might be to remove all white that is > 220,220,220 and that is touching a transparent pixel, and make all transparent pixels 100% transparent? Do you think this might help?

snibgo
Posts: 13034
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Auto gravity to minimize background?

Post by snibgo »

jmspaggi wrote: Any idea why I get the output at Gray instead of the 3 channels like you?
My input was colour, so it did the morphology on each channel. "-colorspace Gray" after extracting alpha would do the work on just one channel, which would be faster.
jmspaggi wrote: ...but for some pieces I have more white around and less background and he doesn't target correctly.
My method relies on the background being entirely transparent.
snibgo's IM pages: im.snibgo.com

jmspaggi
Posts: 29
Joined: 2020-02-09T15:15:17-07:00
Authentication code: 1152

Re: Auto gravity to minimize background?

Post by jmspaggi »

Ok. I think I have been able to apply some nice cleanup to get more background.

Here is a cleanup piece:
Image

Here is the new result of your command in a cleaned-up piece:
Image

Even manually I can hardly select a better 100x100 extract. So I think it's perfect and I will go with that.

Thanks again!

snibgo
Posts: 13034
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Auto gravity to minimize background?

Post by snibgo »

What is the ideal result? Perhaps: the 100x100 square that contains the fewest background pixels. My method doesn't guarantee that. We could start from that result, and try offsetting the square one pixel north, northeast, east and so on. If any of those are better, then repeat from that position. That would find a local best, but doesn't guarantee to find the global best.

An exhaustive search, of all possible 100x100 squares, would guarantee to find the global best. This is not feasible. Ummm...

Ah, but it is feasible! And quite easy and fast (about 4 seconds). Make an image that has white pixels for the jigsaw piece and black for the background ("-alpha extract"). Then the mean of a 100x100 square is a measure of what proportion of the square is the jigsaw, not the background. "-statistic Mean 100x100" calculates all these means. "-define identify:locate=maximum" finds the largest.

Code: Select all

magick ^
  2020-02-22-102908-jpg-part-44.png ^
  -strip -alpha extract ^
  -colorspace Gray ^
  +write x.png ^
  -statistic mean 100x100 ^
  +write s.png ^
  -define identify:locate=maximum ^
  info:
We write x.png and s.png just for debugging, and so you can see intermediate results. Remove for better speed.

The result is:

Code: Select all

  Gray: 65473 (0.999054) 101,249 102,249 103,249
Three pixels are equally good. Pick one, subtract 50 as before to crop:

Code: Select all

magick ^
  2020-02-22-102908-jpg-part-44.png ^
  -strip ^
  -crop 100x100+52+199 +repage ^
  out.png
snibgo's IM pages: im.snibgo.com

jmspaggi
Posts: 29
Joined: 2020-02-09T15:15:17-07:00
Authentication code: 1152

Re: Auto gravity to minimize background?

Post by jmspaggi »

Oh! That super nice and indeed works pretty well!

I applied it on 1000 pictures and I found one interesting situation that I'm wondering if there is an optimization for.

Here is the source picture:
Image

Here is what the script will extract:
Image

But as you can see, it takes a bit of the white, because it's not transparent. It's because I take the first coordinates returned by the script. Is there a way to say that we also want the darker result? Or not really because it works only on the BW representation? Else another option is to take all the coordinates and take the one closer to the center of all the results, but not 100% sure...

It's already way better than what I had before. Thanks a lot!

snibgo
Posts: 13034
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Auto gravity to minimize background?

Post by snibgo »

The problem is that part of the background hasn't become transparent. If you open clean-2020-02-22-104537-jpg-part-03.png in Gimp or other image editor, you will see some of the background is fully opaque.

The best cure is to make the background fully transparent. For the purpose of finding the best 100x100 square, it won't matter much if you also make transparent some pixels around the edge of the jigsaw piece.
snibgo's IM pages: im.snibgo.com

jmspaggi
Posts: 29
Joined: 2020-02-09T15:15:17-07:00
Authentication code: 1152

Re: Auto gravity to minimize background?

Post by jmspaggi »

I think I need to re-scan the pictures. The last scanner I used is doing an ugly shadow that causes many issues to detect and remove the white and replace it with transparent. I will re-scan them this week-end and give all of this another try. I will come back to comment on this. This background removal (WIthout removing the content of the pieces) have been a real challenge :(

jmspaggi
Posts: 29
Joined: 2020-02-09T15:15:17-07:00
Authentication code: 1152

Re: Auto gravity to minimize background?

Post by jmspaggi »

Hi,

providing a feedback here.

I tried many things to remove this ugly background.

I realized quickly that scanner with the cover closed was not a solution. Indeed, even if not all brands are giving the same result, there is still a small shadow arround the piece, that is not 100% white and very hard to remove. Increasing fuzzy make the light pieces to be sometime split in smaller pieces. So I did many other tests

First, I tried with the cover opened, at night, but turning all lights off This gave a pretty nice result. Background is black and scripts have to be adjusted, but as you can see below, the border is acceptable. It's not perfectly straight, but already way better. (You might have to zoom in to see the difference)

Image

Happy from the result I tried again with the cover opened, but at day. This didn't work. The light is too bright and white pieces are swallowed with the background. To prevent that I waited for the room to be a bit less bright. Scanning every hour to see how day light impacts the scan. After the runset, sky is still very clear! But there is no more direct sun in the room, and I got the following result:
Image

Again, you might have to to zoom in, but the result is definitely way better than anything I was able to get before! Borders are nit and white pieces don't have any issue! Look at the botton right corner, it'S perfectly define! The only issue is that you have only abut 2h before it become dark and you start to see that on the scan. But still, I'm super happy with the result!

Now, regarding the script above in this page. I also found what I need to do to center the square.

Imagine it gives back (9,9), (9,10), (9,11), (10, 9), (10, 10), (10, 11), (11, 9), (11, 10), (11, 11). Instead of taking the first returned option (Here 9, 9) I need to find the biggest possible rectangle formed by the points an take its center. Here it's (10, 10) +/- 1/1. I will most probably have to build something extrenally to parse that, but I think the idea will work.

Thanks again for all the help with this!

JM

jmspaggi
Posts: 29
Joined: 2020-02-09T15:15:17-07:00
Authentication code: 1152

Re: Auto gravity to minimize background?

Post by jmspaggi »

Looking at what you provided before, I think this might do exactly what I want:

convert step2.png -statistic mean 150x150 +write s.png -define identify:locate=maximum info:

Where step2.png is the s.png picture from your script... It's like giving a 2nd pass on the same logic and maximizing the surface... I will try to run that on a bigger set. I tried just on one picture and it looks pretty nice so far! Happy!

Locked