False positive for grayscale images using the "saturation test"

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
kazuo
Posts: 4
Joined: 2016-05-27T20:01:19-07:00
Authentication code: 1151

False positive for grayscale images using the "saturation test"

Post by kazuo » 2016-05-27T20:43:02-07:00

Hello,

I'm trying to test for grayscale images using the "saturation test" describe by fmw42 on this post:

viewtopic.php?f=1&t=19580#p81226

I'm getting a lot of false positives from png images upscaled with waifu2x[1]. This album have a sample pair of images http://imgur.com/a/aCvoZ

This is what I'm getting:

Code: Select all

$ convert 001.png 002.png -colorspace HSL -channel g -separate +channel -format "%M: %[fx:mean]\n" info:
001.png: 0
002.png: 0.773941
(ImageMagick 6.9.4)

The upscaled image is clearly (visually) grayscale, but why the saturation value is so high?

Regards,
Kazuo

[1]: https://github.com/nagadomi/waifu2x

User avatar
fmw42
Posts: 23905
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: False positive for grayscale images using the "saturation test"

Post by fmw42 » 2016-05-27T22:27:49-07:00

The second image has been converted to 24bit colorspace sRGB while the first image is 8bit colorspace Gray. But that in principle should not affect the saturation (other than perhaps some slight interpolation differences between channels when scaling)

OK, it is due to HSL being a double hexcone model. My error in using that colorspace in the link you referenced.

So change the command to use -colorspace HSI or HSB or HCL(p). Any of those seems to work fine to get Saturation, since L/I/B are all based upon a single hexcone model.

(Due to HSL being a double hexcone model, this colorspace does not work properly for measuring saturation for L>50% as it approaches the top peak. There white is going to be indetermined saturation and turns out is interpreted as high saturation. In its place, use HCL, HCLp, HSI or HSB above. These are all single hexcone type models and should work fine in the above equation.

Reference: https://en.wikipedia.org/wiki/HSL_and_HSV)

Code: Select all

convert XNiW0WO.png E8ChzKV.png -colorspace HSB -channel g -separate +channel -format "%M: %[fx:mean]\n" info:
XNiW0WO.png: 0
E8ChzKV.png: 0.0253954

Code: Select all

convert XNiW0WO.png E8ChzKV.png -colorspace HSI -channel g -separate +channel -format "%M: %[fx:mean]\n" info:
XNiW0WO.png: 0
E8ChzKV.png: 0.0201001

Code: Select all

convert XNiW0WO.png E8ChzKV.png -colorspace HCLp -channel g -separate +channel -format "%M: %[fx:mean]\n" info:
XNiW0WO.png: 0
E8ChzKV.png: 0.00491875

Code: Select all

convert XNiW0WO.png E8ChzKV.png -colorspace HCL -channel g -separate +channel -format "%M: %[fx:mean]\n" info:
XNiW0WO.png: 0
E8ChzKV.png: 0.00491875
Does your scaling tool allow you to set the colorspace or type for the output so as to preserve it being 8bit grayscale?

kazuo
Posts: 4
Joined: 2016-05-27T20:01:19-07:00
Authentication code: 1151

Re: False positive for grayscale images using the "saturation test"

Post by kazuo » 2016-05-27T23:09:49-07:00

fmw42 wrote:So change the command to use -colorspace HSI or HSB or HCL(p). Any of those seems to work fine to get Saturation, since L/I/B are all based upon a single hexcone model.
Thank you, I'm not getting any more false positives now.
fmw42 wrote:Does your scaling tool allow you to set the colorspace or type for the output so as to preserve it being 8bit grayscale?
I don't know, from the command line arguments the anwser is "no". I just started to use it, but I will ask the upstream.

snibgo
Posts: 10736
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: False positive for grayscale images using the "saturation test"

Post by snibgo » 2016-05-28T05:58:09-07:00

If you are doing no further processing within the command, it can be shortened to:

Code: Select all

convert p2.png -colorspace HCL -format %[fx:mean.g] info:
Note: mean.g finds the mean of the "green" channel. which here is Chroma.

I'll mention that an alternative method is to convert a clone to grayscale, and find the difference. If any pixels are non-black, the input had colour.

Code: Select all

convert p2.png ( +clone -colorspace gray ) -compose Difference -composite -format %[fx:maxima] info:
snibgo's IM pages: im.snibgo.com

User avatar
fmw42
Posts: 23905
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: False positive for grayscale images using the "saturation test"

Post by fmw42 » 2016-05-28T10:12:40-07:00

The question is really does the OP want pure gray or near gray. If pure gray, then looking at the %[type] and %[colorspace] should be enough. If near gray, then snibgo's second method, I believe, would fail if only 1 pixel was not grayscale.

snibgo
Posts: 10736
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: False positive for grayscale images using the "saturation test"

Post by snibgo » 2016-05-28T10:42:55-07:00

Yes, because I have use fx:maxima. For "near gray", fx:mean wold be more suitable.

Yet another option is to grayscale a clone, but then "-metric RMSE -format %[distortion] -compare info:".
snibgo's IM pages: im.snibgo.com

kazuo
Posts: 4
Joined: 2016-05-27T20:01:19-07:00
Authentication code: 1151

Re: False positive for grayscale images using the "saturation test"

Post by kazuo » 2016-05-29T00:18:42-07:00

Thanks for all the options. I'm trying to classify scanned manga/comics images on grayscale and color. The grayscale can be "linear scale" from yellowed/not balanced scans.

I still trying to choose a good test, this album have two images http://imgur.com/a/aNspJ, the first one I want to be classified as grayscale and the second one as color. I'm getting:

Code: Select all

HCL       : 0.00422075 0.0704143 (ratio 16.68)                                          
fx:maxima : 0.08985050 0.4373370 (ratio  4.87)                                          
fx:mean   : 0.00163378 0.0248197 (ratio 15.19)                                          
RMSE      : 0.00330001 0.0532821 (ratio 16.14)  
I as hoping for something more clear cut (like ratio of 100), but I think that I will try to manually tag a bunch of images and see what is the range of the above tests and go for some naive classifier.

Again thanks for the fast and detailed responses!

snibgo
Posts: 10736
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: False positive for grayscale images using the "saturation test"

Post by snibgo » 2016-05-29T09:13:59-07:00

A cartoon image may be mostly grayscale but with a small area of colour. The average saturation of the entire image may fall below your threshold for colour, but you might want this to be classified as "colour".

The various "maximum" methods will find a single pixel of colour, and this may be too sensitive.

So, another option is to break up your image into smaller pieces, say 50x50 pixels. Then find the average saturation of each piece. Then find the maximum of these averages.

Using your last two examples:

Code: Select all

convert 3pTxcy0.png -crop 50x50 -colorspace HCL -scale 1x1! -channel G -separate +channel -evaluate-sequence Max -format %[fx:mean] info:
0.00991836
convert IhafCdq.png -crop 50x50 -colorspace HCL -scale 1x1! -channel G -separate +channel -evaluate-sequence Max -format %[fx:mean] info:
0.457069
The ratio is about 46.
snibgo's IM pages: im.snibgo.com

User avatar
anthony
Posts: 8871
Joined: 2004-05-31T19:27:03-07:00
Authentication code: 8675308
Location: Brisbane, Australia

Re: False positive for grayscale images using the "saturation test"

Post by anthony » 2016-05-29T17:52:14-07:00

One of the things mentioned was about "linear scale".
kazuo wrote:Thanks for all the options. I'm trying to classify scanned manga/comics images on grayscale and color. The grayscale can be "linear scale" from yellowed/not balanced scans.
This is not pure grayscale, but a image in which the colors all fall in a linear sequence, say from a off-yellow color to a dark bluish color, as an example.

I did some work on this.
http://www.imagemagick.org/Usage/compare/#type_linear

The script is a perl script that takes the 9 colors from a 3x3 matrix average colors of an image by area, and uses 3D vector mathematics to determine how far the nine colors are from a 'best fit' line through all the colors. The script wasn't long but it was very vector orientated.

Best fit line..
  • find average of all 9 colors -- the line goes through this
  • Get the average direction of all colors from this point (in a average a positive direction).
  • Make vector from the average center a unit vector.
Compare all points along with that line
  • vector dot product -> distance along that line
  • vector cross product -> distance perpendicular to the line -> error
If the error is less than a threshold, colors in image follow a linear scale, or basically colored greyscale.

NOTE perl required the Math::VectorReal package from CPAN (ASIDE: I wrote it!)
The input is a "cmatrix" file with a list of 9 colors (see IM Examples link above)

Code: Select all

#!/usr/bin/perl
=head1 NAME

image_is_linear -- Determine is a image is a linear color line drawing

=head1 SYNOPSIS

  image_is_linear [options] threshold [metric_list...]
  Options:
     -v    Verbose results on STDERR, (give just one image)
     -t    Return the linear error, (how linear a image is) instead

     -l    only pass linear color image metrics (modified) (>threshold)
     -c    only pass non-linear image metrics unchanged  (<=threshold)

=head1 DESCRIPTION

Given an array of color metrics extracted from an image, determine if all
those metrics are linear to each other (within a given color threshold of the
optimal linear line).  A image with linear color metrics, is throuht to be a
line drawing, rather than color image.

The metrics of a linear image is replaced with a new metric giving the
distances along that line, from its closest approach to the origin (black).

If the metric file list contains a list with the special flag '---' it is just
passed on, for later use by the comparision function.

=head1 AUTHOR

  Anthony Thyssen    16 June 2007           A.Thyssen_AT_griffith.edu.au

=cut
# -----------------------------------------------------------------------
use strict;
use File::Basename;
use Math::VectorReal qw(:all);

select((select(STDOUT), $| = 1)[0]);
select((select(STDERR), $| = 1)[0]);
sub Usage {
  use Pod::Usage;
  pod2usage("@_");
  exit 10;
}

my( $verbose, $linear_error, $linear_metrics, $color_metrics);

ARGUMENT:  # Multi-switch option handling
while( @ARGV && $ARGV[0] =~ s/^-(?=.)// ) {
  $_ = shift; {
    m/^$/  && do { next };     # next argument
    m/^-$/ && do { last };     # End of options
    m/^\?/ && do { Usage };    # Usage Help

    s/^v// && do { $verbose++;         redo }; # more verbose results
    s/^t// && do { $linear_error++;    redo }; # return the linear 'error'
    s/^l// && do { $linear_metrics++;  redo }; # pass only linear color metrics
    s/^c// && do { $color_metrics++;   redo }; # pass non-linear color metrics

    Usage( "Unknown Option \"-$_\"\n" );
  } continue { next ARGUMENT }; last ARGUMENT;
}

# color distance from the 'linear line'
# Note in RGB space,  Black to Primary =  65,536.00
#                     Black to White   = 113,511.68
# Smallest 'Color' image seen        = 800  (Approx - smaller definate linear)
# More Typical Grey/Color Cutoff     = 1000
# Largest Greyscale-like threshold   = 1336   (grey with red parts)
# Largest non-linear threshold seen  = 18645  (lots of extreme colors areas)

my $threshold = 200000;    # no threshold, convert metrics (default)
$threshold = 1500  if $linear_metrics;
$threshold =  800  if $color_metrics;
$threshold = 1000  if $linear_metrics && $color_metrics; # both!!!

$threshold = shift if $ARGV[0] =~ /^\d+$/;  # user selected

$linear_metrics = 1 unless $color_metrics; # pass linear metrics by default.

# ----------------------

my $num = 9;    # number of RGB vectors per image (color matrix only)
my $G = (X+Y+Z)->norm;  # normalized grey vector.

while (<>) {
  s/#.*//; s/\s+$//;
  my( $file, @metrics ) = split;
  next unless $file;
  if ( $file eq '---' ) {  # just pass on special comparision flags
    print "$file\n";
    next;
  }
  #print "$file @metrics\n";

  my @colors;  # the color vector metrics
  push (@colors, vector( shift @metrics, shift @metrics, shift @metrics) )
     while @metrics;
  #$Math::VectorReal::FORMAT = "\n\t%+8.1f %+8.1f %+8.1f";  # debug output
  #print "$file @colors\n";

  $num = @colors unless $num;
  if ( $num != @colors ) {
    die( "Invalid number of Metrics for \"$file\"\n$_\n" );
  }

  # get the best linear vector for the given colors
  my($c_origin, $c_vector) = linear_vector(@colors);

  if ( $verbose ) {
    print STDERR "$file\n";
    $Math::VectorReal::FORMAT = "[ %+8.6f %+8.6f %+8.6f ]";
    print  STDERR "    Color Vector $c_vector\n";
    $Math::VectorReal::FORMAT = "[ %+8.1f %+8.1f %+8.1f ]";
    print  STDERR "    Color Origin $c_origin\n";
    printf STDERR "    Origin %.1f from line\n", $c_origin->length;
    printf STDERR "    Color %.1f%% Greyscale\n", ($c_vector . $G) * 100;
  }

  # Find how non-linear each color metric is...
  my @linear;    # new metrics for a linear image.
  my $error = 0; # how different image is from linearity
  #my $max = 0;   # the largest 'non-linear' distance.
  #my $max_i = 0; # index of the least linear color
  for ( my $i=0; $i<$num; $i++ ) {
    my $v = $colors[$i] - $c_origin;
    my $d = $v.$c_vector;               # distance along line from origin
    my $x = ($v x $c_vector)->length;   # distance away from line

    push @linear, $d;   # new linear metric
    printf STDERR "\t$i : %.4f  %.4f\n", $d, $x  if $verbose;

    $error += $x*$x;   # squared error of difference
    #$max = $x, $max_i = $i  if $x > $max;  # find largest non-linear distance
  }
  $error = sqrt($error/$num);  # normalize the error back to RGB space

  if ( $linear_error ) {
    print int($error), " $file\n";  # just outputing 'linear' error values
  }
  elsif ( $error <= $threshold ) {
    print join(" ", $file, map {int($_+.5)} @linear), "\n"  if $linear_metrics;
  }
  else {
    $Math::VectorReal::FORMAT = "%d %d %d";
    print "$file @colors\n"  if $color_metrics;
  }
}


sub linear_vector {
  my $avg = O;  # The average color of the given colors...
                # The color vector will go though this point.
  for ( my $i=0; $i<@_; $i++ ) {
    $avg += $_[$i];
  }
  $avg /= scalar(@_);

  my $vec = O;  # the average direction of color differences

  for ( my $i=0; $i<@_; $i++ ) {
    my $v = $_[$i]-$avg;
    $v = -$v  if $v.$avg < 0;  # make all directions positive
    $vec += $v;
  }
  $vec = $vec->norm;

  # Find the closest position of this vector to the origin.
  my $org = $avg - ($avg.$vec)*$vec;

  return( $org, $vec );
}
Anthony Thyssen -- Webmaster for ImageMagick Example Pages
https://imagemagick.org/Usage/

kazuo
Posts: 4
Joined: 2016-05-27T20:01:19-07:00
Authentication code: 1151

Re: False positive for grayscale images using the "saturation test"

Post by kazuo » 2016-05-31T19:14:41-07:00

snibgo wrote:So, another option is to break up your image into smaller pieces, say 50x50 pixels. Then find the average saturation of each piece. Then find the maximum of these averages.
This worked very well, I'm getting a very clear distiction in my samples, thanks a lot for the detailed solution
anthony wrote:The script is a perl script that takes the 9 colors from a 3x3 matrix average colors of an image by area, and uses 3D vector mathematics to determine how far the nine colors are from a 'best fit' line through all the colors. The script wasn't long but it was very vector orientated.
Thanks anthony for this code, my images got successful shorted by the aboves method but will save this code for later use!

User avatar
anthony
Posts: 8871
Joined: 2004-05-31T19:27:03-07:00
Authentication code: 8675308
Location: Brisbane, Australia

Re: False positive for grayscale images using the "saturation test"

Post by anthony » 2016-05-31T20:36:19-07:00

The break up method is interesting. You are basically looking for the average saturation of a area that is appreciable to the 50x50 tile.
In other words you want a largish spot of color and not just a single pixel of color.

Hmmm, How about this. Take your image and resize it to 1:50 of the original size then look for the maximum saturation.
Unless you are using a 'Box' resize filter this will not simply cut the image into pieces but instead 'blur' areas of color. single pixels will not count for much, but a spot that is an appriciable size of 50x50 area will. So a faster mething to cutting the image into pieces would be

* convert image into a colorspace with a Saturation or Chroma channel
* resize it smaller by a 1:50 (2%) ratio (or whatever 'spot size' you are interested in
* get the maximum staturation/chroma value.

Added to Im Examples
http://www.imagemagick.org/Usage/compare/#type_color

Will appear when server updates.
Anthony Thyssen -- Webmaster for ImageMagick Example Pages
https://imagemagick.org/Usage/

snibgo
Posts: 10736
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: False positive for grayscale images using the "saturation test"

Post by snibgo » 2016-05-31T21:02:38-07:00

Good thinking, Anthony. I think "-scale" rather than "-resize" most closely reflects my method. Using "-scale":

Code: Select all

convert 3pTxcy0.png -colorspace HCL -scale 2% -format %[fx:maxima.g] info:
0.00990311

convert IhafCdq.png -colorspace HCL -scale 2% -format %[fx:maxima.g] info:
0.43859
Using "-resize":

Code: Select all

convert 3pTxcy0.png -colorspace HCL -resize 2% -format %[fx:maxima.g] info:
0.0101472

convert IhafCdq.png -colorspace HCL -resize 2% -format %[fx:maxima.g] info:
0.497261
There isn't much difference between these results and my earlier post. "-scale" is much quicker, of course.

Resize should be better than my "breaking up" method when a small patch of colour happens to sit on the boundary between two or even four pieces.
snibgo's IM pages: im.snibgo.com

Post Reply