Summary: improved Lanczos enlargement with sigmoidization

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Summary: improved Lanczos enlargement with sigmoidization

Post by NicolasRobidoux »

About a month ago, I discovered a (new?) technique for getting the low pass benefit of negative lobe filters without paying the full haloing "toll" when enlarging (or, more generally, warping without doing much downsampling, for example rotating; this method is not for producing thumbnails).

The process of exploring this idea is mostly described in viewtopic.php?f=22&t=21415 and viewtopic.php?f=22&t=21435.

In this thread, I give a quick summary of how it applies, specifically, to enlarging sRGB images with the popular Lanczos Sinc-windowed Sinc 3-lobe tensor (orthogonal) filter.

Sigmoidization does not affect Lanczos resampling's highly valued property of being interpolatory (meaning that the computed value at an original pixel location does not change when not downsampling; Anthony Thyssen summarizes this by stating that "no-op" does not change the image).

-----

For best results, you probably need a bleeding edge ImageMagick 7 compiled in HDRI mode. I'll be moving the code to ImageMagick 6 "soon". I'll use ImageMagick 7 syntax ("magick" instead of "convert") as a warning that you may not get the same high quality with ImageMagick 6 (or even ImageMagick 7 releases older than a week or two). It's a bit messy to specify which configurations are fine and are not, because sigmoidal-contrast was made "worse" (for this purpose) in Febuary 2012 or so, and was fixed recently.

Very informally, the main idea of sigmoidization is that the most offensive halos involve gamut extremes. In order to "squash" halos triggered by extreme values, we want these values to be "far" from each other and from mid-tones, so that an over- or undershoot not "go as far" when it starts from an extreme value. (It's like changing the distances (metric) within the solar system so that the farther and middle planets are even farther from the Sun and the inner planets than they already are, and so that the outer planets are even farther from each other and the middle and inner planets. This way, "moons" are less likely to wander between the outer planets and the middle planets, and likewise between the inner planets and the middle ones. The "moons" are the ripples that lead to halos.)

The usual sigmoid transformation http://en.wikipedia.org/wiki/Sigmoid_function takes extreme values and pushes them toward each other. The inverse of the sigmoid transformation does the opposite: It pushes the extreme values away from each other, and also away from the mid-tones.
So, we can use the inverse sigmoid transformation to convert pixel values to a "sigmoidal colourspace" in which extreme values are "stretched away from each other and the mid-tones", enlarging in this "colourspace", which has the effect that over- and undershoots triggered by extreme values will not reach as different colours as they would otherwise, and then converting back to sRGB. In order to preserve black/white symmetry, the sigmoidal colourspace is created from the linear light colourspace closest to sRGB, namely linear RGB with sRGB primaries, called RGB in ImageMagick, with a centered midpoint.

Code:

Code: Select all

magick input.png -colorspace RGB +sigmoidal-contrast 9.885,50% -filter Lanczos -resize 400% -sigmoidal-contrast 9.885,50% -colorspace sRGB output.png
Other values of contrast, both higher and lower than 9.885, work well.
Higher values work really well with subcritical ("slightly blurry") images, that is, images that stay away from the Nyquist limit. Sigmoidization is not particularly fond of low quality JPEG.
If you get "colours separation", back down the contrast. What's happening is that mid-tone values are "sharpened" more than extreme ones, consequently different components of the hue are not resampled the same way if some are extreme within RGB and some are not. The differential sharpening leads to something that looks a bit like "bleeding ink" in cheap colour printing.

9.885 is pretty much the lowest contrast which ensures that enlarging an image which is pure black on the left and pure white on the right enlarges to something which is within the somewhat commonly used 2.3 Delta E JND (Just Noticeable Difference) in L*a*b* of being the same. (2.3 L*a*b* JND deviation from pure black corresponds to about [8,8,8] in 8-bit sRGB.)

In my next post, I'll show a few results, based on http://www.wisdom.weizmann.ac.il/~visio ... ageSR.html for no better reason than several people have pointed this method out to me as a new thing worth considering. (Note that the methods tested there have nothing to do with my work. I'm just using them as examples of "state of the art" to measure my results against.)

P.S. Be warned that any calculation involving JND can only be approximate. JND is highly context dependent, and is only roughly approximated in the best case. On my brash, uncalibrated, Samsung monitor, I can easily tell the difference between [15,15,15] and [0,0,0], but [8,8,8] is black to me. This may have to do with LCDs squashing the low blacks as a result of compensating for back lighting bleeding through.

P.S. If instead of having a pure black/pure white within JND when enlarging a binary image with a vertical boundary, you want the same tolerance when enlarging a binary image which is a black square on a white background, then the least contrast that does the job is about 12.1245. I generally like the results with this contrast more, but in some cases it leads to noticeable "color separation". In other words, contrast=12.1245 is less "blindly safe". But even higher values are quite good at "sharpening" images without introducing significant haloing.
In all tests below, I stick to the relatively safe value 9.885. The higher value works reasonably well with text-like images.

P.S. The better the low pass filtering of the sampler, the better results are obtained with sigmoidization. For this reason, EWA resampling allows pushing the sigmoidization farther without colour separation artifacts than equivalently sharp tensor methods. This will be the topic of another summary thread.

P.S. Note that the test images themselves are not very high quality.
Last edited by NicolasRobidoux on 2012-08-28T10:30:06-07:00, edited 31 times in total.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Summary: improved Lanczos enlargement w/ sigmoidization

Post by NicolasRobidoux »

Image
It's a 4x enlargement. The telltale Lanczos halos are not as prominent as usual.
For a comparison with other state-of-the-art schemes, compare with the first group of results at http://www.wisdom.weizmann.ac.il/~visio ... .html#comp.
Last edited by NicolasRobidoux on 2012-08-26T04:34:26-07:00, edited 7 times in total.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Summary: improved Lanczos enlargement w/ sigmoidization

Post by NicolasRobidoux »

Image
It's a 4x enlargement. Compare with the second group of results at http://www.wisdom.weizmann.ac.il/~visio ... .html#comp.
Last edited by NicolasRobidoux on 2012-08-15T13:33:59-07:00, edited 1 time in total.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Summary: improved Lanczos enlargement w/ sigmoidization

Post by NicolasRobidoux »

Image
It's a 4x enlargement. Compare with the third group of results at http://www.wisdom.weizmann.ac.il/~visio ... .html#comp.
Last edited by NicolasRobidoux on 2012-08-15T13:34:40-07:00, edited 1 time in total.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Summary: improved Lanczos enlargement w/ sigmoidization

Post by NicolasRobidoux »

Image
It's a 4x enlargement. Compare with the fourth group of results at http://www.wisdom.weizmann.ac.il/~visio ... .html#comp.
An additional comparison: Henry HO was kind enough to redo this enlargement with the excellent NNEDI3 of Avisynth: viewtopic.php?f=22&t=21435&start=15#p88094.
Last edited by NicolasRobidoux on 2012-08-15T13:35:03-07:00, edited 2 times in total.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Summary: improved Lanczos enlargement w/ sigmoidization

Post by NicolasRobidoux »

Adding more lobes (-define filter:lobes=4, for example) decreases the "jaggies" (staircasing artifact) and sharpens things a nudge at the expense of more haloing (the "bounceback" (second) halo, in particular). But these are not as pronounced as they could be thanks to sigmoidization.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Summary: improved Lanczos enlargement with sigmoidizatio

Post by NicolasRobidoux »

Sigmoidization, combined with other tricks I have up my sleeve, would produce a really fast, high quality, GPU zooming or rotation or warping scheme. If you are with a company or organization interested in implementing this, note that I make my living as a consultant. I'm not moving my family (at this point), but I could spend some time away from home to do this "face to face". For a scheme with less artifacts (but not quite as sharp), have a look at viewtopic.php?f=1&t=21695. I also have a scheme guaranteed not to introduce any haloing (but it's more aliased), implemented in the VIPS and GEGL libraries, which I could also work at implementing on the GPU (or making full use of vector units).
Last edited by NicolasRobidoux on 2012-08-20T05:04:08-07:00, edited 2 times in total.
pipitas
Posts: 168
Joined: 2012-07-15T14:06:46-07:00
Authentication code: 15

Re: Summary: improved Lanczos enlargement with sigmoidizatio

Post by pipitas »

Amazingly impressive results, Nicolas!

Update:

Attention: to make a fair comparison, on the linked page (@ http://www.wisdom.weizmann.ac.il/~visio ... ageSR.html ) for each image always click on the blue "Our SR Result"-button before making a comparison (because all the large image frames are pre-loaded with the lower quality 'nearest neighbour' interpolation).
Last edited by pipitas on 2012-08-19T13:43:50-07:00, edited 1 time in total.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Summary: improved Lanczos enlargement with sigmoidizatio

Post by NicolasRobidoux »

I am not sure yet, but it appears that tensor (orthogonal) Jinc-windowed Sinc is a very slightly better scheme, at least when sigmoidizing. (This is one of the two schemes I like to call "Ginseng".)
It's really hard to do much better than Sinc windowing for Sinc and Jinc windowing for Jinc, so maybe I'm just kidding myself.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Summary: improved Lanczos enlargement with sigmoidizatio

Post by NicolasRobidoux »

@pipitas: Still as impressed when you compare to the actual SR scheme? Or the Fattal scheme? (It's not too hard to win over nearest neighbour.)
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Summary: improved Lanczos enlargement with sigmoidizatio

Post by NicolasRobidoux »

More sample enlargements, with a closely related method, are found here: viewtopic.php?f=1&t=21703.
Post Reply