best downsampling method for DSLR photographs

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

best downsampling method for DSLR photographs

Post by NicolasRobidoux »

Quite an interesting thread on Digital Photography Review: http://forums.dpreview.com/forums/read. ... e=40282142

A few quick comments, off the top of my head (TODO list is a bit too tall these days to actually try stuff):

RE: blending the result of downsampling with (tensor, that is, -resize) Mitchell and the result of downsampling with (tensor) Lanczos http://forums.dpreview.com/forums/read. ... e=41369406:
This is a nice idea. Because the main weakness of tensor Lanczos (and actually all tensor methods with negative lobes) is that their cardinal basis function has a strong "checkerboard" component, I would not be surprised if you actually would get better results by blending tensor Lanczos with the result of downsampling with an EWA (-distort Resize) method, for example, Robidoux, or Lanczos or LanczosSharp (the current built-in, or the "improved versions" discussed in viewtopic.php?f=22&t=19636&start=120#p83719)
When I have time, I actually may figure out what blend of (tensor) Lanczos and some version of EWA Lanczos leads to minimax (= "best worst") over- and undershoot.

RE: Resampling through linear light (the "gamma corrected" toolchain) VS resampling in sRGB http://forums.dpreview.com/forums/read. ... e=41378321:
I would not assume that resampling through linear light is ALWAYS better. Some results from the upcoming Masters thesis of my student Adam Turcotte suggest that, at least when a significant portion of the toolchain involves sRGB (e.g. conversion from RAW; I really don't know exactly what the contributing factors are), resampling directly with sRGB values may give better results. (I hope I am not putting a giant foot in my mouth here, and that my questioning orthodoxy does not come from faulty coding or drawing general conclusions from narrowly defined tests.)
There are very strong heuristic arguments justifying the use of linear light whenever one does any kind of convolution/filtering (this includes resizing, perspective, warping, blurring, USM...). So, "linear light is better" is a very plausible rule of thumb.
Plausible does not make true.
I wager that Eric Brasseur, in his famous post http://www.4p8.com/eric.brasseur/gamma.html, could have fixed things so that the Dalai Lama disappears when downsized through linear light, but looks fine when processed through sRGB.
In other words, I'd like to hear from people who find that resizing with "plain sRGB" gives better looking result, as well as people who get better results through a gamma corrected toolchain, because I'd like to know myself if one of the two is always better, or if it kind of depends. I'd definitely put my money on linear light. But I would not bet with just any odds.

RE: linear light resampling in IM: The ImageMagick code given by Eric Brasseur is not a very accurate way of going through linear light. It is better to use -colorspace. However, be warned that as of IM version 6.7.5, the -colorspace command changed drastically in its use, and that there were bugs detected in the process of checking that the new set up was functioning properly: viewtopic.php?f=3&t=20751. So, be careful drawing conclusions from an IM version that either had colorspace bugs or used a meaning of the -colorspace which is the opposite of what you expect. IM's -gamma only approximates sRGB (this is unavoidable B/C there is more to sRGB than gamma correction). Although -colorspace is an accurate implementation of the equations that deal with converting from sRGB to linear light with sRGB primaries (and the same black and white points), it may not match the sRGB profile that was used to produce the image. In addition, conversion from RAW invariably involves chopping pieces of one machine and/or abstract colorspace to fit it into another. To some extent, you can't go back to a colorspace you've left, unless the conversion is really "controlled". In theory, you can go back with Perceptual intent; my impression is that in practice you generally can't. Which brings me to...

Another linear light warning: I wish I was more of a colour space expert, so I may be, once again, putting my foot in my mouth, but I believe that you need to pay attention to the rendering intent used to produce the sRGB image you are resizing, and there may be black point/white point issues in the conversion as well. There is actually a huge amount of variation between sRGB profiles. What you "think" gives you linear light may actually only give you an immensely crude approximation. Esp. if anything else than Relative rendering intent was used, and you don't import back with a high quality sRGB profile that contains an accurate inverse conversion for the sRGB profile and rendering intent that was used to encode the image you are resizing. To get an idea of what a mess this is, visit http://www.color.org/srgbprofiles.xalter and note that most everybody still uses v2, and that the quality of v2 profiles is all over the place. So, start reading from the bottom, and notice that there are two different recommended v2 profiles. One advantage of v4 over v2, if I remember, is more carefully specified transformations from sRGB to other colourspaces. This is from the site of the ICC, the guardian organization of sRGB. You may think that you are safe using an embedded profile, but not necessarily, because it could be an old v2 profile with terrible inverse transformations, or using a vague definition of Perceptual rendering intent, or because it embedded a profile which has nothing to do with the internal conversion to sRGB it used, or because your toolchain did a crude or damagingly irreversible conversion at some point. Or because Murphy laid down the law. (Profile experts out there, please correct me if I'm wrong.)

Yet another linear light warning: If I understand correctly, the issue is that there is not ONE linear light (because gamuts are truncated), and that even if there was such a thing, the conversion into and out of "the one linear light" (linear RGB with sRGB primaries and the same gamut, as is done in IM using -colorspace, say) necessarily is corrupted by "linear light inaccuracies" acquired along the long road from the actual scene to the image you are resizing. If we lived in a world of perfectly reversible conversions between completely fixed and standardized colour representations with unchanging parameters, we could trust in the "linear light is better" rule of thumb actually being a law. In the real world, unfortunately, we run the risk of making things worse by incorrectly correcting. Let alone that http://www.kenrockwell.com/tech/color-m ... -wimps.htm. Which is why I'm interested in knowing about cases where a linear light resizing toolchain makes things worse.

RE: Using EWA (-distort Resize) with Lagrange or Catrom http://forums.dpreview.com/forums/read. ... e=41366611:
Although I've never tried it, I would be very very surprised if this consistently gives good results. (Back of the envelope...) If you want an EWA sharper than Robidoux, Mitchell, Lanczos2, Lanczos2Sharp, Lanczos, or LanczosSharp, I would try the experimental method EWA LanczosRadius3:

Code: Select all

convert INPUT.IMG \
-filter Lanczos -define filter:blur=.9264075766146068 \
-distort Resize WIDTHxHEIGHT \
OUTPUT.IMG
(or a gamma corrected version), or I would try what is now called RobidouxSharp viewtopic.php?f=22&t=19823#p78728 http://www.imagemagick.org/Usage/resize/#robidoux_sharp.
Last edited by NicolasRobidoux on 2012-05-12T16:23:52-07:00, edited 31 times in total.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: best downsampling method for DSLR photographs

Post by fmw42 »

Nicolas,

Can you give your comments about techniques for updsampling, as well?

Are there techniques you have developed that could be integrated into IM, that are not there now? What about fractal zooming or integer zooming techniques? What is the state of the art today?
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: best downsampling method for DSLR photographs

Post by NicolasRobidoux »

@Fred:
I don't know what's the best method for downsampling, upsampling, or resampling.
I'm still learning. And I have some opinions. Biased opinions, actually.
For example, before taking the time to integrate something like fractal resampling, I'd build in a enlargement method using a subdivision method built on trained neural networks. (I'm not really suggesting we do this. I'm just illustrating "opinions".)
(Biased foot in mouth warning) For most uses, I think that one would get better results by carefully using and combining what's already in ImageMagick, or slightly extending it, than by implementing a cat's meow recently crowned king of the roars by its makers based on limited testing using criteria that crudely match what human viewers "want".

-----

Again, I am very hesitant putting some sort of list that may be taken as "Nicolas' clearly wrong and biased top 10 list" mostly because I am still figuring out what's happening.

I'll have more to say once Adam's thesis is out. But it's going to be opinions with gigantic caveats.
Last edited by NicolasRobidoux on 2012-05-12T11:01:47-07:00, edited 4 times in total.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: best downsampling method for DSLR photographs

Post by fmw42 »

NicolasRobidoux wrote:@Fred: I don't know what's the best method for downsampling, upsampling, or resampling.
I have some opinions. Biased opinions, actually. So, I'll pontificate when my soap box is cleaner.
Opinions are fine. No rush. When convenient is fine. Mainly interested in the complement, namely, upsampling (magnification). Your discussion on downsampling (minification) is quite interesting.

Note that the change from RGB to sRGB started at 6.7.5.5 but may not have been complete or bug free until 6.7.6.7. It is not clear what features were fixed at 6.7.5.5 and what the state was in between that and 6.7.6.7. It is causing me a lot of work changing many of my scripts and testing. However, it has been worthwhile as I found other bugs along the way, which appear mostly to have been fixed by now.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: best downsampling method for DSLR photographs

Post by NicolasRobidoux »

Quote from Eric Brasseur that shines a nice (gamma corrected, of course) light on some of my comments:
On the other hand, there never was a gamma problem within the printing or the movie industry. They have defined tight and scientifically-grounded standards and procedures and they perceive the public software tools as toys. (Note, about this, that some of the tools mentioned here yield incorrect results when they are used bluntly by amateurs, but when used properly by trained technicians they behave perfectly.)
When your entire toolchain, from start to finish, is not carefully built around a few colourspaces between which you can navigate with inconsequential loss, you have to be really careful.
Last edited by NicolasRobidoux on 2012-05-12T11:02:27-07:00, edited 1 time in total.
henrywho
Posts: 188
Joined: 2011-08-17T06:46:40-07:00
Authentication code: 8675308

Re: best downsampling method for DSLR photographs

Post by henrywho »

NicolasRobidoux wrote:RE: Using EWA (-distort Resize) with Lagrange or Catrom http://forums.dpreview.com/forums/read. ... e=41366611:
Although I've never tried it, I would be very very surprised if this consistently gives good results.
I suggested the original poster trying EWA Catrom because his "final" method of Resize+USM is actually giving similar results.

The new high-end digital camera are easily 18~24MP but they are not that pin-sharp when peeped @1:1. When downsized for web (usually sRGB bicubic), they look blurry, not eye-catching at all. At a result, USM is becoming a de-facto final step. Try downsampling the full-sized (7360x4912) out-of-camera backpack JPEG at http://www.dcfever.com/cameras/viewsamp ... cture=6967 down to 640x427 and you will know what I mean. The wooden fence looks rather blurry even with EWA RobidouxSharp. EWA-Catrom gives similar sharpness as the USM'ed version but less artifacts.


This backpack JPEG also demonstrates the advantage of linear-light resizing. The shiny nylon surface suffers a loss of brightness with sRGB resize, especially visible at the black components such as center strip.
Last edited by henrywho on 2012-05-12T04:44:52-07:00, edited 1 time in total.
Bonzo
Posts: 2971
Joined: 2006-05-20T08:08:19-07:00
Location: Cambridge, England

Re: best downsampling method for DSLR photographs

Post by Bonzo »

The new high-end digital camera are easily 18~24MP now but they are not that pin-sharp when peeped @1:1
This is correct and I always add a small amount of sharpening to my photos after processing.

Unmodified crop and a the crop with some sharpening ( GIMP default ): Radius = 5.0 Amount = 0.5 Threshold = 0

Image
Image
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: best downsampling method for DSLR photographs

Post by NicolasRobidoux »

henrywho wrote:The fence looks rather blurry even with EWA RobidouxSharp. EWA-Catrom gives similar sharpness as the USM'ed version with most less artifacts.
@Henry: Thank you, very much. I had not idea. (I was hoist on my own petard: One of the points of my long rant is that one should be careful not to assume that something is good because it's plausible (you actually have to try it). I then proceeded to state that EWA Catrom and Lagrange must be bad because it's implausible that they be good!)

-----

@Henry: When I have a minute (:-(), I'm going to have to check whether vanishing or negative weight sums viewtopic.php?f=22&t=20960 don't bite you with EWA Catrom.
If you feel adventurous, could you please try EWA Sinc-windowed Sinc 2, 3 and 4 (which you need to drive using defines because ImageMagick silently swaps Jinc for Sinc with EWA unless you "hardwire Sinc")? If you get good results with Catrom and Lagrange, I really wonder what these "forbidden" methods would put out. I suspect that when you are downsizing (without warping), this actually may work pretty well when you really want both antialiasing and sharpening.
Last edited by NicolasRobidoux on 2012-05-12T11:03:24-07:00, edited 5 times in total.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: best downsampling method for DSLR photographs

Post by NicolasRobidoux »

Man, don't I feel lucky to have "access" to people who know what they're doing!
Last edited by NicolasRobidoux on 2012-05-12T10:42:59-07:00, edited 1 time in total.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: best downsampling method for DSLR photographs

Post by NicolasRobidoux »

Note that I have also designed "Jinc-windowed Jinc EWA" equivalents to RobidouxSharp, meaning that, like RobidouxSharp, they are the most sharpening "arguably neutral" filters of their class according to a minimax criterion. henriwho has made it clear that sharpening more than "faithfulness demands" may be a good thing when the original image is soft (common now that CCDs are higher definition than optics). These heavily sharpened EWA Lanczoses sharpen a lot, although almost certainly less than EWA Catrom, which as discussed above I put in the "unreasonably sharpening" category, which is exactly why henriwho finds it useful (or, at least, finds it to be a good replacement for a final USM).

I usually call them "Lanczos*Sharpest". I have not promoted them because when enlarging they really do sharpen a lot, which one pays for in artifacts when the image is already sharp. But maybe they have a place in downsampling, or resampling slightly blurry images?

If you use (bleeding edge) IM7 from after colorspace bugs were fixed, you get them with

Code: Select all

magick INPUT.IMG \
-filter Lanczos2 -define filter:blur=0.88826421508540347 \
-distort Resize WIDTHxHEIGHT OUTPUT.IMG

magick INPUT.IMG \
-filter Lanczos -define filter:blur=0.88549061701764 \
-distort Resize WIDTHxHEIGHT OUTPUT.IMG

magick INPUT.IMG \
-filter Lanczos -define filter:lobes=4 -define filter:blur=0.88451002338585141 \
-distort Resize WIDTHxHEIGHT OUTPUT.IMG
If you want linear light resizing and INPUT.IMG is sRGB, insert -colorspace RGB after INPUT.IMG, and -colorspace sRGB before OUTPUT.IMG. With earlier versions of ImageMagick, you must replace magick by convert, and use the classic way of going through linear light.

The above 4-lobe version, in particular, I would never recommend to upsample an already sharpened natural image, but it has a "flavor" which I've not really seen in other schemes. It may be an interesting one to try. I was really impressed with what it does in situations where you'd expect a lot of staircasing.

P.S. Had a quick look, using the backpack image pointed to by Henry. The 4-lobe version does show significant haloing. It is very strongly sharpening. Too much when pixel peeping for my taste. On the other hand, it looks like people like sharpening more than I do. The 3-lobe version, on the other hand, shows less "resonance" with the checkerboard mode than orthogonal Lanczos 3, but I'm not sure it's necessary better. Maybe a nudge, because it does not introduce "noise" in strips which should be flat coloured, and it is slightly less "jaggy".

It's almost as if maximally sharpened EWA Lanczos 4-lobe was specifically designed for those who USM the result or orthogonal Lanczos, and maximally sharpened Lanczos EWA 3-lobe was designed for those who like the sharpness of orthogonal Lanczos but could do with a bit less jagginess and halo. This being said, without pixel peeping, I find it almost impossible to see the difference between the three. Pixel peeping, however, my personal favorite is the maximally sharpened EWA Lanczos 3-lobe. No question. (Tested through linear light.)
Last edited by NicolasRobidoux on 2012-05-12T19:50:07-07:00, edited 14 times in total.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: best downsampling method for DSLR photographs

Post by NicolasRobidoux »

I'm also interested in knowing how well the above maximally sharpened EWA Lanczos 3 does when downsampling, because high on my TODO list is program a Jacobian adaptive blend of standard (tensor) Lanczos 3 and a compatible EWA method, which will take over when downsampling, for GEGL, the new graphics engine of GIMP. TTBOMK, the goat-invasion branch of GIMP silently swaps out Lanczos and replaces it by a Jacobian adaptive method of mine (several other people are credited, including Anthony Thyssen), LBB-Nohalo, known in GEGL as gegl-sampler-lohalo.c, which really does not do at all what someone expects of out Lanczos: It's an almost completely halo free method, in other words, it's just about as different in flavor as can be from standard Lanczos. (Most reasonable people would ditch the "almost" in "almost halo free", but I'm a mathematician, so I believe in sets of measure zero.)
So, I'm pondering what method to glue to standard Lanczos, method which takes over when downsampling becomes significant. (Really, it's there because there is no really good way to simultaneously warp and downsample with tensor methods, and although Anthony and I are scheming to change that, I suspect that we won't fully succeed.) The maximally sharpened EWA Lanczos 3 and 4 are pretty obvious candidates, at least on paper. So are the "Radius = number of lobes" versions of EWA Lanczos of which I gave an example in the top post of this thread.
Unfortunately, I've not looked too carefully at downsampling. You guys do.
And I'm definitely not "a photographer". But it would make sense to choose this method to make photographers happy.

BTW, "you photographers", please have a look at viewtopic.php?f=22&t=20691. One or two moire-prone images that fall within the needed specs (which are a bit looser than the posting makes them out to be; I did not want to sort through unsuitable images and I'm busy, so I put up a steep ramp) would be nice additions, and the official version of the test image bank is not out yet.
Last edited by NicolasRobidoux on 2012-05-12T10:40:09-07:00, edited 8 times in total.
henrywho
Posts: 188
Joined: 2011-08-17T06:46:40-07:00
Authentication code: 8675308

Re: best downsampling method for DSLR photographs

Post by henrywho »

Another interesting sample at http://www.dcfever.com/cameras/viewsamp ... cture=6978

The red grids on the robot's chest and the gold solder points on the PCB are de-colored which I suspect being an artifact of USM. Over-sharpening EWA downsample does not have this problem.
henrywho
Posts: 188
Joined: 2011-08-17T06:46:40-07:00
Authentication code: 8675308

Re: best downsampling method for DSLR photographs

Post by henrywho »

NicolasRobidoux wrote:If you feel adventurous, could you please try EWA Sinc-windowed Sinc 2, 3 and 4 (which you need to drive using defines because ImageMagick silently swaps Jinc for Sinc with EWA unless you "hardwire Sinc")?
Do you mean " ... -define filter:filter=Sinc -define filter:window=Sinc -define filter:lobes=2 -distort resize ... "?

BTW, while I am suggesting EWA Lagrange downsize as a better alternative to the common downsize+USM approach, my current choice of downsizing it still EWA-JincJinc3.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: best downsampling method for DSLR photographs

Post by NicolasRobidoux »

fmw42 wrote:... However, it has been worthwhile as I found other bugs along the way, which appear mostly to have been fixed by now.
As mentioned elsewhere, I have an acronym for this rather common sort of thing: TYF ("Thank You Fred").
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: best downsampling method for DSLR photographs

Post by NicolasRobidoux »

henrywho wrote:...
The red grids on the robot's chest and the gold solder points on the PCB are de-colored which I suspect being an artifact of USM. Over-sharpening EWA downsample does not have this problem.
Which one? What you call JincJinc3 (which is the "improved" version of the current EWA LanczosSharp, with a blur barely below 1)?
Post Reply