ImageMagick v6 Examples --
Lens Correction

Index

ImageMagick Examples Preface and Index

Introduction to Lens Correction

Non-scaling Restraint

Ready-made Parameter Sets

Calibrating From Scratch

• Basic Appoach
• In Practice

Example

Keyboard Example (by El-Supremo)

When taking photographs the images generated are actually distorted by both lens effects, and spherical perspective effects. If you plane to use photos you will generally need to correct for these effects, and that is what will be looked at in this section.

The majority of section was contributed by Wolfgang Hugemann.

---

Introduction to Lens Correction

Wide-angle lenses (or rather zoom lenses when set to short focal length) typically produce a pronounced barrel distortion. This distortion can however be mostly corrected by applying suitable algorithmic transformations to the digital photograph. The most-used lens correction algorithm, introduced by Panorama Tools and used by PTlens, is also offered by ImageMagick, as Barrel Correction Distortion Method.

This distortion is controlled by four transformation parameters a, b, c, d, which have to be chosen sensibly in order to correct the lens distortion produced by a certain camera at a certain focal length. Suitable values for these parameters can hardly be found by trial and error. In the following, I describe how to determine the lens correction parameters effectively by the use of Hugin, a free graphical user interface for Panorama Tools, which is available for various operating systems.

If you don't want to deal with the details of lens correction, you may skip the rest of this page and just buy PTlens, which offers sophisticated lens correction for a huge number of digital cameras and lenses at a reasonable price (by use of its vast lens database). Nowadays, some digital cameras (such as the Nikon P7000) even incorporate lens correction in their internal image processing steps. For photographs taken with cameras that don't offer this possibility, ImageMagick enables you to integrate lens correction as one step of a larger image processing script.

The following text is an abridged version of paper Correcting Lens Distortion (PDF), dealing with applications in accident reconstruction. The explanation given here is a more hands-on approach, concentrating on the ways to get in hold of the adequate lens correction parameters.

Non-scaling Restaint

As described in the Barrel Distortions The barrel distortion is defined by the mathematical formula

  R = (a * r^3 + b * r^2 + c^r + d) * r

with r as the distance to the geometrical image center of the digital photograph and R as the equivalent radius for the corrected image. The radii r and R are normalised by half of the smaller image dimension, such that r = 1 for the midpoints of the equivalent edges of the photograph. When correcting digital photographs, we should pay attention to the non-scaling restraint

  a + b + c + d = 1

which obviously gives a result of R = 1 when the input r = 1. Panorama Tools calculates the parameter d by the other parameters via

  d = 1 - a - b - c

leaving us with three free model parameters, so the parameter d is typically omitted. IM can calculate this automatically if not provided.

---

Ready-made Parameter Sets

PTlens's current lens database, being the "marrow" of the program, is encrypted and can only be read by PTlens itself. Until February 2006, howwever, PTlens's database was coded in XML format, i.e. an easily editable text format. This 2006 version of PTlens's XML database is still (legally) available at Hugin's SourceForge Website and provides data for a lot of older camera models.

When PTlens's database became encrypted, the authors of Hugin tried to establish a free XML coded lens database as an alternative. This database is called LensFun and can be downloaded. It comes with a complete programming interface, but all you basically need is the information for your camera in the XML file. As an example, the lens correction parameters for the once popular Nikon Coolpix 995 are found in the file compact-nikon.xml, which resides in the directory \data\db. The file can be examined by the use of a text editor or an XML viewer:

As can be taken from the camera's technical data sheet, the zoom range of the Nikon Coolpix 995 is 8.2 – 31.0 mm, corresponding to 38 – 152 mm for 35 mm film cameras. This gives a crop factor of 152 / 31 = 4.90, which roughly corresponds to the 4.843 given the XML file. The coefficients of the correction by barrel distortion are supplied for six focal lengths, namely 8.2 mm, 10.1 mm, 13.6 mm, 18.4 mm, 23.4 mm, 28.3 mm and 31.0 mm. The coefficients a and c are, for this lens, set to zero, i.e. the distortion is described only by the second-order term b.

Note that many lens's will also have values for 'a' and 'c' parameters as well, and these should also be interpolated in a similar way.

If we have a photograph DSCN0001.jpg taken with a Nikon Coolpix 995 set to the shortest focal length, this photograph could be corrected by ImageMagick via

The file name extension _pt is used by PTlens to mark corrected images.

For the six focal lengths provided, the correction coefficient b can be read from the XML file. For other focal lengths, the suitable value can be determined by interpolation between the two neighbouring focal lengths. As an alternative, the dependency of b on the focal length f can be approximated by the polynomial

  b = 0.000005142 * f^3 - 0.000380839 * f^2 + 0.009606325 * f - 0.075316854

So the focal length (e.g. read from the EXIF information) is used to calculate the lens correction parameter b in the first step, and then, in a second step, the lens correction (i.e. barrel distortion) is performed using this value as the b parameter.

The Windows section shows a VBScript Example in which the above equations are used, with the focal length being extracted from a Nikon Coolpix 995 photograph via identify.

---

Calibrating from scratch

Basic Approach

When determining the lens parameters, all programs rely on the same paradigm: the ideal perspective mapping should map real world straight lines to straight lines in the image. So if a set of real-world points P₀, P₁, ..., P_n is known to lie on a straight line, their images p₀, p₁, ..., p_n must also fall onto a straight line. Any deviation from this rule has to be attributed to lens distortion.

We need two points to determine the two parameters defining a straight line (e.g. slope and intersection on the y-axis). Each additional point supplied will provide another equation to determine the lens correction parameters. So if our functional approach has only one free parameter b (as for the Nikon Coolpix 995 above), we would have to provide at least three points on a straight line in order to determine the sought lens correction parameter b.

Putting it more concrete: The distortions model only uses the parameter b, i.e. the coordinates of the corrected image X1, Y1 can be calculated from the coordiantes of the digitl photograph by

r = s * sqrt(x1^2 + y1^2)
X1 = [(1-b) + b r^2] * x1
Y1 = [(1-b) + b r^2] * y1
Y1 = k1 * X1 + k2

This results in one equation for each point supplied on the same straight line

[(1-b) + b r^2] * y1 = k1 * [(1-b) + b r^2] * x1 + k2

with: r = s * sqrt(x1^2 + y1^2)

Thus three point would suffice to determine the parameters describing the straight line and the lens distortion k1, k2, b.

In practice, calibration programs mostly use a rectangular grid of straight lines, often a chequerboard, to generate a set of equations and then calculate the mapping parameters by a nonlinear least-squares fit. Some programs generate the set of control points on their own, often using pre-defined templates; other programs require the user to select the control points from the calibration image.

In Practice

In the following, we will demonstrate this technique by the use of Hugin. There is also a ready-made "Simple Lens Calibration Tutorial" on Hugin's Website, but at the time of this writing, it seems to be too simple to provide reliable parameters that can later be used for a multitude of corrections.

In principle, the control points for the calibration could be picked within Hugin itself. Practically speaking, this is however to tedious for the large number of control points that is needed to determine of the lens correction parameters reliably. I will therefore describe how to set-up the control point file manually.

You should take a photograph of a modern building, as proposed on PTlens' website. Follow the instructions given there. The photographs may show perspective distortion:

Then establish a grid on this photograph by determining the pixel coordinates of a lot of grid points. You can use any image viewer to do this, namely one that can store such data. I (working under Windows) used polylines in WinMorph to do so.

Then open the calibration image with Hugin, set the panoramic mapping to rectilinear (on the last tab page) and store the project. Open the Hugin project file (which is a plain text file with the extension PTO) with a text editor and supply a point list. A single line in its section # control points looks like this:

  c n0 N0 x175.0 y87.8 X1533.3 Y62.6 t3

where x, y are the pixel coordinates in the source image and X, Y are the point coordinates in the target image -- which actually are two versions of the same image in this special case. (Usually these would be two different images lying next to each other in a panorama.) The intro c n0 N0 is standard code and the trailer t3 is the numbering of the associated straight line, starting with the index 3. As you can take from the above example, the point coordinates may have fractional parts.

Of course, x, y and X, Y have to lie on the same straight line. They must however not be identical, as the optimiser would refuse to work under such conditions. The easiest approach is to use the reverse ordering for the target coordinates X, Y. You can use any program to establish a set of lines based on the intersection points derived from the photograph. (I used Excel to perform this task.) When ready, copy the point list to the corresponding section of the PTO file, save it and re-open it with Hugin. The result should look like this:

Then switch to Hugin's optimiser on the next tab, choose "Optimize the custom parameters below", pick a, b, c and then press "Optimize now!". The result should give parameters which are close to 0.01. If not, check the point settings on the tab page "Control points" (which is somewhat limited by the fact that each point is used twice, such that you will only see the second half of the control point set). If you have calculated large values for the parameters, the control points are probably out of order or not correctly associated with their corresponding lines.

If you have to re-run the optimiser, turn on the check box "Edit script before optimising" on the right button of the according tab page: Set the start vector a, b, c back to a0.0 b0.0 c0.0 before re-starting the optimiser. Otherwise the (non-linear) iteration will start at an off-point and would probably not yield the correct result.

For a camera equipped with a fixed lens, one does this calibration once and for all. For a camera with a zoom lens, one has to cover the entire range of focal lengths by calibrating at about five different focal lengths.

A ready-made example, both with a calibration image and the corresponding Hugin project has been provided in a ZIP file olympus_c2500l.zip.

---

Examples of Lens Correction

Camp Mobile

The original photograph of the campmobile to the left had to be taken from a rather short distance during dusk, as space was limited due to a steep declivity at the back of the photographer. (The poor lighting conditions explain the blue tint which stems from severe lightening in the post-processing.) The original photograph shows pronounced barrel distortion, visible especially in the horizontal stripe near the top of the image and for the back corner of the build-up. The Nikon Coolpix 995 used for this shot is found in PTlens's database, so the distortion could readily be corrected, as seen in the middle.

The image to the right shows the difference between greyscale versions the two photographs, calculated by subtraction of the two, followed by negation and extreme clipping and Gamma correction. Again, the effects of the correction are best illustrated by the horizontal stripe at the top. The white circle (indicating zero difference) results from the non-scaling restriction: the points on a circle with a diameter equal to the smaller dimension of the image remain unaltered.

Two Keyboard by el_supremo

The photo that I took of my two keyboards has a very obvious barrel distortion in it because it was taken at a focal length of 17mm.

This kind of distortion can be corrected with, for example, Canon's Digital Photo Professional (I have a Canon 50D camera). Other manufacturers of SLR cameras usually provide software to do this kind of correction for their lenses but I wanted to see how well the above examples would work on this photo.

The first step is to go to the LensFun WebSite and download the latest version of their camera database.

Unzip the package (winzip can unzip a .tar.gz file in Windows) and then in the "lensfun/data/db" directory look for the file which corresponds to your camera manufacturer. In my case I looked at "slr-canon.xml" which can be edited with any text editor.

Now I find the information for the specific lens I am using which in this case is an "EF-S 17-85mm". The information for that lens looks like this:

<lens> <maker>Canon</maker> <model>Canon EF-S 17-85mm f/4-5.6 IS USM</model> <mount>Canon EF-S</mount> <cropfactor>1.6</cropfactor> <calibration> <distortion model="ptlens" focal="17" a="0.021181" b="-0.055581" c="0" /> <distortion model="ptlens" focal="20" a="0.019344" b="-0.043786" c="0" /> <distortion model="ptlens" focal="22" a="0.015491" b="-0.026682" c="0" /> <distortion model="ptlens" focal="28" a="0.008084" b="-0.007472" c="0" /> <distortion model="ptlens" focal="30" a="0.005522" b="-0.001763" c="0" /> <distortion model="ptlens" focal="35" a="0.003149" b="0.002207" c="0" /> <distortion model="ptlens" focal="44" a="0" b="0.008269" c="0" /> <distortion model="ptlens" focal="53" a="0" b="0.008792" c="0" /> <distortion model="ptlens" focal="61" a="0" b="0.00738" c="0" /> <distortion model="ptlens" focal="72" a="0" b="0.006226" c="0" /> <distortion model="ptlens" focal="78" a="0" b="0.007095" c="0" /> <distortion model="ptlens" focal="85" a="0" b="0.007288" c="0" /> </calibration> </lens>

The calibration entries give distortion values for a range of focal lengths from 17mm up to 85mm. If the focal length I needed was between two of those values, I could either choose whichever was closest or I could interpolate the values. Since the photo I'm correcting was taken at 17mm I need the information from the first line of the calibration information. That gives me the values:  a="0.021181"  b="-0.055581"  c="0"

These are the three parameters which are used to correct the lens distortion.

However for some older versions of IM, The barrel distortion correction requires a fourth parameter d. Fortunately, it is easy to calculate the value of d from the other three using this simple formula:  d = 1-a-b-c  That means:  d="1.0344".

IM will work out this value automatically, of it is not provided as a distortion argument, but some older versions of IM did not do this.

That makes the actual Barrel Distortion, to correct the lens distortion...

Of course you should not save to JPEG until you have finished processing your image completely, due to the JPEG lossy compression.

In the original photo the distortion is particularly obvious along the bottom of the music stand and along the upper keyboard. These distortions are almost completely gone in the output photo. A visual comparison of this result with that obtained from Canon's software shows essentially the same result.

El-Supremo