Thumbnail creation from user uploaded images

[carl]

Hi,

I am trying to make a program to create thumbnail images from user provided input images. I had some questions regarding how to do this. Generally speaking, I would prefer the images to be small, square images in the same file format as the input image. I am using the MagickWand API, through a Golang wrapper library.

1. What is the the difference between Properties, Profiles, Artifacts and Options? These all appear to be metadata, and there seems to be a lot of overlap in the data exposed here? Are all Profiles color profiles, or are there other kinds? Are all options set by my application, or could they be set by MagickCore when decoding the image? Is it generally safe to remove all of these? The "-strip" option to convert removes *some* of these, but not all of them.

2. I would prefer to drop color profiles, since they seem to be a few KB in size; enough to rival the image data. Is it okay to just convert all thumbnail images to the sRGB color space and depend on browsers to expect images in the sRGB color space?

3. Is it still correct to convert the input image to LAB colorspace (or another perceptually uniform colorspace) before resizing? I would like to avoid any easily avoidable degradation in image quality. From reading online it seems this is the correct way to resize, but perhaps ImageMagick does it already?

My main concern is that when I did this in my code, it didn't make any significant difference in the output thumbnail. I'm wondering if I was just noticing quantization errors due to the colorspace conversion. How do I tell if I did it right? Is there a kind of image that really shows off the difference?

4. Cropping the image down to a square has proved tricky, and I don't think I am doing it right. I am using MagickGetImagePage to get the width and height, and then cropping it down to the largest square that fits inside. The problem is that MagickCropImage doesn't seem to update the ImagePage afterwards.

An additional complication: Trying to do the equivalent of the +repage arg doesn't seem to have an equivalent MagickWand function. In my case, some of the input PNGs have "oFFs" sections that place the image outside of width / height box. Calling MagickCropImage warns that the image would be empty, so I need a way to discarding the offset information in the MagickWand. What should I do?

[snibgo]

What is the the difference between Properties, Profiles, Artifacts and Options?

A property (and an artifact) has a single key with a value. The value can be any size, but is typically a few bytes.

A profile is a data structure, rather than a single key-value pair. It often has thousands of bytes. There are different types of profiles eg 8bim, exif and icc. Icc profiles, aka colour profiles, define the translation between pixel values and actual colours that a device should show.

Is it okay to just convert all thumbnail images to the sRGB color space ...

It depends on what you mean by "okay".

Is it still correct to convert the input image to LAB colorspace (or another perceptually uniform colorspace) before resizing?

Was it ever "correct" to do that? Why do you think that? For ordinary photos (or video frames) the most "correct" method would be with scene-referred data. This can be approximated by converting an image to linear RGB. However, in my experience for ordinary photos the difference beween resizing in sRGB, linear RGB, Lab or any other colorspace is not visible.

The difference is most visible on photos with very high local contrast, eg night scenes with many visible artificial light sources.

On MagickGetImagePage(), see magick-image.c. The function doesn't return the image with and height (which crop will change) but the page width and height (which crop doesn't change). You can change the page dimensions with MagickSetImagePage().

[carl]

Thank you for the response. More direct responses inline:

What is the the difference between Properties, Profiles, Artifacts and Options?

A property (and an artifact) has a single key with a value. The value can be any size, but is typically a few bytes.

Can you clarify the difference? They appear to do the same thing. Why make them separate?

A profile is a data structure, rather than a single key-value pair. It often has thousands of bytes. There are different types of profiles eg 8bim, exif and icc. Icc profiles, aka colour profiles, define the translation between pixel values and actual colours that a device should show.

I understand this, but my question is more at why this is in a different section. For example, why not list everything (profiles, artifacts, etc.) under a single key value store and call that "Properties"?

Is it okay to just convert all thumbnail images to the sRGB color space ...

It depends on what you mean by "okay".

Is it still correct to convert the input image to LAB colorspace (or another perceptually uniform colorspace) before resizing?

Was it ever "correct" to do that?

The Imagick documentation calls attention to it here: http://www.imagemagick.org/Usage/resize/#resize_lab though it does not seem to make a recommendation, or mention what the default behavior is. There are additional sites online that make the same claim, and say that resize must be done in a linear colorspace.

Why do you think that? For ordinary photos (or video frames) the most "correct" method would be with scene-referred data. This can be approximated by converting an image to linear RGB. However, in my experience for ordinary photos the difference beween resizing in sRGB, linear RGB, Lab or any other colorspace is not visible.



The difference is most visible on photos with very high local contrast, eg night scenes with many visible artificial light sources.

On MagickGetImagePage(), see magick-image.c. The function doesn't return the image with and height (which crop will change) but the page width and height (which crop doesn't change). You can change the page dimensions with MagickSetImagePage().

MagickSetImagePage() does not seem to do what the name implies. Calling this function with (w, h, 0, 0) does temporarily reset the page dimensions, but calling MagickCropImage resets them back to what they were before. I have been keenly reading the image magick source code, but the code is not easy to follow. (particularly with GIFS, which have multiple Images within the same wand, but which have different offsets.)

[snibgo]

http://www.imagemagick.org/Usage/resize/#resize_lab says:

... in linear colorspace like RGB (or LAB or LUV).

Yes, RGB is linear. (I generally call it "linear RGB" to be clear.) But LAB and LUV are not. LAB and LUV are more perceptually uniform than sRGB or linear RGB, or AdobeRGB etc.

(In slight defence of that page, I would say that resizing in Lab may give a more pleasing result than resizing in sRGB. But it rarely makes a noticeable difference.)

The Imagick documentation calls attention to it here: http://www.imagemagick.org/Usage/resize/#resize_lab though it does not seem to make a recommendation, or mention what the default behavior is.

The behaviour is always to do resizing in the current colorspace. If you want to do it in a different colorspace, use "-colorspace" (EDIT: or "-profile") before the operation (and possibly change it back afterwards).

There are additional sites online that make the same claim, and say that resize must be done in a linear colorspace.

Any such claims are false. There is no "must". It depends on what you are doing.

A photograph can be considered as a collection of samples of light emitted from parts of some scene. So resizing a photograph can be considered as re-sampling the original scene. And if that's what you want, you should use pixel values that are proportional to the light emitted. For earth_lights_4800.tif, this means going back to NASA source data to get a "scene-referred" image. That is the most "correct" method.

If all we have is a JPEG that has been mangled in many ways (including, for this image, heavy spatial distortion), the best we can do is convert it to linear RGB before resizing. But this doesn't make it "correct", merely "less incorrect".

And, I repeat, for ordinary photos there is no noticeable difference.

I don't have time today to write a test program for MagickSetImagePage() and MagickCropImage(). Perhaps you can show your test.

[snibgo]

I forgot to say: on artifacts and properties, see http://www.imagemagick.org/Usage/basics/#settings

[carl]

http://www.imagemagick.org/Usage/resize/#resize_lab says:

... in linear colorspace like RGB (or LAB or LUV).

Yes, RGB is linear. (I generally call it "linear RGB" to be clear.) But LAB and LUV are not. LAB and LUV are more perceptually uniform than sRGB or linear RGB, or AdobeRGB etc.

(In slight defence of that page, I would say that resizing in Lab may give a more pleasing result than resizing in sRGB. But it rarely makes a noticeable difference.)

To clarify, when I say linear, I mean in terms of visual perception of brightness. It is unclear to me that resizing in plain RGB will not distort the color or the brightness.

As for it being a noticeable difference: That is true based on my own sampling, but being generally good is not enough. I am interested in making sure the corner cases are covered, because The number of images being converted will be in the tens of millions. I won't be able to check all the thumbnails for correctness, so I need to reason about them ahead of time.

The Imagick documentation calls attention to it here: http://www.imagemagick.org/Usage/resize/#resize_lab though it does not seem to make a recommendation, or mention what the default behavior is.

The behaviour is always to do resizing in the current colorspace. If you want to do it in a different colorspace, use "-colorspace" (EDIT: or "-profile") before the operation (and possibly change it back afterwards).

There are additional sites online that make the same claim, and say that resize must be done in a linear colorspace.

Any such claims are false. There is no "must". It depends on what you are doing.

Of course, I can do whatever, but my goal is to have the thumbnail most closely match the original. I am willing to accept the crop and loss of high-frequency data in the output, but losing the color and brightness is not acceptable.

A photograph can be considered as a collection of samples of light emitted from parts of some scene. So resizing a photograph can be considered as re-sampling the original scene. And if that's what you want, you should use pixel values that are proportional to the light emitted. For earth_lights_4800.tif, this means going back to NASA source data to get a "scene-referred" image. That is the most "correct" method.

If all we have is a JPEG that has been mangled in many ways (including, for this image, heavy spatial distortion), the best we can do is convert it to linear RGB before resizing. But this doesn't make it "correct", merely "less incorrect".

I am using "correct" loosely. The goal is not to recreate the original scene of photo, but to make a smaller version that faithful represents the larger one. It's okay if the source image is full of JPEG artifacts, as long as the thumbnail is true to it.

And, I repeat, for ordinary photos there is no noticeable difference.

I don't have time today to write a test program for MagickSetImagePage() and MagickCropImage(). Perhaps you can show your test.

Sure. The code is in Go, but that's just a very thin wrapper around the MagickWand API. Here's the source, and the output:

Code: Select all

package main

import (
	"fmt"
	"io"
	"io/ioutil"
	"os"
	"testing"

	"gopkg.in/gographics/imagick.v2/imagick"
)

func TestThumbnailWithOffset(t *testing.T) {

	// Create a test image
	mw := imagick.NewMagickWand()
	defer mw.Destroy()
	pw := imagick.NewPixelWand()
	defer pw.Destroy()

	mw.NewImage(100, 200, pw)
	mw.SetImageFormat(string("PNG"))
	f, err := ioutil.TempFile("", "testimage.png")
	if err != nil {
		t.Fatal(err)
	}
	defer os.Remove(f.Name())
	defer f.Close()

	// set offset far outside.
	if err := mw.SetImagePage(100, 200, 300, 400); err != nil {
		t.Fatal(err)
	}

	if err := mw.WriteImageFile(f); err != nil {
		t.Fatal(err)
	}
	if _, err := f.Seek(0, io.SeekStart); err != nil {
		t.Fatal(err)
	}
	//////////////////////////////////////////////////////////////////////
	//  TEST STARTS HERE
	//////////////////////////////////////////////////////////////////////
	mw2 := imagick.NewMagickWand()
	if err := mw2.ReadImageFile(f); err != nil {
		t.Fatal("unable to decode image")
	}
	defer mw2.Destroy()

	printPage("start", mw2)

	// Find the largest center square
	w, h := mw2.GetImageWidth(), mw2.GetImageHeight()
	var neww, newh uint
	var x, y int
	if w > h {
		neww = h
		newh = h
		x = int((w - neww) / 2)
		y = 0
	} else {
		neww = w
		newh = w
		x = 0
		y = int((h - newh) / 2)
	}
	newmw := mw2.Clone()
	newmw.ResetIterator()
	defer newmw.Destroy()

	printPage("clone", newmw)

	// Some PNGs have an `oFFs` section that messes with the crop
	if err := newmw.SetImagePage(w, h, 0, 0); err != nil {
		t.Fatal("unable to repage image")
	}
	printPage("repaged", newmw)

	if err := newmw.CropImage(neww, newh, x, y); err != nil {
		t.Fatal("unable to crop thumbnail")
	}
	printPage("cropped", newmw)

	newmw2 := mw2.Clone()
	newmw2.ResetIterator()
	defer newmw2.Destroy()
	printPage("clone", newmw2)

	if err := newmw2.ResetImagePage("0x0+0+0"); err != nil {
		t.Fatal("unable to repage image")
	}
	printPage("repaged again", newmw2)

	if err := newmw2.CropImage(neww, newh, x, y); err != nil {
		t.Fatal("unable to crop thumbnail")
	}
	printPage("cropped again", newmw2)
}

func printPage(comment string, mw *imagick.MagickWand) {
	w, h, x, y, err := mw.GetImagePage()
	if err != nil {
		panic(err)
	}
	fmt.Printf("%s: w=%d h=%d x=%d y=%d\n", comment, w, h, x, y)
}

And the output:

Code: Select all

start: w=100 h=200 x=300 y=400
clone: w=100 h=200 x=300 y=400
repaged: w=100 h=200 x=0 y=0
cropped: w=100 h=200 x=0 y=50
clone: w=100 h=200 x=300 y=400
repaged again: w=0 h=0 x=0 y=0
cropped again: w=100 h=200 x=0 y=50

[snibgo]

I'm not familiar with Go or IMagick, but the code seems clear. The output is what I would expect. What do you think is wrong? Perhaps this line:

Code: Select all

clone: w=100 h=200 x=300 y=400

That line reports on newmw2, which is an unchanged clone of mw2, which hasn't been changed since the "start" report.

[carl]

I'm not familiar with Go or IMagick, but the code seems clear. The output is what I would expect. What do you think is wrong? Perhaps this line:

Code: Select all

clone: w=100 h=200 x=300 y=400

That line reports on newmw2, which is an unchanged clone of mw2, which hasn't been changed since the "start" report.

The part I think is wrong (or at the very least confusing) is that setting the image page to remove the offset didn't actually remove the offset. I want the offset to be +0+0. I am showing in the code 2 different ways of clearing it (using SetImagePage and ResetImagePage). The "clone" is just proving that I am starting from a fresh copy, and not inheriting.


The most troubling part is that the Rectangle being returned by GetImagePage is clearly wrong after the crop. Cropping the image(page?) to 100x100 should not result in a 100x200 image larger than that, and reset to a new offset.

[snibgo]

The part I think is wrong (or at the very least confusing) is that setting the image page to remove the offset didn't actually remove the offset.

Which of your results shows that?

For example, you use SetImagePage(w, h, 0, 0) on newmw, and the page offsets are correctly shown as 0,0. When you crop the image starting at offset (0,50), because those are the values of x,y the new page offsets are (0,50).

Cropping the image(page?) to 100x100 should not result in a 100x200 image larger than that, and reset to a new offset.

But cropping an image will change the image width and height, not the page width and height. As your results show, cropping an image doesn't change the page dimensions.

[carl]

The part I think is wrong (or at the very least confusing) is that setting the image page to remove the offset didn't actually remove the offset.

Which of your results shows that?

For example, you use SetImagePage(w, h, 0, 0) on newmw, and the page offsets are correctly shown as 0,0. When you crop the image starting at offset (0,50), because those are the values of x,y the new page offsets are (0,50).

I guess it isn't clear what a "Page" actually is. Without any further documentation, I am just guessing. Maybe you can answer some questions to clarify it:

1. What is the relationship between an Image and an ImagePage? Does an image /have a/ page or does the page /have a/ image?
2. What is the relationship between a Page and an ImagePage? They have nearly identical sounding names, and hardly any documentation. See MagickGetPage() and MagickGetImagePage
3. Does the page have to contain the image? Can it be wider than the image or taller?
4. What is the Page offset relative to?

Cropping the image(page?) to 100x100 should not result in a 100x200 image larger than that, and reset to a new offset.

But cropping an image will change the image width and height, not the page width and height. As your results show, cropping an image doesn't change the page dimensions.

Look at the second crop:

Code: Select all

clone: w=100 h=200 x=300 y=400
repaged again: w=0 h=0 x=0 y=0
cropped again: w=100 h=200 x=0 y=50

This is clear evidence that cropping DOES change the page width, height, and offsets. It went from all zeros to other values. Do you by chance have contact with the original author of the cropping code?

[snibgo]

Page data (aka "canvas" data) is four integers: width, height, x-offset and y-offset. These are elements of image structures, so every image has a page.

Page width and height are always >= 0. But zero width or height will generally be changed to the image dimension. (IM doesn't regard an image with a zero dimension as a valid image.) Offsets can be negative.

The page data says where the image came from. So if we have a 1000x2000 image that has a page of the same size with zero page offsets, and "-crop" or "-trim" it down to 10x20 pixels starting at (110x120) the page numbers will be (1000,2000,110,120) which may be written in the usual WxH+X+Y format as 1000x2000+110+120. The image dimensions will be 10x20, as requested.

The page data can be used to reconstruct the source image by putting the image back in the same location. For example we can crop a number of rectangles from an image, process all the rectangles, and "-compose Over -layers merge" the rectangles to their original positions.

MagickGetImagePage() gives you the four numbers from an Image structure. They may also be recorded in the WxH+X+Y string format in an ImageInfo structure, and MagickGetPage() gets those.

If our image is 10x20 with page 1000x2000+110+120 and we want a 12x13 crop from the top-left of the image, we can either "+repage -crop 12x13+0+0" or "-crop 12x13+110+120".

3. Does the page have to contain the image? Can it be wider than the image or taller?

The page can be wider or narrower or taller or shorter than the image. Typically the page will be larger than the image or the same size, and the image won't straddle any edge of the page, so the page contains all the image. But the page can contain just part of the image, or none of it.

4. What is the Page offset relative to?

To the top-left of the page.

This is clear evidence that cropping DOES change the page width, height, and offsets. It went from all zeros to other values.

You are correct that zero width or height will be changed by the "-crop" operation, because IM doesn't recognize zero as a valid dimension. (Personally, I think it should.) Non-zero page dimensions will not be changed by "-crop".
The offsets given to "-crop" are relative to the page, not to the image. So we can try to crop to a rectangle within a page that doesn't contain image pixels, and "-crop" will return a warning like:

Code: Select all

convert.exe: geometry does not contain image `ROSE' @ warning/transform.c/CropImage/666.

After a crop, the page offsets will be the coordinates within the page of where the crop originated.

Do you by chance have contact with the original author of the cropping code?

I don't know who that is.