Suggesting a floating point Q32 image format

[fmw42]

Hello,

I have been doing image processing and scripting of image functions since 1978, but am relatively new to IM. I want to build more IM scripts (see my web page at http://www.fmwconcepts.com/imagemagick/index.html), but I find that although -fx can do a lot, it is still restrictive.

I would like to suggest that you implement a 32-bit image format (at least for grayscale images, but better if for truecolor, i.e. 32-bits per channel).

One of the problems that I have is doing image arithmetic - both single image and multiple image operations - and stringing them together to get some particular result. Your -evaluate is a start for single image operations, but it clips the data between each step. Thus doing something like a*u + b does not work in all cases and the result depends upon whether you do it that way or b + a*u, i.e. it is order dependent. I have tried implementing a simple script to do brightness and contrast adjustment in one step using slope and intercept and -evaluate cannot handle all the possible cases of slope and intercept without premature clipping. So I went to -fx to handle this, but is of course rather slow.

For single image operations with functions like you use in -fx (add, subtract, multiply, divide, ln, exp, sqrt, sin, cos, atan etc and polynomial) -evaluate would work fine if you could put multiple evalutes in a single command line, BUT not clip until the end. This could be done by having a floating point image type and not clipping for each evaluate and then add a new function that would use the min and max values from the floating point format and linearly scale them to 0 and Q for 8-bit or 16-bit results in order to convert the floating point format to normal 8 or 16 bit per channel.

If you had a floating point image format, then one could also do things to combine 2 images with functions like add (plus or minus weights), multiply, divide, pow, min, max, mod, etc, like you do in -fx. Some of these can be done now with -compose (plus, minus, multiply) -composite. But again you get clipping after each step.

Also, if you had a complex dataype image (real part and imaginary part, i.e. 64-bit), then you could consider doing Fourier transform processing so as to deblur images that suffer from lens defocus and motion blur and also do filtering such as noise removal in the frequency domain. Also doing image correlation matching is much easier in the fourier (frequency) domain. Of course new functions would need to be written to do many of these things. So I know this is much further in the future.

However, with the floating point image datatype, you have many of the required functions built in -fx and -evaluate could be expanded to more of these functions if it were modified so as not to clip after each operation if the datatype is float.

I hope you will consider this suggestion.

Thanks for listening

Fred Weinhaus

[magick]

ImageMagick supports floating point now (at depths Q8, Q16, and Q32). When you build ImageMagick add the --enable-hdri option to the configure script command line. Some image formats that supports HDRI is EXR, PFM, TIFF, and of course MIFF.

We have support for Q64 but its experimental. Q64 would be ideal for complex pixel operations (real & imaginary pixel components).

[magick]

Do you have any Fourier transform code you could contribute? Or do you know of code we could use that has a liberal use license? If so we'll attempt to include support for Fourier transforms in ImageMagick in the near-future.

[seanburke1979]

Check out FFTW - http://www.fftw.org. Its gpl'ed, so I'm not sure that that is liberal enough, but it supports multidimensional real and complex FFTs. I have used it with IM for some basic feature recognition. It is very fast.


Sean

[seanburke1979]

Incidentally, if there is something I can write to expedite FFT inclusion, send me an email or post a reply with whatever you need.

Sean

[magick]

To expedite, write the interface to the FFT library and post your work here. Otherwise we have a to-do list about 1/2 mile long and we will eventually get to including FFT support into ImageMagick but currently do not have an ETA.

[fmw42]

With regard to HDRI and 32 bit format:

1. I have so far only used the binary releases. I am not that adept at compiling code.
2. I don't think even this will fix my problem. For example, I suspect that -evaluate used in multiple sequences in the same command line will still clip the results. I need to keep all the image results within the line to 32-bit and then at the end scale them from min, max to 0-255, for example to put the results back in a more exportable format. Furthermore, -evaluate is limited in its operations, there is no sin, cos atan, pow, ln, exp. etc. Please correct me if I am wrong.
3. Using -compose someoption -composite is also limited in what it can do as functions - add, multiply, subtract basically. There is no min, max, atan2, divide, etc. Please correct me if I am wrong.
4. Perhaps at some time you should create binaries for each Q. I am on a Mac OS-X, for example.

With regard to the fourier transformations, I am sorry but I don't have that myself nor am I much of a programmer. I can do scripting. But I am sure there are lots of code out there as open source. For example, I would not be surpised if it was not in the book that I believe is titled, "Mathematical Methods in C" and another one like it in C+. What I do have is lots of experience doing image processing in the frequency domain for removing motion blur, lens defocus, noise, and image to image correlation matching. I certainly can advise on how we implemented many of these things in the TRW system that used for many years and how our functions worked and how the images were laid out.

[magick]

HDRI does not clip/clamp the pixel values. If an algorithm produces values outside the normal range of 0 .. QuantumRange they remain untouched (e.g. a red component of -0.91723).

We may include an HDRI binary release in the future but not for the next release, 6.3.5-9.

[erik]

With regard to HDRI and 32 bit format:

1. I have so far only used the binary releases. I am not that adept at compiling code.

+1
I would be very grateful if there was a windows 32 bit enabled binary realease...

best regards
Erik

[fmw42]

I hope that I have not come accross too strongly with my suggestions for floating point and complex datatypes. These were only suggestions. I understand that you at IM have a long list of things to do. However, I was trying to point out where I think IM could profit by expanding its suite, namely, 1) floating point processing (with more functions in -evaluate and -compose), 2) complex datatype processing, and 3) multispectral (more channels than 3 or 4).

With regard to the FFT, the algorithm that was used at TRW was implemented from the original paper by Cooley and Tukey, 1965, Math. of Comp, Vol 19, pp297-301, "Algorithm For Machine Calculation of Complex Fourier Series". Unfortunately, I do not have access to that code.

There is another good book - by Brigham, 1988, "FFT and Its Applications", by Prentice Hall.

Also see

Smith: http://ccrma.stanford.edu/~jos/mdft

There are also references at:

http://en.wikipedia.org/wiki/Cooley-Tukey_FFT_algorithm

and at

http://faculty.prairiestate.edu/skifowit/fft/

With regard to floating point and complex processing functions, I have all the TRW User Guide pages that would explain some of the image formats as we used them and many of the functions for doing meaningful processing. I would be happy to send them to Anthony Thyssen or whoever is appropriate at IM development. And I would be happy to advise as best I can. For example due to symmetry in the FFT domain, we only use the bottom half of the FFT transformed resulting image with the zero frequency point at the top center of the fft image.

I have been helping Anthony to develop the Elliptical Weighted Average resampling for the -distort (perspective) case and have developed a script that could be used as a front end for specifying the perspective as 3 orientation angles. I have also developed several other scripts that are available on my IM web page at http://www.fmwconcepts.com/imagemagick/index.html. These add new capabilities by building upon your basic functionality.

Please let me know if you would like to discuss this further or want any of this material.

Although I am not a "real" software engineer, I do have lots of practical experience and can scripts some and would like to help as best I can.

Fred Weinhaus

[magick]

We have added this discussion to our to-do list. We may be getting back to you for help when we revisit Fourier transforms and/or the other topics you discuss. Currently ImageMagick supports up to 5 channels. The pixel cache has support for an unlimited number of channels-- its just never been a priority. We'll get to adding support for additional channels sometime in the future.

[fmw42]

Thanks for your consideration and adding these to your discussions and (I am sure) long list of activities for future enhancements.