Audio equalizer for images?

Discuss digital image processing techniques and algorithms. We encourage its application to ImageMagick but you can discuss any software solutions here.
Post Reply
JoaCHIP
Posts: 36
Joined: 2004-12-21T04:22:18-07:00
Location: Copenhagen

Audio equalizer for images?

Post by JoaCHIP » 2014-06-19T05:25:08-07:00

Maybe this already exists, and I'm just really bad at using google, but i'll share my little idea anyway.

I deal with audio a lot, and have even coded numerous plugins for that. I've often noticed the similarity between the algorithms used there and what you see in image processing. The most simple example is that "blur" resembles a lowpass filter (attenuates treble) and sharpen resembles boosting the amount of treble.

However, it seems to me that in audio you are presented with much more sophisticated options. A parametric equalizer typically has multiple "bands", and each band has up to three different options: 1) Boost/cut, 2) Frequency, 3) Bandwidth.

Let's say we took this algorithm and converted it into 2D. We would be able to apply EQ to an image, doing some of the tricks that "gaussian blur" and "unsharpen mask" in photoshop currently offers, but in a way more customizable way. Here's a few examples of what would happen, if someone had the time to implement this craziness:

Example 1: Choose to boost by +100% at a high frequency (near nyquist) and a medium bandwidth to reach something that resembles photoshop "sharpen".

Example 2: Choose to boost by -100% at a high frequency (near nyquist) and a medium bandwidth to reach something that resembles photoshop "blur".

Example 3: Choose to boost by +100% at a medium frequency and a wide bandwidth to increase the amount of middle-sized detail in the image (causing a somewhat dreamy old-school looking chemical effect.)

Using several bands of boosts and cuts, you could probably reach all sorts of unusual looking effects. I could also see a few situations where this could be used to solve various problems with the image quality. I've been tempted to trying to write some code to do this, but my coding skills are somewhat mediocre. Has anyone had similar thoughts? Made experiments? Any input is welcome here.

PS: This concept does not really have anything to do with "histogram equalization" btw.

snibgo
Posts: 9378
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Audio equalizer for images?

Post by snibgo » 2014-06-19T07:16:55-07:00

I suspect you can already do all this. The question is simply working out the equivalents. I'd be interested to hear more.

For example, I often find the difference between an image and its blur, then blur the result. I think this is a smoothed rectified high-pass filter.

Today I'm doing lots of "differences between two blurs", which would be a mid-pass filter.

You can interpolate between an image and its blur, to get an image in between. But if you extrapolate in the opposite direction, the image sharpens -- boosting the frequencies that the blur reduced.

I once needed a quick-and-dirty method of finding one sound clip within another. I converted them both to images and used ImageMagick "-subimage-search".
snibgo's IM pages: im.snibgo.com

JoaCHIP
Posts: 36
Joined: 2004-12-21T04:22:18-07:00
Location: Copenhagen

Re: Audio equalizer for images?

Post by JoaCHIP » 2014-11-14T02:54:43-07:00

Your approach would indeed give a highpass filter, but it gets fairly cumbersome (not to mention slow) if you want to achieve more complex equalization this way. I was thinking of a full-featured parametric equalizer here, which means the following controls (for a 1-band version, that is):

A) Boost or attenuate: An amount from e.g. -1.00 ... +1.00
B) Frequency/size: This could be either a value from 0.0001 to 1.00, ranging from super large blur/sharpen size to the smallest possible blur/sharpen size at nyquist limit, or simply a directly specified size in pixels (would still have to be a floating point value though).
C) Bandwidth in octaves: Small values mean that only a very narrow frequency band is affected (watch out for "ringing" effects), large values would affect a wider frequency area around the frequency chosen by parameter B.

Parameter C would be extremely difficult to achieve using snibgo's method, which would also pose other problems such as an increased risk of highlight/black clipping between steps, decreased precision because of the many iterations needed (Q16 would be quite important here) etc. - That's why I think a dedicated algorithm - based on what is known as a "parametric EQ" in the audio world - would be the way to go. You could really do some better sharpening / blurring effects this way, and also some interesting retro-effects. I also think this could give ImageMagick yet another advantage over the competitors.

I'll see if I can construct some examples if anyone is interested?

snibgo
Posts: 9378
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Audio equalizer for images?

Post by snibgo » 2014-11-14T11:11:31-07:00

I see what you mean about bandwidth. From what little I know about frequency filtering, there is also a shape parameter -- eg long smooth cutoff, or a sharp cutoff that can introduce ringing. Is that right? I'd be interested in seeing the visual results.

IM can be compiled as Q8 to Q64, with or without floating point. Images are stored in-memory in that format. An iterating algorithm may need floating-point.

I think this could be a very worthwhile addition to IM.

If you compile IM yourself, adding "-process modules" is easy. A module can take whatever arguments you like, and you can do this without changing the factory IM code. However, the interface is at MagickCore level, not MagicWand. Many examples on my page http://im.snibgo.com/customim.htm
snibgo's IM pages: im.snibgo.com

JoaCHIP
Posts: 36
Joined: 2004-12-21T04:22:18-07:00
Location: Copenhagen

Re: Audio equalizer for images?

Post by JoaCHIP » 2014-11-17T13:56:59-07:00

I've made a quick example which only shows a part of what you would be able to do with this, but because I've been using normal image processing algorithms here, I cannot easily recreate steep filters. The "slope" of these filters is most likely 6dB/octave, because they are probably 1. order filter (1-pole filters). For the following filters I'll use two "EQ bands", and on the first and second examples I'll reverse what is boosted and what is attenuated to show some possible variations.

Band A: Boost: -6 dB, frequency = 1.33 pixels, bandwidth: medium wide
Band B: Boost: +6 dB, frequency = 6.67 pixels, bandwidth: medium wide
Image

Band A: Boost: +6 dB, frequency = 1.33 pixels, bandwidth: medium wide
Band B: Boost: -6 dB, frequency = 6.67 pixels, bandwidth: medium wide
Image

Now you are right that "ringing" may occur when designing more unusual filters, especially very steep ones (low bandwidth). For the above examples I've been unable to adjust the bandwidth, because I'm just using Photoshop to simulate these effects. But with a dedicated filter more fancy effects could even be achieved.

JoaCHIP
Posts: 36
Joined: 2004-12-21T04:22:18-07:00
Location: Copenhagen

Re: Audio equalizer for images?

Post by JoaCHIP » 2014-11-17T14:17:31-07:00

Two more to give some more examples of the effect, now that I'm at it... :

Band A: Boost: -6 dB, frequency = 50 pixels, bandwidth: medium wide
Band B: Boost: +6 dB, frequency = 20 pixels, bandwidth: medium wide
Band C: Boost: -6 dB, frequency = 6 pixels, bandwidth: medium wide
Image

Band A: Boost: -6 dB, frequency = 50 pixels, bandwidth: medium wide
Band B: Boost: +6 dB, frequency = 20 pixels, bandwidth: medium wide
Band C: Boost: -6 dB, frequency = 6 pixels, bandwidth: medium wide
Image

snibgo
Posts: 9378
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Audio equalizer for images?

Post by snibgo » 2014-11-17T14:23:09-07:00

Coo, excellent! boost20_attentuate4 has a lovely "ethereal" look (when downloaded and viewed full-size). Can you put up a version with no attenuation/boost?
snibgo's IM pages: im.snibgo.com

JoaCHIP
Posts: 36
Joined: 2004-12-21T04:22:18-07:00
Location: Copenhagen

Re: Audio equalizer for images?

Post by JoaCHIP » 2014-11-23T11:44:32-07:00

Sorry, I almost forgot to put up the original...

Image
PS: This image is free for use by anyone, so be my guest!

Juce
Posts: 12
Joined: 2011-03-22T07:06:04-07:00
Authentication code: 8675308
Location: Finland

Re: Audio equalizer for images?

Post by Juce » 2015-02-05T03:14:43-07:00

I did few tests with a raised-cosine filter:

Image

A0 = 1.0;
f0L = 0.02; f0H = 0.05;
A1 = 0.5;
f1L = 0.1; f1H = 0.2;
A2 = 1.0;
Image

A0 = 1.0;
f0L = 0.00; f0H = 0.05;
A1 = 2.0;
f1L = 0.1; f1H = 1.0;
A2 = 0.1;
Image

A0 = 0.0;
f0L = 0.00; f0H = 0.01;
A1 = 1.0;
f1L = 0.1; f1H = 0.5;
A2 = 1.0;
Image

A0 = 1.0;
f0L = 0.0; f0H = 0.25;
A1 = 2.0;
f1L = 1.0; f1H = 1.0;
A2 = 2.0;
Image

Drazick
Posts: 12
Joined: 2014-04-27T10:33:50-07:00
Authentication code: 6789

Re: Audio equalizer for images?

Post by Drazick » 2015-08-08T11:59:47-07:00

What about doing it using Wavelets / Image Pyramid?

JoaCHIP
Posts: 36
Joined: 2004-12-21T04:22:18-07:00
Location: Copenhagen

Re: Audio equalizer for images?

Post by JoaCHIP » 2016-07-18T10:43:16-07:00

@Juce: Wow, those are great. Only the third image seems to be affected by the X-Y orientation of the algorithm, but the other examples are great.

Having to type in coefficients directly might scare off most users (and me too, most likely). But I guess it would be a similar operation as calculating coefficients for an audio filter, and creating a UI would resemble creating a UI for a parametriq EQ.

How did you do this? Is there a way in ImageMagick, or did you code this?

User avatar
fmw42
Posts: 22079
Joined: 2007-07-02T17:14:51-07:00
Location: Sunnyvale, California, USA

Re: Audio equalizer for images?

Post by fmw42 » 2016-07-18T10:58:33-07:00

You likely can achieve this by doing highpass, lowpass and bandpass filtering in the frequency domain using FFT processing. Imagemagick has FFT processing. See http://www.fmwconcepts.com/imagemagick/ ... urier.html

You would modulate the magnitude image by a mask that would achieve your gain and attenuation. The mask would be a radial gradient modified by some look up table to increase or decrease the value as a function of distance from the center. The center area is low frequencies and the further out are higher frequencies. The mask would be multiplied by the magnitude image. To get gains greater than 1, you would need to do the processing in HDRI mode.

JoaCHIP
Posts: 36
Joined: 2004-12-21T04:22:18-07:00
Location: Copenhagen

Re: Audio equalizer for images?

Post by JoaCHIP » 2016-07-26T04:56:41-07:00

fmw42: That's a good idea actually. I could probably just script some stuff on top of that to give me some more intuitive controls, and then my script would generate the necessary imagemagick commands to do FFT --> apply boosts/attenuations (frequency curve response) --> inverseFFT.

I'll report back if I get something to work.

User avatar
fmw42
Posts: 22079
Joined: 2007-07-02T17:14:51-07:00
Location: Sunnyvale, California, USA

Re: Audio equalizer for images?

Post by fmw42 » 2016-07-26T10:37:51-07:00

Here are some simple examples using FFT processing.

Input:
Image

Lowpass filter:

Code: Select all

size=`convert cafe.jpg -format "%wx%h" info:`
dim=`convert cafe.jpg -format "%[fx:max(w,h)]" info:`
convert cafe.jpg -fft \
\( -size ${dim}x${dim} radial-gradient: -evaluate pow 3 \) \
\( -clone 0 -clone 2 -compose multiply -composite \) \
-delete 0,2 +swap -ift -crop ${size}+0+0 +repage cafe_lowpass3.jpg
Image

Radial Profile (across diameter of FFT magnitude):
Image


Highboost filter1 (cutoff is at 80% out along radius to avoid amplifying noise; blur is used to avoid sharp breaks to avoid ringing):

Code: Select all

size=`convert cafe.jpg -format "%wx%h" info:`
dim=`convert cafe.jpg -format "%[fx:max(w,h)]" info:`
im6937hdri convert cafe.jpg -fft \
\( -size ${dim}x${dim} xc:white \
\( -size ${dim}x${dim} radial-gradient: -negate \
-black-threshold 25% -white-threshold 80% -fill black -opaque white \
-fill white +opaque black -blur 0x5 -auto-level \) \
-define compose:clamp=off -compose plus -composite \) \
\( -clone 0 -clone 2 -define compose:clamp=off -compose multiply -composite \) \
-delete 0,2 +swap -ift -crop ${size}+0+0 +repage cafe_highboost1.jpg
Image

Radial Profile (amplitude is 1 and bottom is at 1, so top is at 2):
Image


Highboost filter2 (twice the amplitude):

Code: Select all

size=`convert cafe.jpg -format "%wx%h" info:`
dim=`convert cafe.jpg -format "%[fx:max(w,h)]" info:`
im6937hdri convert cafe.jpg -fft \
\( -size ${dim}x${dim} xc:white \
\( -size ${dim}x${dim} radial-gradient: -negate \
-black-threshold 25% -white-threshold 80% -fill black -opaque white \
-fill white +opaque black -blur 0x5 -auto-level -evaluate multiply 2 \) \
-define compose:clamp=off -compose plus -composite \) \
\( -clone 0 -clone 2 -define compose:clamp=off -compose multiply -composite \) \
-delete 0,2 +swap -ift -crop ${size}+0+0 +repage cafe_highboost2.jpg
Image

Note: using a lookup table applied to the gradient would process faster than processing the gradient directly.

jmac698
Posts: 48
Joined: 2013-12-20T01:57:16-07:00
Authentication code: 6789

Re: Audio equalizer for images?

Post by jmac698 » 2016-09-28T17:46:55-07:00

Just wanted to mention, if you want an equalizer for images try http://www.avisynth.nl/users/vcmohan/FF ... Quiver.htm

Post Reply