I have verified that the function is correct.
color component for 'Over' composition.
http://www.w3.org/TR/2002/WD-SVG11-2002 ... ml#SrcOver
Dc' = Sc + Dc(1 - Sa)
where Dc' Sc Dc are all pre-multiplyed by there respective alpha values.
However Dc' (result or combined image) is opaque, as is the background Dc color
that is Da' = 1 and Da = 1
As such, by replacing Dc' with Rc to make it easier, and explicitly specifying the alpha components, the function becomes
Rc = Sc*Sa + Dc*(1-Sa)
now we are trying to recover Sc with a known Sa Rc and Dc so..
Sc*Sa = Rc - Dc + Dc*Sa
Sc = Rc/Sa - Dc/Sa + Dc
Which is exactly the simplified FX formula above.
I would never have thought to do that if it wasn't for the way you did your simultaneous equations!
While trying to sleep last night the above formula kept swirling roun din my mind, and I wondered if I could also verify that the Sa determination was really the negated difference of the two overlay images (Rc1 and Rc2).
Now as the same overlay image was used we have
Sc = Rc1/Sa - Dc1/Sa + Dc1
Sc = Rc2/Sa = Dc2/Sa + Dc2
Rc1/Sa - Dc1/Sa + Dc1 = Rc2/Sa - Dc2/Sa + Dc2
divide by Sa
Rc1 - Dc1 + Dc1*Sa = Rc2 - Dc2 + Dc2*Sa
(Dc1-Dc2)*Sa = (Rc2 - Rc1) + (Dc1-Dc2)
Sa = (Rc2 - Rc1)/(Dc1-Dc2) + 1
and a final result of
Sa = 1 - (Rc1 - Rc2)/(Dc1-Dc2)
And that is a negated difference of Rc1 and Rc2, with a normalization multiplier 1/(Dc1-Dc2) applied before the negation.
The multiplier would have been determined by the difference range between fully-transparent and fully-opaque pixels,
But as you can see you could also determine it more directly too.
For a pair of white and black and black backgrounds Dc1,Dc2 = 0,1 in any order that just becomes...
Sa = 1 - |Rc1 - Rc2|
So this verifies that the difference of the two overlay images does indeed generate a perfect alpha mask. However Dc1 and Dc2 should be as different as posible, and the Rc1 and Rc2 values should also be as different as posible.
The formula also shows that it possible to recover the overlay from two images with known multi-colors backgrounds! Each pixel difference will then need to be normalized by dividing it by the difference between the original background used. Assuming of course that there is a good difference between all the pixels within the two original background images.