I sometimes get unexpectedly large values from "-metric phash". This happens when there is a colour-shift between images, and the image contains reddish pixels, so the Hue channel moments are very different. This will be because of the discontinuity of Hue at 0 = 100%. Perhaps Lab or Luv would be better than HCL.
I had not thought of that, but that may explain some issues I have seen recently in some work I am doing.
The original paper from which I got the idea use YCbCr. But I tested a number of colorspaces (YCbCr, LAB, HSI, OHTA), but found that HCLp seemed to give the best results overall for the (limited) test set of images that I used. However, I did not not test specifically for reddish issues. You may be right that it should use LAB or YCbCr for that reason.
I suspect it is not too hard to switch the HCLp colorspace to something other such as LAB or YCbCr (with a -define or new argument), if anyone is willing to put in the time to test in a beta. Magick would have to let us know how hard that would be.
Also if someone has a better or alternate hash code, it could be added to IM at some point or contributed.