Hi. I'm the original reporter of this issue, and the test case I gave in my bug was actually a simplified version of the real code. The actual inspiration for the report was this code: https://github.com/mapbox/tilelive-mapn ... ert.js#L22
which is part of the test suite for some software I was packaging.
It runs the compare tool (with a metric switch) to compare two images, and then first checks the exit code (to see if the tool worked) and then if that is zero is looks at the metric that was output to see if the two images matched.
The metric in use was PSNR and in the case I was dealing with the result was "inf" which is treated as a match, except that the new exit code of 1 was being treated as a failure. If the change in exit code is expected then that's fine, but I'll need to pass that information on to the authors of the test suite.