Convolution based regtest comparison

On Sat, Jul 27, 2024 at 6:56 PM Werner LEMBERG <[email protected]> wrote:

>
> I've posted a question on StackExchange, searching for a better
> regtest comparison algorithm
>
>
> https://computergraphics.stackexchange.com/questions/14143/search-for-special-image-difference-metric
>
>
I've been busy with Life over the last year, but this question never really
left my mind.  This weekend I had some time and inspiration to prototype
things.

My idea is that we could represent the image diff as a signed image (+1 is
pixel added, -1 pixel removed), and then do a convolution with increasing
window sizes, and use these smoothed images to base a difference metric,
like

 sum_x sum_y (|diff(x,y)| + sum_{r=0..} |smooth_r (x, y)|)

where smooth_r is a convolution of the original diff image with window (1+2
* 2^r)

So, if a symbol is moved, as soon as the window encompasses the difference,
it is canceled out. Thus, small movements only contribute to the
measurements in the unsmoothed image, while big movements also contribute
in the smoothed images. This has the effect of making small movements less
important than big changes.  (See the "shift" examples attached).

I prototyped this at https://github.com/hanwen/lilypond-ci/ (see top 3
commits). I was wondering if this would be an improvement over MAE. I
didn't look around for test data; if someone can point me to commits that
yield interesting image diffs, that would be helpful.

Sample output attached.

Han-Wen

ps. Running test-baseline yields the following complaint. (I am using
-dbackend=cairo). Does this ring a bell for anyone?

lilypond-book.py (GNU LilyPond) dev
Reading
`/home/hanwen/vc/lilypond/input/regression/include-path-modification/out-test/collated-files.tely'
Running texi2pdf on file /tmp/tmpy7ttgyo0.texi to detect default page
settings.

Dissecting...
Writing snippets...
Processing...
Changing directory to '/home/hanwen/vc/lilypond/out/lybook-testdb'
Processing /home/hanwen/vc/lilypond/out/lybook-testdb/
snippet-names-b33d6dd9e734ceab923fa0ec509ee35c.ly
Command '/home/hanwen/vc/lilypond/out/bin/lilypond \
-I /home/hanwen/vc/lilypond/input/regression/include-path-modification
-dseparate-page-formats=ps -djob-count=1 -dseparate-log-files
-ddeterministic -dinclude-eps-fonts -dgs-load-fonts --header=texidoc
-dcheck-internal-types -danti-alias-factor=1 -dbackend=cairo -I
 "/home/hanwen/vc/lilypond/input/regression/include-path-modification"  -I
 "."  -I
 "/home/hanwen/vc/lilypond/input/regression/include-path-modification"
-daux-files  -dread-file-list -dno-strip-output-dir -dtall-page-formats=eps
  "/home/hanwen/vc/lilypond/out/lybook-testdb/
snippet-names-b33d6dd9e734ceab923fa0ec509ee35c.ly"' returned non-zero exit
status 1.
make[3]: *** [../../../make/ly-rules.make:36: out-test/collated-files.texi]
Error 1
make[3]: Leaving directory
'/home/hanwen/vc/lilypond/input/regression/include-path-modification'
make[2]: *** [../../../make/lysdoc-targets.make:21: lysdoc-test] Error 2

-- 
Han-Wen Nienhuys - [email protected] - http://www.xs4all.nl/~hanwen