Re: [Gimp-developer] deriving transform by comparing image before/after edit

Thanks Adalbert,

I don't want to reverse the transformation, just characterize the 
forward transformation enough so that I can train neural nets to suggest 
one I like when shown any photo. You might say I want to use my 
before/afters to derive a filter for each photo, then train nets with 
large numbers of

    input_pic_i -> filter_i

so the net can learn to improvise a new filter for each new pic to my 
taste.

As someone just put it on reddit:

    Params = f(jpeg, histogram) [f() is a neural net]

    and then for each pixel in the jpeg: rgb_transformed = engine(rgb,
    Params)

Alternatively, I could try having nets predict the output pic's pixels 
directly, but it seems maybe too big for consumer GPU's.

Interface-wise, a plugin would 'watch' you edit and give you suggestions 
if you asked.

Here's an example of how neural nets can pick up my taste in pairing photos:

    http://phobrain.com/pr/home/siagal.html

Bill


On 12/28/20 1:21 AM, Adalbert Hanßen wrote:
> Bill,
>
> unfortunately the result of a convolution operation can not always be
> inverted. The most simple example is a convolution matrix (=folding
> matrix) with all zeroes. It definitely wipes the whole image and that
> can not be undone!
>
> Unfortunately, the situation is such that the result of a matrix
> operation almost never can be inverted. Here an explanation with a
> little bit more reference to mathematics, leaving out the details of not
> square sized pictures and convolution matrices like those in Gimp, which
> only have up to 5*5 entries (think of them as being larger, expanded
> with all zeroes around it to all sides to pad them to the image size):
>
> Let M denote the Matrix and FT(M) the fourier transform of it (expanded
> to the image size). The Fourier transform is a terribly complicated
> thing and in fact, the result of a real valued picture becomes a complex
> valued picture, i.e. two pictures: one for the real part and another one
> for the imaginary one. Fortunately in mathematics (but not in Gimp)
> there also is an inverse Fourier transform which is as complicated as
> the FT itself, but once one has a program for FT, one also has one for
> the inverse FT. Of course, the FT and its inverse operate on complex
> valued "pictures", but you normally only look onto the real part of them
> in the non-transformed image. (The FT image is often referred to as in
> the frequency range: small and "high frequency" details in the picture
> become large in the FT and vice versa, it is all really "convolved", as
> one may say in the double meanig of this word).
>
> Let P denote the picture and FT(P) the fourier transform of the picture.
>
> Let M#P denote the convolution of M and P
>
> Then FT(M#P) = FT(M)*FT(P) (point-wise multipliaction)
>
> If FT(M) has entries with zeroes in it, these zeroes will cancel their
> respective pixels in FT(P). This of course means that FT(M#P) can no
> longer be FT(P). Unfortunatly almost all FT(M) would have zeroes
> somewhere, if the convolution matrix would have image size.
>
> The matrix convolution can be computed very effectively and it is quite
> powerful, especially for small matrices. Fortunately it now works in
> Gimp 2.10.22 (it did not work in earlier versions of 2.10) and
> fortunately now we can load and store the convolution matrices.
>
> I frequently convolve scans of documents with something like this
>
> 0    0    0    0    0
>
> 0   -1   -1   -1   0
>
> 0   -1    c   -1   0
>
> 0   -1   -1   -1   0
>
> 0    0    0    0    0
>
> wherer c ins the centre-weight, a number which is larger than the
> negative value of the sum of all other (negative) weights, e.g. c
> somewhere in the range 9...12.
>
> This matrix convolution subtracts from each pixel a weighted mean of the
> surrounding ones. The purpose of such a filter is to remove background
> like shadows or stains from the document before I apply OCR to it
> (that's why I do this operation). By choosing c larger than the negative
> sum of the surrounding ones, you can give the original picture more
> emphasis than to the reduction of the background.
>
> My convolution filter has some "differentiating property" which lets it
> emphasize noise in the picture. In order to mitigate that, one can first
> apply some gaussian filter. Another way is to integrate the the blurring
> (local average) into the convolution matrix, i.e. simultaneously do some
> kind of smoothing for the centre, for example
>
> -1  -1  -1  -1  -1
>
> -1    c   c   c  -1
>
> -1    c   c   c  -1
>
> -1    c   c   c  -1
>
> -1  -1  -1  -1  -1
>
> with some appropriate centre weight such that the sum of the c's ie a
> bit larger that the negative sum of the -1's around. The centre part is
> a box convolution which makes the image unsharp. (It would be nice if
> one could change all the centre- weights in one place, but once you
> think about that, you trigger the desire to shape the centre weights
> like a bell shaped function, which comes closer to gaussian unsharpness.
> Indeed, Gaussian unsharpness can almost be approximted by several times
> folding with a box function with only positive values).
>
> Of course it would be great, if the convolution matrix could be made
> larger than 5*5 for such folding matrices to remove background.
>
> For removing stains and shadows, it would also be beneficial if one
> could apply some gray level transformation to each pixel of the image
> (e.g. given by a polynominal) before the pixel value is mutiplied with
> the matrix entry in the convolution operation and to apply some other
> gray  level transformation (one which mostly anihilates the first one)
> on the way back before the convolution image is shown.
>
> In addition I would whish me something where I could apply two
> convolution matrices to an image: the first one acting like now, but the
> second one just to compute some "comparison value" which is used to set
> the pixel to "black" if the outcome of the first mentioned convolution
> is greater or equal to the outcome of the second convolution. Think of
> the second convolution going to some "alpha channel".
>
> But it would become necessary to be able to store and reload all those
> matrices and the coefficients of the gray level transformations and the
> coefficients of the comparison operation with that second convolution 9n
> norder to be usefull.  Great things could be done with Gimp if there
> would be some easy to perate playground for such image manipulations!
>
> Try my background removal operation with the appended example file,
> which is just a small part from an image taken from a newspaper. Play
> with it with different values of the centre weight. Of course, it would
> be best to first reduce the immage to a grayvalue-only image, i.e.
> without colours.
>
> Adalbert
>
>
> Am 27.12.20 um 07:52 schrieb Bill Ross via gimp-developer-list:
>> Given a pair of before/after jpeg photos edited with global-effect
>> commands (vs. operations on selected areas), is it possible to derive
>> transformation matrix/matrices that reproduce the result from the
>> original? Presumably by iterating over all the pixels in a 1:1 mapping.
>>
>> My hope is to train neural nets to predict the matrix operation(s)
>> required. Example:
>>
>> http://phobrain.com/pr/home/gallery/pair_vert_manual_9_2845x2.jpg
>>
>> Thanks,
>>
>> Bill
-- 
Phobrain.com
_______________________________________________
gimp-developer-list mailing list
List address:    [email protected]
List membership: https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list