Re: [linux-safety] [ELISA Development Process WG] [PATCH] mm: vmscan: provide a change to the development-process group

> -----Original Message-----
> From: [email protected] <development-
> [email protected]> On Behalf Of Lukas Bulwahn
> Sent: Thursday, September 17, 2020 5:44 PM
> To: Paoloni, Gabriele <[email protected]>
> Cc: Lukas Bulwahn <[email protected]>; linux-
> [email protected]; [email protected]
> Subject: Re: [linux-safety] [ELISA Development Process WG] [PATCH] mm:
> vmscan: provide a change to the development-process group
> 
> 
> 
> On Thu, 17 Sep 2020, Paoloni, Gabriele wrote:
> 
> > > -----Original Message-----
> > > From: [email protected] <development-
> > > [email protected]> On Behalf Of Lukas Bulwahn
> > > Sent: Thursday, September 17, 2020 10:44 AM
> > > To: [email protected]
> > > Cc: [email protected]; Lukas Bulwahn
> > > <[email protected]>
> > > Subject: [ELISA Development Process WG] [PATCH] mm: vmscan: provide
> a
> > > change to the development-process group
> > >
> > > I think this change is needed for safety, whatever that might mean to
> you.
> > >
> > > I am unqualified to really make a change here, as I have no clue what this
> > > code does, nor what my change does, but sure, the testing and
> verification
> > > reference process can now point out the required next steps in the
> > > reference process to test this code and code change.
> > >
> > > Good luck :)
> > >
> > > Not intended for distribution to the general kernel mailing lists.
> > >
> > > Signed-off-by: Lukas Bulwahn <[email protected]>
> > > ---
> > > I would like to submit such a patch, what do I need to do according to
> > > the expected testing and verification recommendations for safety-
> related
> > > systems?
> >
> > Probably the problem is not just limited to submitting patches, but I think it
> > is a good starting point.
> >
> > >
> > > Please help me. What do I need to compile, what test do I need to run,
> > > which verification tool do I need to employ for this change?
> >
> > Right. I have tried to reformulate the problem as "ok I am looking at or I am
> > trying to change one or more code lines, so I would like to understand if
> such
> > code lines are tested today and how".
> > Right now if we had a s maintained structural coverage report we could tell
> > if the impacted code lines are covered and by which tests...I could not find
> it
> > so I guess that so far we are missing it....am I wrong?
> >
> > As next step I went to look at the function wrapping the impacted code line
> > that in this case is kswapd. I could not find it tested in the Kunit framework.
> >
> > As next steps then I would run the available tests in
> > https://elixir.bootlin.com/linux/latest/source/tools/testing/selftests
> > in conjunction with GCOV trying to figure out if that line is covered already
> > somehow.
> >
> > There are probably smarter, better, faster ways to elaborate on this...so
> > here I just put down what I would do in order to figure this out...
> >
> 
> Gab, these are GREAT ideas and approaches. So, do we intend to provide
> methods, tools and documentation to really do these things you suggest?
> 
> That makes the intensions more tangible to me; if we try to find out
> how we and anyone else can get that information in some 'easy' way.
> 
> You can imagine that for a single-line change, you are suggesting probably
> quite some few days of work of investigation to get this information,
> right?

Unless together with the test frameworks in Linux we start maintaining a
corresponding structural coverage report that would instantly map the
affected code lines against the corresponding test.

Then reading the corresponding test we can figure out if the tests are still
valid (i.e. - black box - have I changed any behavior with my patch compared
to the expected result of the tests? or - white box - am I able to reach my
new change using the current test suites?)

My feeling is that we just need to get started putting together a more
structured test framework and report and them the problem as presented
by this patch would be very straightforward

Thanks
Gab

> 
> 
> Lukas
> 
> 
> 
> 

---------------------------------------------------------------------
INTEL CORPORATION ITALIA S.p.A. con unico socio
Sede: Milanofiori Palazzo E 4 
CAP 20094 Assago (MI)
Capitale Sociale Euro 104.000,00 interamente versato
Partita I.V.A. e Codice Fiscale  04236760155
Repertorio Economico Amministrativo n. 997124 
Registro delle Imprese di Milano nr. 183983/5281/33
Soggetta ad attivita' di direzione e coordinamento di 
INTEL CORPORATION, USA

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.