Re: Test Data for DigiKam

Le sam. 24 sept. 2022 à 23:19, Steven Robbins <[email protected]> a écrit :
>
> Hello,
>
> In the interests of transparency, I have to say that I am unsure what to do.
>
> My original impression was that the decision was made to separate test and
> code and that the KDE sysadmins would have guidance on this.  That didn't turn
> out as I had imagined.
>
> After digging around a bit, my impression now is that the 7GB size of
> digikam's git repo is the issue.  As far as I can tell, the only way to fix
> that is to prune and re-write the repo.  That's not something I am highly
> motivated to do.
>
> Finally, I worry that separating data from code is going to place enough of a
> barrier to running tests that fewer will do so.  Testing is fundamentally
> important to me so this aspect is troubling.
>
> I don't really understand why Digikam has a 7 GB repo -- when the source
> tarball amounts to 300 MB (500 MB when unpacked).  I would speculate that
> there were things added then deleted in the past?
>
> Further, I wonder whether this can be handled more simply.  Atlassian [1] has
> a few ideas -- and maybe a git shallow clone for developers would help?
> Probably coupled with using GIT-LFS **in the main repo** for the large files.

Hi steven,

We have a more simplest and elegant solution. I just remember an old
feature that we have implemented in the past, when the project was cut
in parts for the plugins.

Look in this file :

https://invent.kde.org/graphics/digikam/-/blob/master/.gitslave

It includes the documentation git repository which can be checked out
with git at the same time as source code. Doc is also a very huge
repo, this is why we use a separated place to store files :

https://invent.kde.org/documentation/digikam-doc

To checkout source and doc, developers will use the "download-repos"
script, which is called "gits" script. This last one will parse the
.gitslave configuration file to process multiple checkouts locally.
Note that developers need to patch previously the .bashrc
configuration with "export GITSLAVE=.gitslave" definition, else it
will not work.

https://invent.kde.org/graphics/digikam/-/blob/master/download-repos
https://invent.kde.org/graphics/digikam/-/blob/master/gits

If we store the data test files in the famous dedicated repository :

https://invent.kde.org/graphics/digikam-test-data

And if we append the url to .gitslave, this repo will be downloaded
automatically with source code.

If we choose this solution, i propose to check in cmake script if the
test data is present, and in this case to enable unit tests which use
this data.

The advantage is to not run all the unit tests in the CI which is
limited in all cases to one hour. The disadvantage is to run unit
tests in developers computers for these unit-tests using data files.

What do you think about this proposal ?

My best

Gilles





>
> [1] https://www.atlassian.com/git/tutorials/big-repositories
>
> So that's where I am.  I'd like to hear your thoughts.
>
> -Steve