Re: Moving the documentation to git: Current status

On Thu, 26 Dec 2019 at 17:09, Andreas Heigl <[email protected]> wrote:

> Hey everyone.
>
> A short update from my side.
>
> I've done some more research and testings and as far as I can see
> everything should now be ready to move the toolchain for the
> documentation from SVN to git.
>
> The actual rendering of the documentation works irrelevant of the used
> VCS. So what would need to change (after all the repos are created and
> added as remote on svngit.php.net) is the way the sourcecode is checked
> out on rsync.php.net as well as on docs.php.net. IMO the best would be
> to first change docs.php.net and when everything works out as expected
> modify rsync.php.net.
>
> Additionally there is http://doc.php.net/revcheck.php. Under
> https://github.com/phpdoctest/doc-base/blob/master/scripts/revcheck.php
> there is an adapted version that fetches the relevant information from
> git and displays them. Currently in a somewhat different UI but I assume
> that that is dependent on some header-file that I didn't include in my
> test-file. For a general view have a look at
>
> https://htmlpreview.github.io/?https://github.com/phpdoctest/meta/blob/master/revcheck.dummy.html
>
> The format for the Files-Section[1] has changed a bit due to the changed
> options. There are no Revision numbers any more but instead the number
> of commits to the english file as well as to the translated file are
> noted since the translated file was last based on the english file.
> Additionally the number of days that passed since the last revision of
> the english as well as the translated file are noted. In the end those
> metrics are more meaningfull than the SVN revision-numbers of the
> english and the translated file that are currently used.
>
> To calculate the number of revisions the datetime of the english
> revision is taken and the number of revisions to the translated file
> since that datetime is calculated.
>
> So from my side the followings steps are separating us from having the
> PHP-Documentation in git:
>
> * Create language repos for es, fr, it, ja, pl, pt_BR, ro, ru, tr, uk
> and zh on git.php.net (someone with access on
> https://wiki.php.net/systems/php-git1)
>

Done. Thanks Nikita.


> * Grant force-push access to a user that can push the modified commits
> from svngit.php.net (can be me, can be a dedicated user) (someone with
> karma)
>

Force-push is allowed for all repos except key branches on php-src (and the
playground
repo for testing).

We have a VCS account "phpdocbot" that can be used by automated systems
such as
this, rather than using your own VCS account. Please send me an SSH public
key, to
add to that account, that will be used on the svngit box for the pushes.  I
have granted
karma to the git repos for that account.


> * Add the remotes to the respective repos on svngit.php.net and set up
> automated forcepush (myself)
> * Switch source-code retrieval on docs.php.net from svn.php.net to
> git.php.net (someone with access according to
> https://wiki.php.net/systems/euk2)
>

I can do that.


> * Switch revcheck-generation from the current script to
> https://github.com/phpdoctest/doc-base/blob/master/scripts/revcheck.php
> (someone with access according to https://wiki.php.net/systems/euk2)
>

Anyone (with doc-base karma) can add the new script to doc-base (either as
a new script
or replacing the current one), it will get pulled down to the docs box.


> * Verify that everything works as expected (Documentation team. Who
> would that be? @salathe?)
>

I can do that.


> * Switch source-code retrieval for documentation on rsync.php.net from
> SVN to git (Someone with access according to
> https://wiki.php.net/systems/sc2)
>

And that.


> * Setup github mirroring and processes to merge merge-requests from
> github into the documentation git (Someone with access. Who would that
> be???)
>

Anyone with gitolite-admin access can enable github mirroring (including
myself, Nikita, Derick)
when everything is ready. For merging PRs, we should use the normal
workflow as with
all of the other git.php.net repos.


> * Rewrite setup-instructions for documentation and send email to docs
> teams regarding new process (myself with help from? @salathe?)
>

That sounds like something I should do.


> * Make SVN readonly (someone with access on
> https://wiki.php.net/systems/svn2)
>

Removing commit karma is better, I will do that gladly when the time comes.


> * Shutdown cronjobs on svngit.php.net (myself)
>
> Important is that unless the SVN is switched to read-only mode *no* – I
> repeat: NO – contribution can come in via git/github. That would be
> overwritten by the force-push from svngit.php.net
>
> So from these 11 tasks that I identified here I can do 3. The other
> tasks need to be done by others (or someone needs to grant me the access
> so that I can do them)
>
> So from my Point of View the technical and logistical issues are solved
> to move to git. Now it's a matter of actually *doing* it.
>
> Or did I miss something?
>
> Looking forward to your feedback
>
> Cheers
>
> Andreas
>
> [1]
>
> https://htmlpreview.github.io/?https://github.com/phpdoctest/meta/blob/master/revcheck.dummy.html#files
>
>
> Am 23.12.19 um 19:03 schrieb Andreas Heigl:
> > Hi everyone.
> >
> > I have been a bit quiet lately due to work and family as well as
> > wrapping my head around different issues with the current approach to
> > moving the docs from SVN to git.
> >
> > But it looks like the first hurdle is done!
> >
> > What happened so far:
> >
> > The process has been established to move the documentation continuously
> > from SVN to git. For that Rasmus has set up a DO-Dropplet that every 2
> > hours fetches the latest Changes from the SVN repository and rebases
> > them into a git-repo using git-svn rebase. That is currently done for
> > all languages that have been changed within the last 2 years (which
> > means (in alphabetical order): de, en, es, fr, it, ja, pl, pt_BR, ro,
> > ru, tr, uk and zh).
> >
> > During that process a file is created and then updated that contains the
> > revision of the english base-file that each file is based on. That
> > information so far is kept inside the translated file in a comment with
> > the content "en-revision: [revision-number]". The first tries to replace
> > this revision number inside the file with the commit-hash were not
> > working as that meant modifying the file so that in a future rebase the
> > change was not known to git-svn and a rebase could not be done.
> > Therefore I moved that information out into a separate file. The
> > advantage is now, that the process of generating the statistics about
> > the translation status now only needs to read this single file instead
> > of each single file to retrieve the hash (or revision number) of the
> > english base-file.
> >
> > The next step would now be to automatically push those changes to the
> > corresponding repo at git.php.net. For that to work though there are two
> > things to be solved:
> >
> > 1. The git-repos for the languages es, fr, it, ja, pl, pt_BR, ro, ru,
> > tr, uk and zh need to be created.
> > 2. We need to decide how to continue with the repos for en and de. As
> > far as I understood Peter has created a shallow clone of the english
> > repository there. That repos master-branch would need to be overwritten
> > in the process as the git push would need to do a force-push as changes
> > to the branch can not automatically be merged. That would become far too
> > complex. Also the german repo seems to be updated somehow... How will
> > the process be here?
> >
> > As soon as that is solved and done, the background processes need to be
> > modified so that they use the git-repos as the single source of truth
> > and not the SVN repo. That means two different things mainly: The
> > building of the actual documentation and the creation of the meta-docs
> > that contain the information about the status of the different documents
> > in relation to the base-document.
> >
> > I will be working on those modifications in parallel so that – once the
> > todos are resolved – the process of moving the background processes can
> > be started ASAP.
> >
> > Once the background processes are modified and deployed, the SVN can be
> > switched to ReadOnly and all changes can go in via git.
> >
> > The workflow with git would change a bit in several ways from the
> > current workflow:
> >
> > * Every language lives in its own repository. So you would need to clone
> > two repositories: the english one and the one you want to translate.
> > Drawback, you need to clone multiple repositories if you want to
> > contribute to multiple languages, Advantage though is, that you only
> > need to clone one repository if you want to contribute to one language.
> > * The reference to the english base-file is not kept inside the
> > translated file any more but in a separate file. Remembering to modify
> > that file might be the most challenging part. Perhaps having a pre-push
> > hook, that checks that this special file is modified according to the
> > modified files can make sense.
> > * The new process uses git, so that contributions via a github mirror
> > are now possible so that the same processes can be used for
> > contributions via github PRs that are currently used for the
> > PHP-Sourcecode. So contributions from non-regulars need to be handled
> > differently. This raises the question who will create those processes
> when?
> >
> > Did I forget anything? Feel free to ask!
> >
> > Looking forward to your feedback.
> >
> > Cheers
> >
> > Andreas
> >
> > PS: The process is documented at
> > https://github.com/phpdoctest/meta/wiki/Current_Ideas
> > PPS: salathe: I'd be especially grateful for your feedback
> >
>
> --
>                                                               ,,,
>                                                              (o o)
> +---------------------------------------------------------ooO-(_)-Ooo-+
> | Andreas Heigl                                                       |
> | mailto:[email protected]                  N 50°22'59.5" E 08°23'58" |
> | http://andreas.heigl.org                       http://hei.gl/wiFKy7 |
> +---------------------------------------------------------------------+
> | http://hei.gl/root-ca                                               |
> +---------------------------------------------------------------------+
>
>