Re: accumulating projects in kate lead to excessive kate startup time due to git

On Monday, 20 June 2022 17:53:42 CEST Thiago Macieira wrote:
> On Monday, 20 June 2022 00:51:55 PDT Milian Wolff wrote:
> > I wonder, is there a reason why QProcess is not leveraging posix_spawn?
> > That should help with this exact problem if I'm not mistaken, see e.g.
> > [1]. But then again, this is certainly not an area where I'm proficient
> > in. What's your take?
> 
> For several reasons.
> 
> The most important of which is that posix_spawn is implemented using fork()
> +execve() inside glibc, so it doesn't improve anything.

This is not true anymore, I believe. See e.g. [1] which stats:

> glibc's posix_spawn was rewritten in 2016 to use clone() with CLONE_VM

[1]: https://gist.github.com/nicowilliams/a8a07b0fc75df05f684c23c18d7db234?
permalink_comment_id=2373305#gistcomment-2373305

> It just moves the
> code doing fork() down into a large function whose implementation
> duplicates what we have, poorly.

Can you revisit this assessment please in light of more modern 
implementations?

> The other visible reason is the fact that QProcess allows you to do
> something in the child process before execve() as part of its API. So we
> need to be able to control the forking.

What exactly does this mean - which part of the API of QProcess is not 
portable to posix_spawn?

> I thought of using vfork() instead, but that has a lot of extra restrictions
> too. QProcess would probably have to revert back to regular fork if the
> user set a child process modifier function, the same way it reverts to
> regular fork() instead of clone() on Linux. But it might help because it
> does not mark the child process's page as COW to the parent.

What is a "process modifier function"? I can find [2] but that seems windows 
specific to me?

[2]: https://doc.qt.io/qt-5/qprocess.html#setCreateProcessArgumentsModifier

> > ```
> > 
> >     assert(!system_has_forkfd());
> > 
> > ```
> > 
> > Why is that? I would really like to learn a bit more on this. For
> > starters,
> > I expanded my trivial little snippet to get a similar subprocess behavior
> > with posix_spawn (obviously lacking error reporting and such), which shows
> > _very_ promising behavior on my machine:
> > 
> > https://invent.kde.org/-/snippets/2241
> 
> I stopped working on spawnfd after QNX got fork(). I don't remember why that
> assertion is there.
> 
> I can make a patch to use vfork. If you can test that, it would be a good
> data point too.

I'll try that out, you'll hear back from me once I have some data to share.

Thanks a lot for your insight already Thiago, much appreciated!

-- 
Milian Wolff
http://milianw.de