Re: Interest inquery: kde4-nosemantic overlay

Steven J. Long posted on Tue, 16 Jul 2013 03:01:50 +0100 as excerpted:

> Duncan wrote:
>> Martin Vaeth posted:
>> > Does your framework manage the ebuilds in some overlay, or is it
>> > actually running tranparently during emerge (probably with a patched
>> > version of portage), allowing e.g. to change metadata without
>> > maintaining the ebuild in a separate local overlay?
>> > (I guess that it is the former, but your choice of the path
>> > /etc/portage/... suggests the latter)
>  
>> the framework I've setup is repo agnostic and would find the ebuilds in
>> whatever repo, main tree or active overlay, they appear in.
> 
>> I've occasionally been frustrated by this, but until this gentoo/kde
>> semantic-desktop policy change, particularly with epatch_user
>> eliminating the need for most of the ebuild edits I used to do, it was
>> just easier to do the manual hacking any time I needed to change an
>> actual ebuild.
> 
> Hmm why was a local overlay inappropriate? I thought you could mask
> packages from specific overlays (or am i wrong about that?)

I believe I may have misunderstood Martin's original comment intent.  The 
below might more accurately address the situation.  (Assuming I'm not 
getting too sleepy to think straight...)

I do have a local overlay, and use it for one-offs, where the upstream 
ebuild's likely to include the patch relatively quickly, or where it only 
applies to a specific version so an update is likely to invalidate the 
patch in any case.

But in the kde case that wouldn't work as the patches will need to be 
reapplied long-term -- no upstream inclusion, as I didn't want to deal 
with manually overlaying/patching every single revision bump, etc.

Actually, for kde I run the betas from the overlay, and (since sometime 
in 4.10) have been running (and generally rebuilding about twice a week) 
the live-branch ebuilds (4.10.49.9999, 4.11.49.9999) when upstream 
branches along with the first rc.  Particularly the live-branch ebuilds 
(and even more the live-trunk -9999 ebuilds, but I've not gotten /that/ 
brave yet) get updated in-place to adapt to upstream code changes as well 
as gentoo system changes, and I wanted my hassle-free application of my 
no-semantic patches until they no longer applied, at which point I wanted 
to know about it so I could redo them.

So applying the patches direct-in-repo made most sense for me, with 
updates overwriting my changes, and the patches then reapplied on top of 
the updates.

But making that an option's probably a good idea, agreed.

> Not that it matters, in that I'm glad you've gone down this path. I'd
> just like to know.
> 
> From what you've written, the first thing that springs to mind is
> /etc/portage/postsync.d/ which has q-reinitialize in it. I have that
> activated, and run eix-sync after emerge --sync in update, which takes
> care of overlays. Which answers why postsync isn't sufficient in and of
> itself. Hmm I guess that means that means qgrep et al aren't complete,
> which I've never noticed as I don't use overlays.

There's also the fact that my patches change dependencies -- that's what 
they're DESIGNED to do, after all, kill the semantic-desktop deps -- thus 
invalidating portage's metadata cache, so if emerge --regen isn't 
triggered afterward, every emerge invocation will take longer.

What my sync script (local version) basically does:

1) test-me: see if the tree partition is mounted and mount it if 
necessary.

2) setup-parms: grab niceness and parallel jobs from my portage 
configuration, etc.

3) emerge sync and layman sync (backgrounded in parallel, using wait).

4) ebuild-patch (that's my ebuild patching function, the interesting bit).

5) emerge $EJobs --regen

Then after the portage cache is updated, in background/parallel:

6) emerge --fetch (options) @world

7) eupdatedb (for esearch, with portage niceness applied)

8) q -qr (with portage niceness applied)

9) Then (while 6-8 are still backgrounded), spit out a "syncs done, 
fetches and db updates continuing in background" message, after which it 
returns me to a prompt while the background jobs complete.


Obviously that'll need generalized for non-local use.  1 and 2 could be 
generalized into presync.d (ordered), 6-9 into postsync.d (parallel), 3 
into simply sync.d (parallel), and 4 and 5 into immediate-postsync.d (or 
some such, ordered).

Then people could simply drop scriptlets into the appropriate *sync.d dir 
and let the general script handle it.  Meanwhile, #4, the ebuild-patch 
step that's of interest here, would become just another scriptlet file in 
immediate-postsync.d.

> Additionally, like Martin, I assumed you were doing this in a local
> overlay, not on the tree itself, with resultant rsync wipeout. So if we
> can sort that out, by writing the output to:
>  "${local_overlay:=/usr/local/portage}/${ebuild#"$PORTDIR"}"
> I'd be a lot happier. It'll fit much better in the process of pushing to
> an external overlay as well.

An option makes sense...

> The other thing I was going to mention is that update -s wraps the whole
> process, to filter the rsync output so it's only one line (the
> performance thing I mentioned in the other post: I never expected that
> to work so well but my mentor does the awk so I just tried in bash, and
> was amazed. :) So we could:
> a) get the list of ebuilds that have actually been changed via rsync, or
> b) check what eix is reporting after (if the user has eix.)

I think the generalized *sync.d approach should still be wrappable.  And 
anything the wrapper handles can simply be left out of the *sync.d dir 
it'd otherwise be located in.

> That's the trouble with glue-scripting: you have to consider the
> interaction of quite a few disparate commands, and various end-user
> setups.
> 
> It's also what makes it useful, and fun to work on. :-)

Indeed. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman