Re: [Security-Discuss] 10.1 x86_64 update mirrors are out of sync

On Tuesday 12 July 2005 19:36, Vincent Danen wrote:
[..]
> >> What exactly was the problem?  Which packages were missing?
> >
> > After reading the security-announce mails I tried to do an update this
> > morning, but "urpmi --auto-select" came up with no packages.
> > So I checked the date of the hdlist.cz file, it was dated as
> > 07.07.2005
> > and the cpio-RPM had not the recent version - a typical sync problem.
> > I resynced from your corporate rsync server, but nothing changed.
> > I looked at ftp.proxad.net - still the old packages.'
>
> Silly question, but you did a "urpmi.update -a" first, before the --
> auto-select, right?  Otherwise urpmi is working with the hdlist.cz or
> synthesis list on the local machine which wouldn't of course contain
> any reference to the new packages.

Yes, I did a "urpmi.update update_source" first. ;)

> Also..  "this morning" doesn't mean much to me.  It's still morning
> for me (11:30am) so could morning have been 8hrs ago?  Longer?  Shorter?

Sorry, I meant 10:00am CEST.
Btw. where are you situated?

> I ask because it of course takes time for the mirrors to catch up.  I
> waited about 3hrs (I think?) before announcing the packages last
> night which usually is enough time to get it to the "primaries"
> anyways.  Not sure if proxad is a "primary" or not.
>
> Also, what do you mean by the corporate rsync server?  I didn't know
> we had one.  Are you updating corporate server or desktop?  Because
> that one, from what I understand, should be damn-near instantaneous
> (AFAIK, I'm uploading directly to the corp repository).

No, this is independent from the Corporate Desktop or Server.
We are a member of the Corporate Mandriva Club. One benefit of this is
access to a rsync server at Mandriva with the latest Commercial Club packages
and the whole update tree.

[..]
> > You are right about the external mirrors, you can't do anything
> > about them.
> > But we also have access to the Mandriva corporate rsync server.
> > We should not be affected from these problems - but we are.
> > In most cases the corporate server is also out of sync, so the
> > glitch has
> > to happen somewhere inside Mandriva (according to the main list,
> > it was a hard disk running full this time).
>
> What is the main list?  And was this hard disk being full something
> that happened today?  And I still don't know about this corporate
> rsync server.  So many questions... =)

The main list? ;) [email protected] 
And I'm refering to this posting:
http://archives.mandrivalinux.com/cooker/2005-07/msg01068.php

> It could very well be something inside of Mandriva, but I (as you
> probably have guessed by now) know almost nothing about the
> infrastructure.  I can guess how many hops the packages take (or at
> least how many they used to... it could very well have changed in the
> last year or two), but I don't know what happens to the intermediate
> steps.
>
> I do know that for the corp products I upload to what I believe is
> the accessible repository and if it isn't, it's one hop away from
> it.  For "regular" updates, I upload to our development system, which
> our rsync system mirrors from, and the primaries get from the rsync
> system; the other mirrors from the primaries, etc.  I assume,
> possibly incorrectly, that the "corporate rsync server" is the rsync
> system the mirrors use.
>
> If it is, then it's 2 hops from the originating system; if it's not
> the same server, then I don't know how many hops it is.

Although there are several hops involved, that shouldn't pose a 
problem as rsync is quite robust ... unless something happens
like a hard disk running full of course, many systems, many sources
of errors. :-\

> > Don't get me wrong, accidents can happen from time to time.
> > But if you look through the archives of this list, you will find
> > several
> > postings from me about mirror problems, outdated hdlist.cz files or
> > wrong public keys on the packages.
> > I guess for every 10 security updates there is one of these problems.
>
> Yes, I know.  You're absolutely right, and each time I forward the
> problems on to the people who can fix them.  How they fix them I do
> not know.  But I don't have the access to fix/change things on my own
> so there are communication delays, etc.  Anyways, I won't make
> excuses.  You and I both agree there are issues with the current
> system.  I don't know how else I can improve the situation beyond
> making my own repository (the master) publically accessible.  But
> that would cost me too much bandwidth.  =)

Maybe you can talk with the maintainer of the mirror system and ask
him whether he can think of some improvements.
If this doesn't work you could talk with the management and apply for 
a fatter pipe and really open you own rsync server. ;)

> > Maybe this update mechanism has some hidden depths I do not see,
> > but for me it looks like it's a matter of digging through all the
> > steps
> > necessary to ship an update package and then to write scripts
> > which do the signing of the package, updating of the hdlists and
> > sending of the packages and hdlists to the master mirror.
>
> I can dig through my steps.  But I know that's not where the problem
> is.  Packages are signed, hdlists updated, all the "maintenance" is
> done before the packages even get uploaded anywhere.  All I ever do
> is upload the file tree (well, rsync it).  All the "maintenance" is
> done here.  So errors with package signing (which happens on occasion
> although I'm not sure why since nothing ever changes), or hdlist
> generation, etc. would be on my end.  Unless something isn't being
> synced or gets corrupt in transit, etc.

Do you have read access to the "master mirrors" at Mandriva?
If yes, you could write a script which compares the update tree there
with yours before you send out the advisories (just an idea).

> The problem is somewhere once everything gets released from my
> system, but once it's out of my system, it's largely beyond my control.

Yes, seems like a tricky situation. But thanks for the explanation from 
your side. From the outside it looks very odd that there are nearly no
problems with the RPMS themself, they install smooth, no side effects,
real good QA. But then on the way to the customer things suddenly 
get bumpy, missing hdlists, wrong keys, some RPMS missing.
It looks like someone is sitting there and copying the files by hand 
and then - of course - forgets some of them from time to time.
But it's more like a many cooks, many systems problem.

> > Am I wrong?
>
> A little bit, but mostly not.

Yeah, that's me. :)

cu
Michi

____________________________________________________
Want to buy your Pack or Services from Mandriva? 
Go to http://store.mandriva.com
Join the Club : http://www.mandrivaclub.com
____________________________________________________