Re: gar2vcheck

Sokar--

Let me take a crack at answering your concerns:

Starting with the last, first:

>
> And as an aside... this is probably a conversation that should be on
> the devel list... but maybe I'm just picky...
>

You're right, and I'd be happy to either stop posting or redirect to 
another list, but this and the associated lists have not been very 
active lately, and I was interested in getting feedback from whoever 
was actually involved now. Also, the original motivation was to answer 
a question you asked on this list:

> And also a question:  Is there an 'easier way' to monitor when an
> update to a package that we use comes out?  (other than the down list
> alphabetical thing i am doing now)  Cause its definitely not a small #
> of packages, and subscribing to all those devel lists (some of which
> don't have an 'announce' list, or a 'list' at all) would just be a
> damn nightmare.

I should have done a better job separating the offer to help with 
vcheck/gar2vcheck questions from other musings about a gar-related 
project of interest to (perhaps only) me.

I'll try to answer your specific concerns about the version checking 
below, but keep in mind that these are separable from the other 
speculative ideas I raised.

Let me try to answer some of your other concerns:

First, vcheck is not suitable for unattended naive fetching of all 
upstream packages; it has been extremely useful to me in doing exactly 
what you've asked in your question, which is to greatly reduce the 
effort in monitoring updates to packages I follow. I 'm currently using 
it to actively follow and download when changed over 80 diverse 
open-source packages with a single command line and a few minutes per 
day. I could make it a cron job and reduce that to no time or effort on 
my part. The price is configuring it carefully when you first begin 
tracking the package, and updating if an upstream site so fully 
reconfigures that your configuration fails. Most of the time, once you 
get it right, it stays stable through continued version updates. 
gar2vcheck is even less suitable for completely automated use; it is a 
starting point for generating a syntactically correct vcheck 
configuration that may work for some packages.

With that in mind:

On Saturday, February 18, 2006, at 04:14  PM, Sokar Someone wrote:

> I have some concerns about the gar2vcheck thing, maybe someone can
> address them...
>
> How do we handle the following:
> sourceforge (the current bane of my existence)

sourceforge is a pain in the neck. I have lots of sites I follow 
successfully that are sourceforge projects, though. I see, for 
instance, that for net-snmp, I use
URL = http://prdownloads.sourceforge.net/net-snmp
but then need an explicit download spec:
dlexplicit = 
http://easynews.dl.sourceforge.net/net-snmp/net-snmp-__RAWVER__.tar.gz
Most of the sourceforge projects and files can be done similarly: find 
the version from the project page, then find a valid, reliable 
sourceforge mirror from their list and use the version info discovered 
automatically in the first step to program the download.

For ease of configuration, I try to stay away from dlexplicit, but it 
is often the only way to (a) bypass interactive web pages and (b) 
download a family of files of the same version (e.g., a gpg signature 
and the associated tarball...).

Again, it's only painful the first time, and each similar site can be 
done analogously. Also, you'd have the same issues maintaining the 
upstream info in gar by hand, but wouldn't have any chance of its 
maintaining itself over simple version updates....

>
> websites that have been hijacked and now send you a garbage file of
> the version that you request

I haven't had this problem in years of using vcheck. I do choose the 
packages I follow carefully. When vcheck tells me it's updated my 
archive, I go to the web/ftp site myself to look at the announcement, 
the changelog, etc. to confirm that it's a worthwhile change. Since my 
vcheck configuration is set up to add to my archive, not overwrite 
older versions, I can always blow away a non-useful or malicious 
download before opening... The manual checking of the web site means 
that you wouldn't save much time the first time around when most of the 
packages updated, but would be safer, and then you'd begin saving huge 
amounts of time subsequently as the newly updated packages change 
relatively infrequently, and you'd only need to check out changed 
packages, and those only if you were interested in updating.

>
> people who have changed their versioning scheme

Can be a problem. Rarely...usually the more mature, solid open-source 
projects have a well-established versioning scheme. On the rare 
occasion I've run into problems, there are symptoms...for instance 
vcheck errors on that package start mounting up, and I investigate and 
find that the site is still up but has been so reorganized as to 
require manual configuration change. I don't, but could and should save 
my vcheck config to CVS whenever it requires such manual intervention.

>
> also is there a way we could go through and check for new / changed
> dependencies?  (of course this would involve changing the makefiles to
> include minimum version #'s of the depends, maybe on a comment line?)

Not that I'm aware of...we could look into extracting dependency info 
and building the vcheck config to look at coordinated sets of projects. 
On the other hand, if you just regularly update the (g)archive, then it 
really isn't a dependency issue...all the required versions of 
dependencies will be available from the local archive when needed. I 
don't know how to determine what those dependencies would be 
automatically, though. I guess the 2 most likely options are: leave the 
known dependencies alone unless building the package fails or the 
documentation instructs a change, or update the version of all 
dependencies to the latest available in the archive, and try to build...

>
> And if we do try to do all this automation are we going to have
> 'verified' versions of the garchive that we know will build (at least
> 1 person from the devel team has gotten it to build w/o much trouble)
> and 'use at your own risk' versions that someone can use if they want
> an update we haven't 'verified' yet?

That's part of the missing piece I was asking about. I know that the 
vcheck/gar2vcheck combination can be a useful tool to apply manually 
when the project needs simple version updating to freshen. I think it 
would be simple to have a gar parameter that would allow a 
user/developer/package maintainer to optimistically try to update one 
or more packages, and try to build an "alpha" version from the 
automatically updated files. I think that such versions should never be 
baselined until they've been successfully built, tested, and moved to 
the head/baseline by the package owner or other authorized developer. 
Still, seems like a useful tool to have to have a shot at efficiently 
updating all those packages that have been orphaned by their original 
owners...

>
> Also, am i detecting a want to fork this in to a 'bbc sized' build and
> a 'how ever big you want' build?  Or is this just a user option to put
> out there for the automated builds?

The rest of my rambling had to do with a particular use/interest of 
mine, and as you pointed out, it has more to do with gar development 
than lnx--bbc. It's my own observation that while the goal of having 
the bbc-sized image is nice, and the distribution has always been 
well-implemented, it may be of increasingly less interest as people 
have broadband connections to download lots of nice liveCDs with lots 
of tools, etc. The build system, however, is of particular and enduring 
interest to me, personally. For example, I just read yesterday of a 
small distribution that implements the open-source PBX, Asterix, on a 
read-only system (compact flash based). In order to do this, the 
developer hacked up Gentoo at some considerable effort, to remove 
packages and customize so it'd run from read-only media. I think gar 
could easily evolve into a nice way to quickly generate such a 
distribution from a common set of packages with little additional 
effort. This is not directly related to version updating and archiving 
but motivates my interest in having a more general, traceable, easily 
updated archive.

Bottom line on vcheck: very useful, not automatic, work in configuring 
per package, but then works well, no longer maintained by author
Bottom line on gar2vcheck- my hack, of questionable utility, not 
automatic, may be useful for generating vcheck config starting point 
from gar tree, I'd be willing to work some more on it to make it work 
better, easier to automatically update vcheck config without losing 
hand written customizations, if it will get used...

Thanks,
Bill