smoke testing on VMS (Re: HP hobbyist license)

[trying this again after removing the hefty t/logs directory from the attachment]

On Jun 2, 2012, at 11:20 AM, Nicholas Clark wrote:

> It would be *very* useful to have a VMS smoker automatically processing
> smoke-me branches.

I would suggest learning to crawl before we try to walk.  A manual way to "smoke branch X on VMS" that could be initiated remotely with one or two commands would be a quantum leap over what you have now and probably does 80% of what you want done anyway.  I actually have this working and have it packaged up so it's relatively easy to get going in a new environment (see below).

> I've not looked at the smoke testing code, but the biggest
> concerns I'd have are
> 
> 0: Can the machine cope with it? I suspect that smoke-testing is a full time
>  job.

I think it's a fairly old machine, pretty similar to the hobbyist system I have at home.  It takes me 3 1/2 hours to run a smoke on a RAM disk with only two configurations.  It will take much longer on a real disk.  You could probably run several smokes a week at most, but not several a day.

> 1: Are HP happy with it? :-)

I can't speak for HP, but it seems that the system is there for porting and testing open source software, which is what you're doing.  It may not allow outbound mail, though, so retrieving the test results will need some work.  

> 2: Presumably the smoker process fires off with cron, or somesuch. I don't
>  know any VMS sysadmin type stuff to translate how things are done from
>  Unix.

Here's what I do locally from my OS X laptop:

% ./branch_snap.sh <branch_name>
% ssh user@host 'submit/noprint/notify/param=("<branch_basename>", "<base_url>") [.Test-Smoke]runsmoke.com'

where <branch_name> is the name of the branch you want to smoke, <branch_basename> is the basename of the branch ("make_ext" for "smoke-me/make_ext") and <base_url> is a network location from which the generated snapshot may be fetched.  branch_snap.sh creates a snapshot from a local git repository and looks like:

#!/bin/csh

set branch=`basename "$1"`
echo "Working on $branch"
cd ~/perlrep/perl
git checkout $1 || exit 2
git pull
git clean -xdf
if (-e "~/Sites/$branch.zip") then
 rm "~/Sites/$branch.zip"
endif
perl Porting/make_dot_patch.pl > .patch
git archive --format=zip --output="$HOME/Sites/$branch.zip" HEAD
zip -pp="$branch/" ~/Sites/$branch.zip .patch
git clean -xdf

where ~/perlrep/perl would need to be a git repository and ~/Sites a location to store the snapshot.  In my case, it's also a location that is served up on my local network via http.  Modify as necessary to move the snapshot to a location from which it can be pulled.  Note: I believe zip -pp requires a pre-release version of Info-Zip; the goal is to add the .patch file to an existing archive, so any other means you can think of for accomplishing the same thing would be fine.

Now to the fun part. runsmoke.com is part of my own bastardized version of Test::Smoke.  I've attached that here as a zip file, but it's 317K and will probably get rejected by the mailing list.  Hopefully it will get through to Nicholas.  If anyone else wants it let me know and I can perhaps put it up somewhere accessible.  Just unzip it in your home directory -- that's all there is to installing it.

The "submit" command above (initiated remotely via ssh) starts a batch job, sort of like at.  The output will go to runsmoke.log in your home directory.  What runsmoke.com does is the following:

1.) Uses curl to fetch the snapshot from the specified URL.  If the system doesn't have curl, we can get it for you, or we can switch to using wget, or scp, or ftp, etc.

2.) It cleans up the existing branch directory if it exists.

3.) It unpacks the snapshot.

4.) And runs Test::Smoke in the directory for the specified branch.

and here is runsmoke.com inline for handy reference:

=====runsmoke.com=======
$! Run with:
$!       SUBMIT/NOPRINTER/NOTIFY/PARAM=("branch" [, "url" ] ) runsmoke
$!
$! Find out the directory this procedure is in.
$ whoami = f$parse(f$environment("PROCEDURE"),,,,"NO_CONCEAL")
$ topdir = f$parse(whoami,,,"DEVICE") + f$parse(whoami,,,"DIRECTORY")
$! strip end and add back what we need for rooted logical
$ topdir = topdir - ".][000000" - "][" - "].;" - "]" + ".]"  
$! 
$ define/translation_attributes=concealed SMOKE_ROOT 'topdir'
$ set default smoke_root:[000000]
$ show default
$
$ branch = p1
$ base_url = p2
$!
$! If we get a URL, fetch from it; otherwise assume the snapshot
$! is already here.
$!
$ if base_url .nes. ""
$ then
$   write sys$output "Fetching snapshot of ''branch' from ''base_url' ..."
$   if f$search("''branch'.zip") .NES. "" then delete/noconfirm 'branch'.zip;*
$   curl -s "''base_url'/''branch'.zip" --output 'branch'.zip
$ endif
$!
$ if f$search("''branch'.DIR") .NES. "" 
$ then 
$   write sys$output "Cleaning up existing ''branch' directory..."
$   rmtree :== 'perl' -e """use File::Path; rmtree(\@ARGV,0,0);""
$   define dt "[.dt]" ! override DECWindows definition
$   if f$trnlnm("TERM") .NES. "" then deassign TERM
$   rmtree 'branch'.DIR
$ endif
$!
$ write sys$output "Unpacking snapshot ..."
$ unzip -q 'branch'.zip
$!
$ write sys$output "Running smoke ..."
$ define SMOKE_BRANCH_DIR SMOKE_ROOT:['branch']
$ perl smokeperl.pl "--nofetch" "--nomail" "-c=smoke_config"
$!
$ write sys$output "All done."
$ exit
=====runsmoke.com=======

The arguments to smokeperl.pl and the contents of smoke_config can of course be fiddled with.

I might be forgetting something, but this is pretty close to everything needed to run smoke reports on VMS.

________________________________________
Craig A. Berry
mailto:[email protected]

"... getting out of a sonnet is much more
difficult than getting in."
                Brad Leithauser