Re: adding a new source and dest driver for AlienBrain

On Sun, Dec 14, 2003 at 01:07:28AM +0100, Timothee Besset wrote:
> Hello
> 
> I am considering adding a source and destination driver for NxN's
> AlienBrain. It's a bit tricky as you'll see so at this point I'm mostly
> playing around with it see if it's feasible.

Great!

> To start with something, I added VCP::Dest::ab and VCP::Source::ab copied
> from the null version. I'm not clear wether I need to hook my stuff
> somewhere else?

I'd copy from a "real" back end instead of the null backends.  Choose
based on which one is most like ab:

    - CVS branches in revision number space (foo#1.1.2.1 is a branch of
      foo#1.1), has no atomic changes, and VCP can read RCS files
      directly.
    - p4 is the nicest to work with.  It branches in name space
      (foo/file can be a branch of bar/file) and captures enough
      metadata to allow us to reproduce a source repository fairly
      accurately.  There is experimental support for P4::Client, an
      interface to the experimental p4 api library (the library is
      solid, mind you, but has some minor issues that keep us from using
      it reliably all the time).
    - There's an in-development svn back end out on the web, but it's
      still working through svn-specific and VCP internals issues (I
      refactored a lot and broke it :/).
    - VSS also branches in name space, but is awkward to work with for
      many reasons: missing metadata, poor command line tools and
      inconsistent data model unstable operation.

The back ends have three parts, usually:

    - VCP::Dest::foo: handle_header(), handle_rev(), handle_footer() and
      several other functions.  If the dest handles changes, then
      handle_rev() must accumulate changes until a rev for a different
      change arrives or handle_footer() is called.

    - VCP::Source::foo: handle_header(), copy_revs(), and
      handle_footer() scan the repository and emit metadata.  get_file()
      retrieves files as needed by the downstream filters or dest.

    - VCP::Utils::foo: These are common infrastructure for the dest &
      source.

Anyway, one of cvs, p4, or vss is likely to be a better starting point
than the null backend.  Or carry on, but read those and copy-n-edit as
needed.

> When I run 'perl -Ilib bin/vcp' I don't see ab in the list
> of source and dest I get prompted for.

You won't yet, those are for production ready backends and they're
hand-crafted in the ui_machines directory using state machines specified
in XML to specify the flow of the UI.  Leave that until last, the
command line and config files are far more appropriate for rapidly
evolving backends.

> - ab has a command line interface on unix. The command line program itself
> kind of sucks. I'd rather avoid having to spawn the command line
> executable to perform the operations. It uses java in a bunch of .jar from
> which I could get the commands executed directly maybe.

The command line is often a good place to start, it lets you use
relatively debugged and well defined access points (in general, not sure
how stable/usabel ab's CLI is).  Other access methods are usually much
faster:

- Going direct to disk (if the repository has a supported / published
  external on-disk representation) is pretty speedy and, for CVS at
  least, takes about as much work as parsing the log file emmitted by
  the cvs CLI.

- Implement a direct-to-backend protocol like P4::Client.  This has all
  the advantages of the CLI without the pain and suffering involved in
  repeatedly spawning child processes.  It's more flexible than direct
  file reads because the repository can be located over the net.

- Read / generate a data dump for the backend.  p4d can dump all
  metadata as a "checkpoint" and import checkpoints and "journal" files.
  We've not implemented this because it's in proprietary format and we
  don't want to have to track it every change.

NXN claims to offer a Perl API, that might be a really good way to go.

> - I don't know much perl or java. Just the basics. I'm a C++ and python
> guy mostly.

Well, not sure what to say there :).  Perl as used in most of VCP should
be pretty straight forward OO-ish coding, more C++ like than Java like.

- Barrie