Re: cvs2svn, vendor branches, and missing default branch

On 1/3/15 4:32 PM, Michael Haggerty wrote:
> On 01/03/2015 04:14 AM, John Hood wrote:
>> I've got a problem with some old repos where cvs2svn is choosing
>> revisions differently than CVS for vendor imports.  I"m not sure it's
>> actually a CVS2SVN *bug*, because the files involved have unusual
>> history, and are possibly corrupt in a sense.  But CVS2SVN is creating
>> output that's different from what CVS saw.
>>
>> The files in question are files that have seen two vendor imports, and
>> no revisions any place else.  They appear to be constructed as one would
>> expect, except that they lack a 'branch' header to set the default
>> branch.  That header seems to have disappeared around the time of the
>> second vendor import, because new tags on trunk start showing up as
>> revision 1.1 at that time, instead of 1.1.1.1 or 1.1.1.2.  The second
>> vendor import seems to have been incorrectly/incompletely done, since
>> the 1.1.1.2 revisions aren't used anywhere in the history.  The files
>> appear to be syntactically valid, and semantically valid apart from the
>> missing 'branch' header.  That may or may not be valid, but its absence
>> does mean that a little bit of explicit revision history on these files
>> at certain points in time is lost.
>>
>> With these files, CVS chooses revision 1.1.1.1 for checkouts on trunk,
>> but cvs2svn chooses the second vendor import, formerly known as 1.1.1.2
>> in CVS, after the time of the second import.  This only happens with
>> these files.  Files that were created by the first vendor import and
>> unchanged by the second don't have this problem, of course, since they
>> don't have a 1.1.1.2.  Files that have been imported twice, and updated
>> on HEAD, are OK (though there are the usual footprints of messy CVS
>> merges).  I think cvs2svn is making a more logical choice about which
>> revision to use, but unfortunately, it's different from what CVS
>> chooses, and we need to preserve the CVS history.  It looks like
>> cvs2svn's behavior is a logical result of how it grafts vendor branches
>> into trunk-- resulting in something slightly different from CVS' heuristic.
>>
>> I see this problem in two CVS repositories so far-- the one I'm working
>> on converting and discussing here, and another which I've not looked
>> closely at yet.  Both have long history, dating back to 1996-1999.  Both
>> were hosted on FreeBSD machines, but by completely different
>> organizations.  I doubt it's significant, but in the first repository,
>> the two vendor imports were done in two different CVS modules, which are
>> submodules of another which is almost always what people use-- it's
>> generally rare for the submodules to be checked
>> out/edited/tagged/branched independently. It looks like the first vendor
>> import was done by a single person, but the second imports were done by
>> different people a couple of weeks apart.
>>
>> I've attached a destroyed CVS repository with two of these files from
>> the two submodules. It's also got a few other files with plenty of
>> revisions to give cvs2svn something to use for dates and correlation.
>> There's also a copy of that repo that's been reduced nicely to the
>> minimum history needed to show this problem.
>>
>> I took a stab at fixing the problem in cvs2svn, but the code and
>> internal state dealing with vendor branches is complicated enough that I
>> didn't find the right one- or two-line fix.  I have an external
>> workaround which involves making a force commit and then editing
>> revision numbers to point at the force commit, before the cvs2svn
>> conversion.  It gets me a reasonable SVN history.  But I'd like to fix
>> it more cleanly; there's perhaps 10 more repos behind this one that will
>> be converted.
> Thanks for the very thorough error report!
>
> The handling of vendor branches in CVS is quite confusing, and CVS
> discards critical information such as when the default branch was
> switched back from the vendor branch to HEAD.  So the code makes some
> guesses here and I suppose that one of them is wrong.
It *is* quite confusing, which is why I had to report the bug instead of 
fixing it.  :)
>
> The two methods most closely related to your problem are
>
> * collect_data.py:_FileDataCollector._process_ntdbrs()
>
> * cvs_file_items.py:CVSFileItems.process_historical_ntdb()
>
> My guess is that the second of these functions is being too aggressive.
> Could you try the following change and see if it helps:
>
> diff --git a/cvs2svn_lib/cvs_file_items.py b/cvs2svn_lib/cvs_file_items.py
> index d339047..b8f395d 100644
> --- a/cvs2svn_lib/cvs_file_items.py
> +++ b/cvs2svn_lib/cvs_file_items.py
> @@ -533,7 +533,7 @@ class CVSFileItems(object):
>       rev_1_2_id = rev_1_1.next_id
>
>       if rev_1_2_id is None:
> -      rev_1_2_timestamp = None
> +      return False
>       else:
>         rev_1_2_timestamp = self[rev_1_2_id].timestamp
That seems to have worked in a test that covers all branches and 
significant tags in this repo.  I still need to check the actual 
revision history for sanity, and date-based checkouts.
>
> Given your description of the problem and of CVS's behavior, I don't see
> how we can justify treating vendor branch revisions as HEAD branch
> revisions if there is no 1.2 revision to tell when to switch back to
> HEAD (i.e., I think this change would be justified in general, not just
> in your special case).
I confess to some unease about the simple fix though it seems to work.  
There was some case I was concerned about when I was looking through the 
source, but I've forgotten what it was now.

regards,

   --jh

------------------------------------------------------
http://cvs2svn.tigris.org/ds/viewMessage.do?dsForumId=1670&dsMessageId=3093717

To unsubscribe from this discussion, e-mail: [[email protected]].