Re: CVS conversion options

anatoly techtonik wrote:
> On Tue, Oct 13, 2009 at 7:22 AM, Michael Haggerty <[email protected]> wrote:
>> anatoly techtonik wrote:
>>> I want two CVS modules 'mod1' and 'mod2' to be merged into one 'pj1'
>>> project, and make every tag they share include both module.
>>> [...]
>>> Is there any way to accomplish that I need?
>> Create a "fake" CVS repository with the layout that you want the SVN
>> repository to have.  Put both "mod1" and "mod2" under a single
>> subdirectory "pj1", and treat "pj1" as a single project.  You can use
>> symlinks if you want to avoid making actual copies of the CVS repository
>> directories.
>>
>>    mkdir $TMP/fakecvs
>>    cd $TMP/fakecvs
>>    mkdir CVSROOT # cvs2svn needs this
>>    mkdir pj1
>>    ln -s $REALCVS/mod1 $REALCVS/mod2 pj1/
>>    for f in pj2 pj3 pj4
>>    do
>>        ln -s $REALCVS/$f .
>>    done
>>
>> Then do a multiproject conversion with projects pj1, pj2, ...
> 
> If I have several of them - can I convert all these once at a time?
> Won't this break the historical order of revisions like it can happen
> if I convert one project at a time?

The whole point of multiproject conversions is to convert multiple
projects in one run of cvs2svn.  See the FAQ [1] for instructions.  All
of the projects that are converted in a single run of cvs2svn have their
commits added to SVN in proper chronologically-interleaved order.

> Is it possible to embed this functionality into cvs2svn itself?

Certainly.  I don't think it should be too much work.

> I can
> see that add_project() can be modified to accept list of CVS source
> directories instead of just the only one. So, I've tried to analyze if
> it is possible, but need some help:

I think that cvs2svn needs more information than a list of directories,
because you also have to tell it how the directories should be assembled
to create the project's directory hierarchy in SVN.  In other words, you
need something like

    add_project(
        'pj1', # Overall SVN project name
        source=[
            # (path_to_subproject, subproject_svn_subpath):
            ('/realcvs/mod1', 'mod1'),
            ('/realcvs/mod2', 'mod2'),
            ('/realcvs/submod', 'mod2/submod'),
            ],
        ...)

This suggests that the add_project() method should ensure that the
resulting SVN paths and filenames do not conflict with each other.
E.g., in the example above, it is important that there is no directory
"realcvs/mod2/submod" or file "realcvs/mod2/submod,v" or
"realcvs/mod2/Attic/submod,v", because any of these would result in a
subversion path "mod2/submod", which would conflict with the "submod"
project's main subversion path.

If all conversions would take place on operating systems that support
symlinks, then the implementation of this feature could be
trivial--cvs2svn could *itself* create a temporary fake CVS repository
consisting of symlinks to the original CVS repository paths and convert
*that*.  But this would make error messages more obscure and you say you
want to support Windows users, so let's continue...

I don't have the time to verify all of your analysis, but I will add a
few comments below.

> [1] add_project() is defined in svn_run_options.py
> [2] project_cvs_repos_path (that is to be converted to a list) is a
> first argument to add_project()
> [3] project_cvs_repos_path is directly passed to Project()
> constructor, resulting project is added to a list of projects and
> that's all for svn_run_options.py
> 
> [4] Project() constructor in project.py receives
> project_cvs_repos_path argument that is to be transformed to the list
> [5] project_cvs_repos_path is used to set (6)
> Project.project_cvs_repos_path, (7) Project.cvs_repository_root and
> (8) Project.cvs_module, it is also used in (9) Project.__str__()
> 
> [6] Project.project_cvs_repos_path is transformed to the list. This
> affects CVSDirectory.get_filename() in some cases. Why can't
> CVSDirectory be self-sufficient?

I think this is mostly just to save space, to avoid storing the full CVS
path and the SVN path components redundantly, and possibly to avoid some
path name cutting and pasting.  It might also be that the pathname
components are used in keys in the RepositoryMirror and the fact that
they are interned helps save more space in that context.

Presumably the CVSDirectory objects corresponding to subproject root
paths would have to be treated specially, knowing their own full paths
and not using the project path anymore.

> [7] Project.cvs_repository_root is not transformed to the list. We
> just need to make sure that all paths share the same repository.

There is no technical requirement that all of the subprojects within a
project come from the same CVS repository.  The "patch together things
with symlinks" method does not impose such a requirement.  I believe
that the cvs_repository_root is only stored to tell CVS if the --use-cvs
option is being used.

> [8] Project.cvs_module should be converted to a list and it probably
> causes most issues. cvs_module is the path to conversion root within
> CVS repository - not ordinary "CVS module" listed in CVSROOT/modules
> file. cvs_module is used for comparisons of projects in (10)
> Project.__cmp__(), in  (11) CVSRevisionReader.get_content_stream() and
> in (12) InternalRevisionReader._KeywordExpander.source() function

You might want each cvs_module to have its own cvs_repository_root.

> [9] Project.__str__() that uses self.project_cvs_repos_path - no idea
> if it is used for some logic or just in error messages. I guess the
> best way to figure this out is to set logging and see for myself, but
> I am pretty limited here on windows both in time and tools at my
> disposal to quickly setup development environment for cvs2svn. Servers
> are running linux, but again - I find it very competing and
> time-consuming to study code from shell/vim.

I can't remember it being used for important things except probably for
the SVN log message for the initial project commits.  But I suggest that
each project have a unique name (see above), so this unique name is
probably fine for Project.__str__().

> [10] Project.__cmp__() - the same here. It is very hard to track usage
> of comparisons with static visual code analysis. Here I again need you
> expertise to say if conversion of cvs_module to a list won't break
> some parts of application.
> 
> [11] CVSRevisionReader.get_content_stream() - that's also requires
> developers attention
> 
> [12] InternalRevisionReader._KeywordExpander.source() - here I also
> can't be sure what workaround is needed.
> 
> 
> So, are these modifications possible? Will they do that is expected?

It sounds like you are on the right track.

> BTW, do you have Google Wave account?

No.  But I'm often on IRC ([email protected] in the #cvs2svn
channel).

Michael

------------------------------------------------------
http://cvs2svn.tigris.org/ds/viewMessage.do?dsForumId=1670&dsMessageId=2407398

To unsubscribe from this discussion, e-mail: [[email protected]].