Interface and documentation fixups for cvs2{svn,git,hg.bzr}.
"Eric S. Raymond" <[email protected]> Mon, 7 Jan 2013 06:16:53 -0500
| Newsgroups | gmane.comp.version-control.subversion.cvs2svn.devel |
|---|---|
| Organization | Eric Conspiracy Secret Labs |
| Message-ID | <[email protected]> |
------=_Part_2939_1598092016.1357597159775 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline These are all backward-compatible. The main theme is reducing the number and complexity of command-line options required to invoke the tool. I've also improved the documentation and removed almost all Subversion-related fossils from the way the non-cvs2svn manual pages are generated. It's possible the second patch could be generalized to make the --username option optional for all target systems. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a> ------=_Part_2939_1598092016.1357597159775 Content-Type: text/x-diff; charset=us-ascii; name=0001-Enable-a-Unix-style-streaming-interface-and-fix-the-.patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0001-Enable-a-Unix-style-streaming-interface-and-fix-the-.patch From a5d0cbef0314d4d9f711d5876d563c86418da310 Mon Sep 17 00:00:00 2001 From: "Eric S. Raymond" <[email protected]> Date: Sun, 6 Jan 2013 22:50:36 -0500 Subject: [PATCH 1/4] Enable a Unix-style streaming interface and fix the generated man page. With this patch, it is now possible to omit the --blobfile and --dumpfile options. When this is done, the generated import stream is emitted to standard output and the statistical information shunted to standard error. Care is taken not to bloat the tool's working set in this case. This patch also revamps the generation of the cvs2git man page and cleans up several of its small fossils from cvs2svn. Left untouched is the big one, the "SUBVERSION PROPERTIES" section; that will require another patch. --- cvs2svn_lib/git_output_option.py | 25 ++++++++++++++++++++----- cvs2svn_lib/git_run_options.py | 31 ++++++++++++++++++++++--------- cvs2svn_lib/log.py | 8 ++++++++ cvs2svn_lib/run_options.py | 14 +++++++------- 4 files changed, 57 insertions(+), 21 deletions(-) diff --git a/cvs2svn_lib/git_output_option.py b/cvs2svn_lib/git_output_option.py index d7afa4b..2a08432 100644 --- a/cvs2svn_lib/git_output_option.py +++ b/cvs2svn_lib/git_output_option.py @@ -24,6 +24,8 @@ For information about the format allowed by git-fast-import, see: import bisect import time +import sys +import os from cvs2svn_lib.common import InternalError from cvs2svn_lib.log import logger @@ -144,13 +146,18 @@ class GitOutputOption(DVCSOutputOption): # value needs to be large to avoid conflicts with blob marks. _first_commit_mark = 1000000000 - def __init__( - self, dump_filename, revision_writer, - author_transforms=None, - tie_tag_fixup_branches=False, - ): + def __init__(self, + blob_filename, + dump_filename, + revision_writer, + author_transforms=None, + tie_tag_fixup_branches=False, + ): """Constructor. + BLOB_FILENAME is the name of the file to which the content blobs + for the git-fast-import stream should be written. + DUMP_FILENAME is the name of the file to which the git-fast-import commands for defining revisions should be written. (Please note that depending on the style of revision writer, the actual file @@ -172,6 +179,7 @@ class GitOutputOption(DVCSOutputOption): """ DVCSOutputOption.__init__(self) + self.blob_filename = blob_filename self.dump_filename = dump_filename self.revision_writer = revision_writer @@ -546,6 +554,13 @@ class GitOutputOption(DVCSOutputOption): DVCSOutputOption.cleanup(self) self.revision_writer.finish() self.f.close() + if logger.is_streaming_set(): + for line in open(self.blob_filename): + sys.stdout.write(line) + for line in open(self.dump_filename): + sys.stdout.write(line) + os.remove(self.blob_filename) + os.remove(self.dump_filename) del self.f diff --git a/cvs2svn_lib/git_run_options.py b/cvs2svn_lib/git_run_options.py index 70979e2..9f76d3b 100644 --- a/cvs2svn_lib/git_run_options.py +++ b/cvs2svn_lib/git_run_options.py @@ -19,6 +19,7 @@ from cvs2svn_lib.common import FatalError from cvs2svn_lib.context import Ctx +from cvs2svn_lib.log import logger from cvs2svn_lib.dvcs_common import DVCSRunOptions from cvs2svn_lib.run_options import ContextOption from cvs2svn_lib.run_options import IncompatibleOption @@ -35,24 +36,30 @@ from cvs2svn_lib.git_output_option import GitOutputOption class GitRunOptions(DVCSRunOptions): - short_desc = 'convert a cvs repository into a git repository' + short_desc = 'convert a CVS repository into a git fast-import stream' synopsis = """\ .B cvs2git -[\\fIOPTION\\fR]... \\fIOUTPUT-OPTIONS CVS-REPOS-PATH\\fR +[\\fIOPTION\\fR]... [\\fIOUTPUT-OPTIONS] CVS-REPOS-PATH\\fR .br .B cvs2git [\\fIOPTION\\fR]... \\fI--options=PATH\\fR """ long_desc = """\ -Create a new git repository based on the version history stored in a -CVS repository. Each CVS commit will be mirrored in the git -repository, including such information as date of commit and id of the -committer. +Translate the version history stored in a CVS repository into a git +fast-import stream that can be used to create a live repository with +.BR git-fast-import +or any compatible importer. .P -The output of this program are a "blobfile" and a "dumpfile", which -together can be loaded into a git repository using "git fast-import". +The CVS history's per-file commits and tags will be analyzed and +grouped into changesets, each with a common commit date and author. +.P +Called without options, this program simply dumps the translated stream to +standard output (and some statistical information to standard error). For +backward compatibility with older versions, you can specify two filenames +with command-line options in which to separately dump the content blobs +and commits; in that case statistical information goes to standard output. .P \\fICVS-REPOS-PATH\\fR is the filesystem path of the part of the CVS repository that you want to convert. This path doesn't have to be the @@ -164,7 +171,12 @@ A directory called \\fIcvs2svn-tmp\\fR (or the directory specified by return if not (options.blobfile and options.dumpfile): - raise FatalError("must pass '--blobfile' and '--dumpfile' options.") + import tempfile + if not options.dumpfile: + options.dumpfile = tempfile.mkstemp(prefix="cvs2gitdump")[1] + if not options.blobfile: + options.blobfile = tempfile.mkstemp(prefix="cvs2gitblobs")[1] + logger.set_streaming() if options.use_external_blob_generator: ctx.revision_collector = ExternalBlobGenerator(options.blobfile) @@ -189,6 +201,7 @@ A directory called \\fIcvs2svn-tmp\\fR (or the directory specified by ctx.output_option = NullOutputOption() else: ctx.output_option = GitOutputOption( + self.options.blobfile, self.options.dumpfile, GitRevisionMarkWriter(), # Optional map from CVS author names to git author names: diff --git a/cvs2svn_lib/log.py b/cvs2svn_lib/log.py index 1ab4614..62e8c75 100644 --- a/cvs2svn_lib/log.py +++ b/cvs2svn_lib/log.py @@ -76,6 +76,14 @@ class _Log: return self.log_level >= level + def set_streaming(self): + "Prevent the logger from using standard output." + self._out = sys.stderr + + def is_streaming_set(self): + "Has streaming operation been set?" + return self._out == sys.stderr + def _timestamp(self): """Return a timestamp if needed, as a string with a trailing space.""" diff --git a/cvs2svn_lib/run_options.py b/cvs2svn_lib/run_options.py index 5d98fcd..4b4ac02 100644 --- a/cvs2svn_lib/run_options.py +++ b/cvs2svn_lib/run_options.py @@ -360,11 +360,11 @@ class RunOptions(object): help=( 'if a file appears both in and out of ' 'the CVS Attic, then leave the attic version in a ' - 'SVN directory called "Attic"' + 'subdirectory called "Attic"' ), man_help=( - 'If a file appears both inside an outside of the CVS attic, ' - 'retain the attic version in an SVN subdirectory called ' + 'If a file appears both inside and outside of the CVS attic, ' + 'retain the attic version in a subdirectory called ' '\'Attic\'. (Normally this situation is treated as a fatal ' 'error.)' ), @@ -384,8 +384,8 @@ class RunOptions(object): 'respectively. P must match the whole symbol name' ), man_help=( - 'Transform RCS/CVS symbol names before entering them into ' - 'Subversion. \\fIpattern\\fR is a Python regexp pattern that ' + 'Transform RCS/CVS symbol names before entering them into the ' + 'output history. \\fIpattern\\fR is a Python regexp pattern that ' 'is matches against the entire symbol name; \\fIreplacement\\fR ' 'is a replacement using Python\'s regexp reference syntax. ' 'You may specify any number of these options; they will be ' @@ -500,9 +500,9 @@ class RunOptions(object): group.add_option(ContextOption( '--username', type='string', action='store', - help='username for cvs2svn-synthesized commits', + help='username for synthesized commits', man_help=( - 'Set the default username to \\fIname\\fR when cvs2svn needs ' + 'Set the default username to \\fIname\\fR when this program needs ' 'to generate a commit for which CVS does not record the ' 'original username. This happens when a branch or tag is ' 'created. The default is to use no author at all for such ' -- 1.7.9.5 ------=_Part_2939_1598092016.1357597159775 Content-Type: text/x-diff; charset=us-ascii; name=0002-Make-the-irritating-username-option-unnecessary-for-.patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0002-Make-the-irritating-username-option-unnecessary-for-.patch From 4166a5ea5ad28409b0657ecaca36c75baa8d93fe Mon Sep 17 00:00:00 2001 From: "Eric S. Raymond" <[email protected]> Date: Sun, 6 Jan 2013 23:22:51 -0500 Subject: [PATCH 2/4] Make the irritating --username option unnecessary for cvs2git. This is a bit of a kluge based on the way that cvs2hg evades the problem. It's not clear to me where the default username is used, anyway. --- cvs2svn_lib/git_run_options.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/cvs2svn_lib/git_run_options.py b/cvs2svn_lib/git_run_options.py index 9f76d3b..e470758 100644 --- a/cvs2svn_lib/git_run_options.py +++ b/cvs2svn_lib/git_run_options.py @@ -85,6 +85,11 @@ A directory called \\fIcvs2svn-tmp\\fR (or the directory specified by ('git-fast-import', '1'), ] + def __init__(self, *args, **kwargs): + # Override some default values + ctx = Ctx() + ctx.username = "cvs2git" + super(GitRunOptions, self).__init__(*args, **kwargs) def _get_output_options_group(self): group = super(GitRunOptions, self)._get_output_options_group() -- 1.7.9.5 ------=_Part_2939_1598092016.1357597159775 Content-Type: text/x-diff; charset=us-ascii; name=0003-Don-t-set-up-cvs2svn-option-parsing-unless-we-re-cvs.patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0003-Don-t-set-up-cvs2svn-option-parsing-unless-we-re-cvs.patch From a040b695cf5696663db40b63e561a2cfa5a6a3b2 Mon Sep 17 00:00:00 2001 From: "Eric S. Raymond" <[email protected]> Date: Mon, 7 Jan 2013 05:36:51 -0500 Subject: [PATCH 3/4] Don't set up cvs2svn option parsing unless we're cvs2svn. This change also causes the SUBVERSION PROPERTIES section to be omitted from generated man pages when appropriate. --- cvs2svn_lib/dvcs_common.py | 2 -- cvs2svn_lib/run_options.py | 39 ++------------------------------------- cvs2svn_lib/svn_run_options.py | 34 +++++++++++++++++++++++++++++++++- 3 files changed, 35 insertions(+), 40 deletions(-) diff --git a/cvs2svn_lib/dvcs_common.py b/cvs2svn_lib/dvcs_common.py index 5c8e1b6..40051f7 100644 --- a/cvs2svn_lib/dvcs_common.py +++ b/cvs2svn_lib/dvcs_common.py @@ -98,8 +98,6 @@ class DVCSRunOptions(RunOptions): self.project_symbol_strategy_rules = [symbol_strategy_rules] def process_property_setter_options(self): - super(DVCSRunOptions, self).process_property_setter_options() - # Property setters for internal use: Ctx().file_property_setters.append( KeywordHandlingPropertySetter('collapsed') diff --git a/cvs2svn_lib/run_options.py b/cvs2svn_lib/run_options.py index 4b4ac02..3f74668 100644 --- a/cvs2svn_lib/run_options.py +++ b/cvs2svn_lib/run_options.py @@ -239,7 +239,8 @@ class RunOptions(object): parser.add_option_group(self._get_output_options_group()) parser.add_option_group(self._get_conversion_options_group()) parser.add_option_group(self._get_symbol_handling_options_group()) - parser.add_option_group(self._get_subversion_properties_options_group()) + if self.progname == "cvs2svn": + parser.add_option_group(self._get_subversion_properties_options_group()) parser.add_option_group(self._get_extraction_options_group()) parser.add_option_group(self._get_environment_options_group()) parser.add_option_group(self._get_partial_conversion_options_group()) @@ -1019,42 +1020,6 @@ class RunOptions(object): # branches and tags: options.symbol_strategy_rules.append(HeuristicPreferredParentRule()) - def process_property_setter_options(self): - """Process the options that set SVN properties.""" - - ctx = Ctx() - options = self.options - - for value in options.auto_props_files: - ctx.file_property_setters.append( - AutoPropsPropertySetter(value, options.auto_props_ignore_case) - ) - - for value in options.mime_types_files: - ctx.file_property_setters.append(MimeMapper(value)) - - ctx.file_property_setters.append(CVSBinaryFileEOLStyleSetter()) - - ctx.file_property_setters.append(CVSBinaryFileDefaultMimeTypeSetter()) - - if options.eol_from_mime_type: - ctx.file_property_setters.append(EOLStyleFromMimeTypeSetter()) - - ctx.file_property_setters.append( - DefaultEOLStyleSetter(options.default_eol) - ) - - ctx.file_property_setters.append(SVNBinaryFileKeywordsPropertySetter()) - - if not options.keywords_off: - ctx.file_property_setters.append( - KeywordsPropertySetter(config.SVN_KEYWORDS_VALUE) - ) - - ctx.file_property_setters.append(ExecutablePropertySetter()) - - ctx.file_property_setters.append(DescriptionPropertySetter()) - def process_options(self): """Do the main configuration based on command-line options. diff --git a/cvs2svn_lib/svn_run_options.py b/cvs2svn_lib/svn_run_options.py index c2a7185..ca1a06d 100644 --- a/cvs2svn_lib/svn_run_options.py +++ b/cvs2svn_lib/svn_run_options.py @@ -487,7 +487,39 @@ A directory called \\fIcvs2svn-tmp\\fR (or the directory specified by del self.project_symbol_strategy_rules[:] def process_property_setter_options(self): - super(SVNRunOptions, self).process_property_setter_options() + """Process the options that set SVN properties.""" + ctx = Ctx() + options = self.options + + for value in options.auto_props_files: + ctx.file_property_setters.append( + AutoPropsPropertySetter(value, options.auto_props_ignore_case) + ) + + for value in options.mime_types_files: + ctx.file_property_setters.append(MimeMapper(value)) + + ctx.file_property_setters.append(CVSBinaryFileEOLStyleSetter()) + + ctx.file_property_setters.append(CVSBinaryFileDefaultMimeTypeSetter()) + + if options.eol_from_mime_type: + ctx.file_property_setters.append(EOLStyleFromMimeTypeSetter()) + + ctx.file_property_setters.append( + DefaultEOLStyleSetter(options.default_eol) + ) + + ctx.file_property_setters.append(SVNBinaryFileKeywordsPropertySetter()) + + if not options.keywords_off: + ctx.file_property_setters.append( + KeywordsPropertySetter(config.SVN_KEYWORDS_VALUE) + ) + + ctx.file_property_setters.append(ExecutablePropertySetter()) + + ctx.file_property_setters.append(DescriptionPropertySetter()) # Property setters for internal use: Ctx().file_property_setters.append(SVNEOLFixPropertySetter()) -- 1.7.9.5 ------=_Part_2939_1598092016.1357597159775 Content-Type: text/x-diff; charset=us-ascii; name=0004-Default-the-repository-path-to-the-current-directory.patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0004-Default-the-repository-path-to-the-current-directory.patch From a2446fc3c21d9ed4ca12f5fcdb94ea98fba453f5 Mon Sep 17 00:00:00 2001 From: "Eric S. Raymond" <[email protected]> Date: Mon, 7 Jan 2013 05:49:45 -0500 Subject: [PATCH 4/4] Default the repository path to the current directory. The objective here is to reduce the amount of ceremony required to invoke this tool. With this change, on top of my previous three comments, going to a CVS module directory and simply typing "cvs2git" suffices to get a streamed history on standard output. --- cvs2svn_lib/bzr_run_options.py | 6 +++--- cvs2svn_lib/dvcs_common.py | 4 ++-- cvs2svn_lib/git_run_options.py | 4 ++-- cvs2svn_lib/hg_run_options.py | 19 ++++++++++--------- cvs2svn_lib/svn_run_options.py | 7 +++++-- 5 files changed, 22 insertions(+), 18 deletions(-) diff --git a/cvs2svn_lib/bzr_run_options.py b/cvs2svn_lib/bzr_run_options.py index 2ffd3ce..5fdb798 100644 --- a/cvs2svn_lib/bzr_run_options.py +++ b/cvs2svn_lib/bzr_run_options.py @@ -37,7 +37,7 @@ class BzrRunOptions(DVCSRunOptions): synopsis = """\ .B cvs2bzr -[\\fIOPTION\\fR]... \\fIOUTPUT-OPTIONS CVS-REPOS-PATH\\fR +[\\fIOPTION\\fR]... \\fIOUTPUT-OPTIONS [CVS-REPOS-PATH]\\fR .br .B cvs2bzr [\\fIOPTION\\fR]... \\fI--options=PATH\\fR @@ -55,12 +55,12 @@ Plugin, available from https://launchpad.net/bzr-fastimport. .P \\fICVS-REPOS-PATH\\fR is the filesystem path of the part of the CVS -repository that you want to convert. This path doesn't have to be the +repository that you want to convert. This path doesn't have to be the top level directory of a CVS repository; it can point at a project within a repository, in which case only that project will be converted. This path or one of its parent directories has to contain a subdirectory called CVSROOT (though the CVSROOT directory can be -empty). +empty). If not specified, it defaults to the current directory. .P It is not possible directly to convert a CVS repository to which you only have remote access, but the FAQ describes tools that may be used diff --git a/cvs2svn_lib/dvcs_common.py b/cvs2svn_lib/dvcs_common.py index 40051f7..8d8f447 100644 --- a/cvs2svn_lib/dvcs_common.py +++ b/cvs2svn_lib/dvcs_common.py @@ -18,6 +18,7 @@ Git, Mercurial, or Bazaar). """ +import os import sys from cvs2svn_lib import config @@ -106,8 +107,7 @@ class DVCSRunOptions(RunOptions): def process_options(self): # Consistency check for options and arguments. if len(self.args) == 0: - self.usage() - sys.exit(1) + self.args.append(os.getcwd()) if len(self.args) > 1: logger.error(error_prefix + ": must pass only one CVS repository.\n") diff --git a/cvs2svn_lib/git_run_options.py b/cvs2svn_lib/git_run_options.py index e470758..586c560 100644 --- a/cvs2svn_lib/git_run_options.py +++ b/cvs2svn_lib/git_run_options.py @@ -40,7 +40,7 @@ class GitRunOptions(DVCSRunOptions): synopsis = """\ .B cvs2git -[\\fIOPTION\\fR]... [\\fIOUTPUT-OPTIONS] CVS-REPOS-PATH\\fR +[\\fIOPTION\\fR]... [\\fIOUTPUT-OPTIONS] [CVS-REPOS-PATH]\\fR .br .B cvs2git [\\fIOPTION\\fR]... \\fI--options=PATH\\fR @@ -67,7 +67,7 @@ top level directory of a CVS repository; it can point at a project within a repository, in which case only that project will be converted. This path or one of its parent directories has to contain a subdirectory called CVSROOT (though the CVSROOT directory can be -empty). +empty). If not specified, it defaults to the current directory. .P It is not possible directly to convert a CVS repository to which you only have remote access, but the FAQ describes tools that may be used diff --git a/cvs2svn_lib/hg_run_options.py b/cvs2svn_lib/hg_run_options.py index f64738e..3c3d428 100644 --- a/cvs2svn_lib/hg_run_options.py +++ b/cvs2svn_lib/hg_run_options.py @@ -26,13 +26,12 @@ class HgRunOptions(DVCSRunOptions): synopsis = """\ .B cvs2hg -[\\fIOPTION\\fR]... \\fIOUTPUT-OPTION CVS-REPOS-PATH\\fR +[\\fIOPTION\\fR]... \\fIOUTPUT-OPTION [CVS-REPOS-PATH]\\fR .br .B cvs2hg [\\fIOPTION\\fR]... \\fI--options=PATH\\fR """ - # XXX paragraph 2 copied straight from svn_run_options.py long_desc = """\ Create a new Mercurial repository based on the version history stored in a CVS repository. Each CVS commit will be mirrored in the Mercurial @@ -40,13 +39,15 @@ repository, including commit time and author (with optional remapping to Mercurial-style long usernames). .P \\fICVS-REPOS-PATH\\fR is the filesystem path of the part of the CVS -repository that you want to convert. It is not possible to convert a -CVS repository to which you only have remote access; see the FAQ for -more information. This path doesn't have to be the top level -directory of a CVS repository; it can point at a project within a -repository, in which case only that project will be converted. This -path or one of its parent directories has to contain a subdirectory -called CVSROOT (though the CVSROOT directory can be empty). +repository that you want to convert. This path doesn't have to be the +top level directory of a CVS repository; it can point at a project +within a repository, in which case only that project will be +converted. This path or one of its parent directories has to contain +a subdirectory called CVSROOT (though the CVSROOT directory can be +empty). If not specified, it defaults to the current directory. +.P +It is not possible to convert a CVS repository to which you only have +remote access; see the FAQ for more information. .P Unlike CVS or Subversion, Mercurial expects each repository to hold one independent project. If your CVS repository contains multiple diff --git a/cvs2svn_lib/svn_run_options.py b/cvs2svn_lib/svn_run_options.py index ca1a06d..147298e 100644 --- a/cvs2svn_lib/svn_run_options.py +++ b/cvs2svn_lib/svn_run_options.py @@ -101,7 +101,7 @@ class SVNRunOptions(RunOptions): synopsis = """\ .B cvs2svn -[\\fIOPTION\\fR]... \\fIOUTPUT-OPTION CVS-REPOS-PATH\\fR +[\\fIOPTION\\fR]... \\fIOUTPUT-OPTION [CVS-REPOS-PATH]\\fR .br .B cvs2svn [\\fIOPTION\\fR]... \\fI--options=PATH\\fR @@ -114,7 +114,10 @@ repository, including such information as date of commit and id of the committer. .P \\fICVS-REPOS-PATH\\fR is the filesystem path of the part of the CVS -repository that you want to convert. It is not possible to convert a +repository that you want to convert. If not specified, it defaults +to the current directory. +.P +It is not possible to convert a CVS repository to which you only have remote access; see the FAQ for more information. This path doesn't have to be the top level directory of a CVS repository; it can point at a project within a -- 1.7.9.5 ------=_Part_2939_1598092016.1357597159775--