Re: Handling of warnings from cvs (cvs checkout: Skipping `$Log$' keyword due to excessive comment leader)

Michael Haggerty wrote:
> On 11/15/2010 10:15 AM, Jonathan Nieder wrote:

>> That took 33 hours.  Is it possible to import a single module instead?
>
> Yes, you can import single modules.  Just point cvs2git at the
> subdirectory of the CVS repository corresponding to the module that you
> want to convert.

Perfect.  I think the first time I tried that I hadn't downloaded the
CVSROOT yet.  Sorry for the noise.

>> Failing that, is it possible to resume?
>
> It is impossible to resume a conversion in such a way that only a single
> module is reprocessed.

That's useful to know.

>> Should I be using --use-external-blob-generator instead?
>
> I usually recommend it, as it is *vastly* faster than using CVS.
> However, be aware that it doesn't do anything with the "$Log$" keyword,
> so if you rely on that keyword being expanded correctly you cannot use
> that option.  (Even so, it might be useful for testing.)

Not expanding is great.

Side notes:

It crashes unless I redirect output to a seekable file.

 ----- pass 4 (FilterSymbolsPass) -----
 Filtering out excluded symbols and summarizing items...
 Starting generate_blobs.py...
 [...]
   File "/tmp/cvs2svn/trunk/cvs2svn_lib/generate_blobs.py", line 90, in write_blob
     f.seek(0, 2)
 IOError: [Errno 29] Illegal seek

I can avoid that by redirecting output to a seekable file, but then
fast-import crashes.

Documentation patch below.

> Hmmm, the file doesn't look corrupt after all.  And (contrary to my
> first guess) the problem is not triggered by the unusual characters in
> the comment line (which, for the record, looks like this:
>
>     comment	@.\" @;
>
> ).  In fact, it seems to be caused by some lines in the file text that
> contain the "$Log$" keyword with a lot of text preceding it.  Judging
> from the file text, I suspect that this file was not intended to have
> its keywords expanded.  So you might consider fixing the problem by
> marking the file as binary (with "cvs admin -kb").

Yes, I think that will do it.  I suspect the only reason keyword
expansion is not disabled already is that the file was deleted in
1993, a few years before -kb worked.

> I'm glad we could help the NetBSD project.

To be clear, I've never worked with the NetBSD project before.  I'll
be happy if my experiments help.

Thanks.
Jonathan
-- 8< --
Subject: cvs2git: clarify documentation of --use-external-blob-generator

The blob generator runs without heavy supervision by the main cvs2git
script, which is great for parallelism.  The only gotcha is that the
blob generator and main cvs2git need access to separate parts of the
fast-import stream, which breaks the pattern

	cvs2git <options> | git fast-import

that was great for parallelism.  Document this requirement (blobs
and revisions must go to different files) to avoid confusion.

Can we salvage the parallelism by streaming blobs directly to
fast-import and queuing up revs in a file?  Alas, no: the blob
generator seeks around in its output file to be able to read back old
blobs.  Document that requirement, too (the blob file cannot be a
pipe).

Balance out the new cautionary notes by removing an old one: the
--use-external-blob-generator code is solid and no longer deserves
the fearsome "experimental" tag.

Signed-off-by: Jonathan Nieder <[email protected]>
---
 There is a "git fast-import" patch in flight that would allow
 removing that second requirement by asking fast-import for a
 reminder about blobs past.

 The last discussed iteration of that patch is
 http://thread.gmane.org/gmane.comp.version-control.git/159117/focus=159116

 and there is there is also a cleaned-up version available:
 http://repo.or.cz/w/git/jrn.git/shortlog/refs/topics/db/fast-import-cat-blob?hp=593ce2bea

 Thoughts welcome.

diff --git a/cvs2svn_lib/git_run_options.py b/cvs2svn_lib/git_run_options.py
index c506e9e..2b40106 100644
--- a/cvs2svn_lib/git_run_options.py
+++ b/cvs2svn_lib/git_run_options.py
@@ -122,16 +122,19 @@ A directory called \\fIcvs2svn-tmp\\fR (or the directory specified by
         '--use-external-blob-generator',
         action='store_true',
         help=(
-            'EXPERIMENTAL -- use an external Python program to extract file '
-            'revision contents (much faster than --use-rcs or --use-cvs but '
-            'not yet well tested)'
+            'Use an external Python program to extract file revision '
+            'contents (much faster than --use-rcs or --use-cvs but '
+            'leaves keywords unexpanded and requires a separate, '
+            'seekable blob file to write to in parallel to the main '
+            'cvs2git script.'
             ),
         man_help=(
-            'EXPERIMENTAL -- Use an external Python program to extract the '
-            'file revision contents from the RCS files and output them to '
-            'the blobfile.  This option is much faster than '
-            '\\fB--use-rcs\\fR or \\fB--use-cvs\\fR but is still '
-            'experimental.'
+            'Use an external Python program to extract the file revision '
+            'contents from the RCS files and output them to the blobfile.  '
+            'This option is much faster than \\fB--use-rcs\\fR or '
+            '\\fB--use-cvs\\fR but leaves keywords unexpanded and requires '
+            'a separate, seekable blob file to write to in parallel to the '
+            'main cvs2git script.'
             ),
         ))

------------------------------------------------------
http://cvs2svn.tigris.org/ds/viewMessage.do?dsForumId=1667&dsMessageId=2681997

To unsubscribe from this discussion, e-mail: [[email protected]].