Tests fail on Windows

I'm trying to run the latest trunk on Windows using Python 2.6.2 and
there are some test failures.  Tests 68-75 and 90 fail due to
http://cvs2svn.tigris.org/issues/show_bug.cgi?id=124 .  Test 160 would
also fail due to this but it is XFailed.  Are there any ideas how to
address this?  It seems like it is going to become more of a problem
now that Subversion 1.6 is out.  Besides that issue there are several
other problems which are specific to Win32.  I've worked up a patch
which is attached.

1) Trying to call out to GNU sort always uses Windows sort even if GNU
sort is available and appears in the PATH before Windows sort.  The
subprocess module uses the Win32 CreateProcess which always looks in
the Windows system32 directory before it looks in the directories
listed in the PATH environment variable.  Since the Windows sort.exe
is in the system32 directory it will always be chosen.  A simple
workaround is to launch the sort in a shell.  When the shell (cmd.exe)
searches it only examines the directories in the PATH so putting the
directory with GNU sort ahead of the Windows system32 directory will
cause GNU sort to be chosen.

2) Symbol transforms aren't working.  In the the test for renaming
multiply-defined symbols the constructor for SymbolMapper is passed
Unix-style paths (see
test-data/multiply-defined-symbol-cvsrepos/cvs2svn-rename.options).
But the transform() method gets called with native paths.  I've tried
to fix this by converting the paths to native when the SymbolMapper is
constructed.  I've also made a similar change to SubtreeSymbolMapper
which is untested.

3) The symbol_transform test creates a directory with a very long
name.  Some files inside this directory eventually exceed the Windows
limit of 256 characters in an absolute path which causes the test to
fail.  I've tried to work around this by using a hash of the arguments
in the directory name instead of using them literally.  This keeps the
length down and preserves the existing guarantee that directory names
won't have illegal characters in them.  The downside is that the
directory names in the temp directory become somewhat harder to
decipher.

4) The run-tests.py tries to set things up so that svn will output
timestamps in UTC instead of using the local timezone.  The comment
says "I have no idea if this works on Windows".  Well it doesn't work
on Windows unfortunately.  Timestamps keep getting output in the local
timezone.  There doesn't seem to be any way to override this behavior
on Windows.  The timestamp_chaos test ends up failing with all the
times off by a constant number of hours.  It doesn't seem too hard to
handle timestamps in the local timezone and in fact the comments
already make reference to doing this.  They don't seem to correctly
describe what is going on now with svn configured to output UTC times.
 A small change to interpret the times that are hardcoded in
run-tests.py as UTC instead of local allows the test to run with svn
outputting local timestamps.  As a bonus this makes the existing
comments correct again.

-- 
James Abbatiello

------------------------------------------------------
http://cvs2svn.tigris.org/ds/viewMessage.do?dsForumId=1667&dsMessageId=2372170

To unsubscribe from this discussion, e-mail: [[email protected]].

Index: cvs2svn_lib/passes.py
===================================================================
--- cvs2svn_lib/passes.py	(revision 4814)
+++ cvs2svn_lib/passes.py	(working copy)
@@ -106,7 +106,8 @@
       ]
 
   try:
-    call_command(command, stdout=open(outfilename, 'w'))
+    call_command(command, stdout=open(outfilename, 'w'),
+                 shell=(sys.platform=='win32'))
   finally:
     if lc_all_tmp is None:
       del os.environ['LC_ALL']
Index: cvs2svn_lib/symbol_transform.py
===================================================================
--- cvs2svn_lib/symbol_transform.py	(revision 4814)
+++ cvs2svn_lib/symbol_transform.py	(working copy)
@@ -138,6 +138,7 @@
   def __setitem__(self, (cvs_filename, symbol_name, revision), new_name):
     """Set a mapping for a particular file, symbol, and revision."""
 
+    cvs_filename = cvs_filename.replace('/', os.sep)
     key = (cvs_filename, symbol_name, revision)
     if key in self._map:
       Log().warn(
@@ -185,6 +186,7 @@
       symbol_map = {}
       self._map[symbol_name] = symbol_map
 
+    cvs_path = cvs_path.replace('/', os.sep)
     if cvs_path in symbol_map:
       Log().warn(
           'Overwriting symbol transform for\n'
Index: run-tests.py
===================================================================
--- run-tests.py	(revision 4814)
+++ run-tests.py	(working copy)
@@ -43,6 +43,11 @@
 import os.path
 import locale
 import textwrap
+import calendar
+try:
+  from hashlib import md5
+except ImportError:
+  from md5 import md5
 from difflib import Differ
 
 # Make sure that a supported version of Python is being used:
@@ -439,16 +444,11 @@
 
   conv_id = name
 
-  _win32_fname_mapping = { '/': '_sl_', '\\': '_bs_', ':': '_co_',
-                           '*': '_st_', '?': '_qm_', '"': '_qq_',
-                           '<': '_lt_', '>': '_gt_', '|': '_pi_', }
-  for arg in args:
-    # Replace some characters that Win32 isn't happy about having in a
-    # filename (which was causing the eol_mime test to fail).
-    sanitized_arg = arg
-    for a, b in _win32_fname_mapping.items():
-      sanitized_arg = sanitized_arg.replace(a, b)
-    conv_id += sanitized_arg
+  # Under Win32 there are some characters that are forbidden in filenames
+  # and there is a limit on the total length of a path to a file
+  # Use a hash to avoid problematic characters and to keep the path short
+  if args:
+    conv_id += "-" + md5("".join(args)).hexdigest()
 
   if passbypass:
     conv_id += '-passbypass'
@@ -3145,7 +3145,8 @@
       '2007-01-01 22:00:00', # revision 1.3 of both files
       ]
   for i in range(len(times)):
-    if abs(conv.logs[i + 1].date - time.mktime(svn_strptime(times[i]))) > 0.1:
+    if abs(conv.logs[i + 1].date -
+           calendar.timegm(svn_strptime(times[i]))) > 0.1:
       raise Failure()
 
 
@@ -3651,7 +3652,6 @@
   # Configure the environment for reproducable output from svn, etc.
   # I have no idea if this works on Windows too.
   os.environ["LC_ALL"] = "C"
-  os.environ["TZ"] = "UTC"
 
   # The Subversion test suite code assumes it's being invoked from
   # within a working copy of the Subversion sources, and tries to use