Re: wget LFS patch

Jim Wright <[email protected]>
Newsgroups gmane.comp.web.wget.patches
Message-ID <[email protected]>
Here is a small, ugly patch.  I intended to clean this up and submit it
long ago but so much for intentions.  With the short deadline for feature
freeze, I want to get this in ASAP.  This patch is against the second
release of wget-1.10-alpha1.  This enhances wget so that when retrieving a
file using FTP it will not leave a partial file behind in case of errors.

This is critical for my application, where I am doing an hourly mirror of
1,500 FTP servers.  Any newly downloaded file is processed.  Because wget
doesn't have terribly useful exit codes when mirroring, there is no
easy way to distinguish a complete download from a partial download.
Thus my solution is to never allow partial downloads to remain.

A proper patch would include a command line option --no-partial-ftp
to control this behavior.  A proper patch would include an update to
the documentation.  A proper patch would include a change log entry.
I can do this or others can contribute, once this idea is accepted.

Thanks for your consideration.

Jim Wright
Data Flow/Archive Software Engineer
Plate Boundary Observatory
no-partials.patch (text/plain, 880 B)
--- wget-1.10-alpha1/src/ftp.c	2005-04-01 17:41:04.000000000 -0700
+++ wget-1.10-alpha1-JRW/src/ftp.c	2005-04-05 14:50:29.475179982 -0600
@@ -1246,6 +1246,9 @@
 	case HOSTERR: case CONIMPOSSIBLE: case FWRITEERR: case FOPENERR:
 	case FTPNSFOD: case FTPLOGINC: case FTPNOPASV: case CONTNOTSUPPORTED:
 	  /* Fatal errors, give up.  */
+	  /* don't leave a partial file!  should be conditional on command switch */
+	  logprintf (LOG_NOTQUIET, "Removing partial file: %s\n", locf);
+	  unlink(locf);
 	  return err;
 	  break;
 	case CONSOCKERR: case CONERROR: case FTPSRVERR: case FTPRERR:
@@ -1354,6 +1357,9 @@
       fd_close (con->csock);
       con->csock = -1;
     }
+  /* don't leave a partial file!  should be conditional on command switch */
+  logprintf (LOG_NOTQUIET, "Removing partial file: %s\n", locf);
+  unlink(locf);
   return TRYLIMEXC;
 }
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.