Re: VMS patch set (extensive) - first try (almost)

From: Hrvoje Niksic <[email protected]>

> > I don't mean to imply anything about the cause.  I merely note that
> > removing this (gratuitous) fflush() changes:
> [...]
> 
> Point taken, except for fflush being gratuitous -- it serves a useful
> purpose.

   I'd debate "useful".

> > More likely, the VMS/RMS/C-RTL combination takes fflush() seriously,
> > and that calling it for every 16KB buffer probably renders useless
> > all the disk caching and multiple buffering which would otherwise
> > improve the file write speed.
> 
> That would be just weird.  fflush is simply supposed to force a write
> of the user-space-buffered data, not to call sync or something like
> that.

   Weird or not, there it is.  For what it's worth, the C RTL docs do
say:

      The fflush function flushes the HP C RTL buffers. However, RMS has
      its own buffers. The fflush function does not guarantee that the
      file will be written to disk. (See the description of fsync for a
      way to flush buffers to disk.)

      If the file pointed to by file_ptr was opened in record mode and
      if there is unwritten data in the buffer, then fflush always
      generates a record.

And the current opens should be specifying stream mode rather than
record mode.  (If I were a paying customer, I might log a formal
complaint/inquiry.)

> It's not only about ls -l/du beforehand.  When you press ^C, I'd like
> the file to contain the data Wget told me it received.  Not flushing
> data regularly can, in a sense, lose data.

   You're joking, right?  For a moment I thought that you might actually
be serious about this.  I know that I'd be surprised if anyone running
VMS would expect anything in particular at the end of a file being
written by a program which was interrupted asynchronously.  (Ha-ha. 
Good one.  First a speech about sync, then a joke about losing data from
CTRL/C.  And I fell for it.)

> >> I don't think it makes sense for this to be OS-dependent.
> >
> > I don't think that it makes any sense to do the fflush().  If you
> > do, then it certainly makes no sense to do it on VMS, where it does
> > actual (and easily measurable) harm.  Perhaps I'm just
> > old-fashioned, but I always thought that fflush() was intended for
> > cases where someone intended to look at the written data, and I
> > expect it to hurt performance.
> 
> It hurts performance if you were to do it after every byte or
> something like that.  But Wget does it after every received *packet*,
> which is IMHO quite reasonable.
> 
> > Has anyone tried a similar test on any other OS?
> 
> Not that I know of.

   A quick test on Tru64 suggests that it makes little or no difference
there.  So, perhaps it _does_ make sense for this to be OS-dependent.

   And while fooling around with this, I was warned (by gcc) about some
embarassing typos in my ftp-ls.c, so, there's a suplementary patch file,
"src/ftp-ls.c_patch2" at:

      http://antinode.org/ftp/wget/patch1/
      ftp://antinode.org/wget/patch1/

------------------------------------------------------------------------

   Steven M. Schweda               (+1) 651-699-9818
   382 South Warwick Street        sms@antinode-org
   Saint Paul  MN  55105-2547