RFC 14 (v2) Modify open() to support FileObjects and

This and other RFCs are available on the web at
  http://dev.perl.org/rfc/

=head1 TITLE

Modify open() to support FileObjects and Extensibility

=head1 VERSION

   Maintainer: Nathan Wiger <[email protected]>
   Date: 04 Aug 2000
   Version: 2
   Mailing List: [email protected]
   Number: 14

=head1 ABSTRACT

Currently, C<open()>, C<opendir()>, C<sysopen()>, and other file open
functions are given handle arguments, whose values are twiddled if a
filehandle can be created:

   open(HANDLE, "<$filename");
   open PIPE, "| $program";
   open my $fh, "<$filename";
   opendir DIR, $dirname;
   sysopen(HANDLE, $filename, O_RDWR|O_CREAT, 0666);

There are several problems with this approach:

   1. The calling style is uncharacteristic of other Perl funcs
   2. There is no way to support a list of return values
   3. There is no way to overload or extend them

In order to make these functions more consistent with other constructor
like functions (i.e. new(), etc), they should be changed to instead
return B<first-class fileobjects>:

   $fo = open "<$filename" or die;
   $po = open "|$program" or die;
   $do = opendir $dirname or die;

This would make these functions more internally consistent within Perl,
as well as allowing for the power of true B<fileobjects> and the
extensibility of C<open()>.

=head1 DESCRIPTION

=head2 Overview

First, this RFC assumes that B<fileobjects> will be $ single-whatzitz
(thanks Tom) types, which seems to have reached an overall informal
consensus.

As many have observed, the current filehandle mechanism is insufficient
and largely "a historial accident". The goal of the redesign of file
handles into full-fledged B<fileobjects> is to make them as flexible and
powerful as other objects within Perl, which still retaining a means to
interact with them simply. Since we are redesigning filehandles to be
true B<fileobjects>, we should revise their constructor functions as
well, returning B<fileobjects> and providing extensibility.

As shown above, in the simplest case this would change the C<open()>
function to:

   $fo = open $filename or die;

If successful, C<open()> and its relatives will return B<fileobjects>.
On failure, they will return undef. This still allows the user to test
the return value of C<open()> (as shown above) by checking for a "true"
(handle) or "false" (undef) condition.

=head2 New Syntax of C<open()>

The syntax of the C<open()> function would be changed as follows:

   $fileobject, [ @params ] = open $file, [ $class ], [ @args ];

Before examining C<opendir>, C<sysopen>, and others, let's examine this
syntax more closely.

   $fileobject  -  Replacement object for current filehandles.

   @params      -  Optional parameters that may be returned in a list
                   context. These may be things such as the owner for
                   a true file, or the content-type for a web document.

   $file        -  File to open. This might be a real file or directory,
                   but might also be a website, port for a socket, or
                   ftp server.

   $class       -  The class from which to load the appropriate file
                   methods, the default being an optimized IO::File.

   @args        -  Optional arguments to pass to the class's C<open()>.

The C<open()> function, as I propose it, is an overloaded and extensible
function that differs from other constructors in that it returns a valid
B<fileobject>. This object can then be used in C<read>, C<print>, and
other such file functions. 

=head2 Simple Scalar Form

In the simplest, "looks like Perl 5" form, C<open()> can take one file
parameter, which is then opened per the descriptor provided and the
corresponding B<fileobject> returned. Here are some examples (note that
C<my> has been left out for clarity):

   # Read from a file
   $fo = open "</etc/passwd" or die;
   print while (<$fo>);
   close $fo;

   # Write a file to a pipe
   $mailpipe = open "|/usr/lib/sendmail" or die;
   ($motd, $owner) = open "</etc/motd";       # return owner in list
   die unless $owner == 0;                    # owner not root
   while (<$motd>) {
      print $mailpipe;
   }
   close $motd;

   # Go fork yourself
   ($myself, $pid) = open "-|" or exec 'ls';  # return PID in list
   print while (<$myself>);
   close $myself;   # not myself anymore, hah! ;-)

In addition, the C<$file> argument becomes optional in this new syntax.
If not supplied, it defaults to C<$_>, making it consistent with other
Perl functions:

   for (@filenames) {
      my $fo = open;      
      push @open_handles, $fo;
   }

   # ... stuff happens ...

   for (@open_handles) {
      close;
   }

Perhaps this specific example is ugly (and useless), but there are
probably other situations in which one could take advantage of this.

=head2 True First-Class FileObjects

One major limitation of Perl's current filehandles is that they are
bareword scalars, with no object properties or power. The redesign of
simple filehandles into first-class B<fileobjects> allows us to give
them full object-oriented power, while still allowing them to be used
in a simple manner as shown above. Each object can contain methods to
allow us to access features of that B<fileobject> much more efficiently.

Here are some proposed default accessor methods of B<fileobjects>. Each
of these would return the appropriate value, or undef if not available.
This is a brief listing; the intent would be to support all of the
current C<FileHandle> methods and then some.

   $fo->filename  -  Name of the file, web document, port, etc
   $fo->type      -  One of 'pipe', 'file', 'ipc', etc, ala want()
   $fo->mode      -  Way the file was opened (|,<,>+,etc)
   $fo->fileno    -  System file number
   $fo->dup       -  Returns a duplicate of the current B<fileobject>
   $fo->pid       -  Return current PID of the process (if pipe/fork)

In addition, these functions would allow you to modify key elements of
the B<fileobject>:

   $fo->autoflush -  Sets buffer flushing policy
   $fo->untaint   -  Removes tainting from that data source
   $fo->options   -  Some syscalls, like C<socket()>, allow you to set
                     options which affect the handling of C<$fo>

If we decide that B<fileobjects> should be persistent across C<close()>
operations, we could define the following functions:

   $fo->open      -  Object methods to open/close C<fileobjects>
   $fo->close
   $fo->is_open   -  1 or undef, depending on the state of the object
   $fo->is_closed

Why would a B<fileobject> be persistent across C<close()> operations? If
it contains lots of properties, it may be a waste if we simply want to
close it to make sure buffers are flushed or bandwidth is not wasted on
TCP connections. We could use C<close()> to flush buffers and tidy
things up, but not destroy the object until the end of the script or
C<undef $fo> was called (similar to C<FileHandle>).

=head2 Extensible Class Bindings

In addition to the standard file form, C<open()> can also now take an
optional class name from which to load the appropriate methods. This
gives us easy access to methods that open Directory, Socket, HTTP, FTP,
or other types of files, meaning we no longer have to start from the
ground up every time we want to open a new type of "file".

Here are some examples which could be use to provide essentially native
file access for many different media:

   # Open a file with IO::File
   # This is used as the default class so doesn't have to be specified
   $motd = open ">/etc/motd" or die;
   print $motd @data;
   close $motd;

   # Open a directory with IO::Dir
   # Note opendir() could just be a synonym for this
   $dir = open "/usr/bin", IO::Dir or die;
   @files = grep !/^\..*/, <$dir>;
   close $dir;        # closedir() no longer needed

   # Open a client socket with IO::Socket
   # Note socket() could just be a synonym for this
   # By overloading < and > we can do clients and servers!
   $socket = open "< 25", IO::Socket, PF_INET, SOCK_STREAM, TCP;
   @input = <$socket>;
   close $socket;
   do_something(@input);

   # Open a remote webpage
   $http = open "http://www.perl.com/", HTTP::Request, GET;
   @doc = <$http>;
   print @doc if $http->content_type eq 'text/html';
   close $http;

   # Open an ftp connection
   $ftp = open "ftp.perl.com", Net::FTP;
   $ftp->cwd('CPAN') or die;
   @files = <$ftp>;          # overloading as dir
   close $ftp; 

Many will notice that this implementation looks similar to C<tie()> or
as a special type of C<new()>. This is true; I could see that this new
C<open()> may even be implemented as a special type of C<tie()>. The
distinction here is that C<open()> will return a valid B<fileobject>,
which can then be manipulated by Perl's file methods, regardless of its
actual physical characteristics. This could lead to great optimizations,
unlike C<tie()> which can't be optimized at all because nothing's known
ahead of time.

=head1 IMPLEMENTATION

The open() functions would have to be altered to accept this new syntax
and return full-fledged B<fileobjects>. The functions C<opendir()>,
C<sysopen()>, C<socket()>, and possibly many others must be rewritten
to serve as "shortcuts" to the new extensible C<open()>.

The close() function would remain unchanged, acting on the B<fileobject>
(or C<$_> if none is specified).

Because all objects would be instantiated as B<fileobjects>, a separate
C<closedir()> would be unnecessary and should be removed.  The
C<readdir()> function should also be removed.

In order to prevent performance hits, anything that is packaged as a
default file type (such as files, pipes, directories, sockets, ipc,
and so on) must be highly optimized for interaction with this new
version of C<open()>. Basically, anything living under the C<IO::>
tree should be ripped apart and put back together again, or replaced
with a new version under C<Open::> or something similar.

With regards to now-obsolete functions (such as C<opendir>, C<sysopen>,
C<socket>, and many more), we should make one of two choices:

   1. Make them shortcuts to the long form
   2. Remove them outright

If we choose #2 (I think #1 might actually be the right answer), we
should probably make nicknames for these modules, such as "dir" for
IO:Dir, "socket" for IO::Socket, and so forth, so a person can type:

   $dir  = open "/usr/bin", dir;   # instead of IO::Dir

Finally, to make the full, extensible class form useful, we will have to
figure out what type of modular hooks it will have. They may look like
C<tie()>, with hooks for every operation, or they may be completely
different, just requiring an C<OPEN()> function in the module. If all
modules agreed to return a consistent object from this C<OPEN()> call,
then the file routines could potentially be highly-optimized, since
there would only have to be one core set. If possible, this is what I'd
like to see.

=head1 REFERENCES

RFC 33: Eliminate bareword filehandles
RFC 30: STDIN, STDOUT, and STDERR should be renamed 
Tom Christiansen's great analysis of file object methods
Tim Jenness's suggestion to use optimized IO objects for all I/O