Re: Re: scgi server start

"""Daemonizer with command-line front end; with optional Quixote support.

This module provides generic functions to daemonize and change user, a
TopLevel class to manage the command-line options for these, and a
QuixoteTopLevel class for SCGI and SimpleServer applications.  Many Linux
distributions come with a start-stop-daemon program that does most of this, but
Mac OS X does not, hence the need for a Python version.

At the lowest level is the daemonize() function by Chad Schroeder, borrowed
from the Python cookbook.  

TopLevel gives your application the following command-line options:
   -h, --help        Show this help message and exit
  --daemon           Run as system service (daemon) (requires --pidfile)
  --stop             Stop a running daemon (requires --pidfile)
  --pidfile=PIDFILE  Record daemon PID in this file
  --user=USER        Run as this user (name or UID)
  --group=GROUP      Run as this group (name or GID)

QuixoteTopLevel adds the following options
  --simple           Run under the simple server
  --scgi             Run under the SCGI server

GENERIC USAGE ==============================================================
    import toplevel
    opts, args = toplevel.TopLevel().main()
    #
    # We may have forked or changed user.  You can examine 'opts' if you
    # care.  Depending on the OS, any open files or database connections may
    # have been closed by the fork.
    #
    # Put your application-specific code here.

If you need additional command-line options, subclass TopLevel and override
.get_parser() and perhaps .check_usage() .  Remember to call the superclass
methods!

Functions change_user(), daemon(), stop(), daemonize(), etc. can be used
standalone.

QUIXOTE USAGE ==========================================================
    import toplevel
    from quixote.publish import Publisher
    from my_directory import MyDirectory
    class MyTopLevel(toplevel.QuixoteTopLevel):
        simple_host = '127.0.0.1'
        scgi_max_children = 1
        # Override other class attributes as needed.

        def create_publisher(self):
            app = MyDirectory()
            return Publisher(app)
            # You could return a different publisher depending on self.opts.

    MyTopLevel.main()

NOTES ON QuixoteTopLevel CLASS ATTRIBUTES =================================

.simple_host and .scgi_host should be '' to listen on all interfaces.  Or
specify the domain or IP to listen on one interface.  Note that some versions
of Mac OS X do not recognize "localhost", use "127.0.0.1" instead.

If you are using mod_scgi's SCGIMount directive (recommended), you can leave
.scgi_script_name at the default.  If you're using SCGIServer,
.scgi_script_name **must** match the <Location> directive the SCGIServer is
contained in.  Otherwise your application won't work.

QuixoteTopLevel will raise RuntimeError if the user chooses --scgi and
.scgi_max_children is greater than 1.  If you know your application is
multiprocess safe, you can override .is_multiprocess_safe() and return True.
Although multiple children do not have the shared-memory problem of threads,
you can still get into trouble with files or sessions.  Use file locking if you
write to any files during a request.  Quixote's default session store (a
dictionary) is not safe: each child has its own dictionary, and requests go to
different children at random.  If the application keeps "forgetting" your
session, this is probably why.  The session2 package at
http://quixote.idyll.org/session2/ has multiprocess-safe persistent session
stores using the filesystem, Durus, MySQL, PostgreSQL, and others.  

If .simple_banner is true, a message is printed on stdout telling which URL
browsers should connect to.

I've never used .simple_https so I'm not sure where you put the socket into SSL
mode.  I'm also not sure the simple_banner message is correct in this case.

QuixoteTopLevel is provided as a convenience.  It may not be suitable for all
Quixote applications.

CHANGELOG:
- 2005-08-31: add os._urandomfd kludge.  Docstring. 
- 2005-09-22: add TopLevel and QuixoteTopLevel classes, remove corresponding public
  functions and the esoteric 'config' object.

KLUDGES:
- After the daemonize() call in daemon(), do "os._urandomfd = None".  This is to
  avoid a "bad file descriptor" exception (observed on Mac OS X).
  Apparently Python or Quixote calls os.urandom() before the fork, and the file
  descriptor for /dev/urandom is invalid in the child, so we have to make the
  'os' module reopen the file.
"""
import errno, grp, os, pwd, signal, sys
try:
    import grp, pwd
except ImportError:  # Modules not available on Windows.
    grp = pwd = None
from optparse import OptionParser
from quixote.publish import Publisher

#### GENERIC FRONT END FOR APPLICATIONS ####
class TopLevel(object):
    usage = None
    description = None

    def main(self):
        parser = self.get_parser()
        self.opts, self.args = opts, args = parser.parse_args()
        errmsg = self.check_usage()
        if errmsg:
            print "ERROR:", errmsg
            parser.print_help()
            sys.exit(1)
        if hasattr(opts, "user"):
            change_user(opts.user, opts.group)
        if opts.stop:
            stop(opts.pidfile)    # Exits program, does not return.
        if opts.daemon:
            daemon(opts.pidfile)
        return opts, args
        
    def get_parser(self):
        parser = OptionParser(usage=self.usage, description=self.description)
        self.add_parser_options(parser)
        return parser

    def add_parser_options(self, parser):
        pao = parser.add_option
        pao('--daemon', action="store_true", dest="daemon",
            help="Run as system service (daemon) (requires --pidfile)")
        pao('--stop', action="store_true", dest="stop",
            help="Stop a running daemon (requires --pidfile)")
        pao('--pidfile', action="store", dest="pidfile",
            help="Record daemon PID in this file")
        if pwd:
            pao('--user', action="store", dest="user",
                help="Run as this user (name or UID)")
        if grp:
            pao('--group', action="store", dest="group",
                help="Run as this group (name or GID)")

    def check_usage(self):
        opts = self.opts
        if opts.daemon and opts.stop:
            return "can't specify both --daemon and --stop"
        if opts.daemon and not opts.pidfile:
            return "--daemon requires --pidfile"
        if opts.stop and not opts.pidfile:
            return "--stop requires --pidfile"
        if opts.stop and opts.pidfile and not os.path.exists(opts.pidfile):
            return "--pidfile refers to non-existent file"
        return None

#### FRONT END FOR QUIXOTE APPLICATIONS ####
class QuixoteTopLevel(TopLevel):
    scgi_host = '127.0.0.1'
    scgi_port = 3000
    scgi_script_name = None
    scgi_max_children = 1

    simple_host = ''
    simple_port = 8080
    simple_https = False
    simple_banner = True

    title = "Quixote application"

    def main(self):
        super(QuixoteTopLevel, self).main()
        if self.opts.scgi:
            self.run_scgi()
        else:
            self.run_simple()

    def run_scgi(self):
        from quixote.server import scgi_server
        if self.scgi_max_children > 1 and not self.is_multiprocess_safe():
            m = "configured application is unsafe for multiple SCGI children"
            raise RuntimeError(m)
        scgi_server.run(self.create_publisher, 
            host=self.scgi_host, 
            port=self.scgi_port,
            script_name=self.scgi_script_name, 
            max_children=self.scgi_max_children,
            )

    def run_simple(self):
        from quixote.server import simple_server
        if self.simple_banner:
            title = self.title
            scheme = self.simple_https and "https" or "http"
            host = self.simple_host or "[ALL INTERFACES]"
            port = self.simple_port
            print "%s listening on %s://%s:%s/" % (title, scheme, host, port)
        simple_server.run(self.create_publisher, 
            host=self.simple_host,
            port=self.simple_port,
            )

    def get_parser(self):
        parser = super(QuixoteTopLevel, self).get_parser()
        pao = parser.add_option
        pao('--simple', action="store_true", dest="simple", 
            help="Run under the simple server")
        pao('--scgi', action="store_true", dest="scgi", 
            help="Run under the SCGI server")
        return parser

    def check_usage(self):
        errmsg = super(QuixoteTopLevel, self).check_usage()
        if errmsg:
            return errmsg
        opts = self.opts
        if (not opts.stop) and count_true(opts.scgi, opts.simple) != 1:
            return "must specify --simple or --scgi flag (but not both)"
        return None

    def create_publisher(self):
        raise NotImplementedError("subclass responsibility")

    def is_multiprocess_safe(self):
        return False


#### GENERIC FUNCTIONS ####
def count_true(*values):
    """Return the number of true values."""
    return len(filter(None, values))

#### USER-CHANGING FUNCTIONS ####
def change_user(user=None, group=None):
    # JJ Behrens says we must set the group first, so we do.
    if group is not None:
        assert grp is not None, "Can't change group on %r platform." % sys.platform
        if group.isdigit():
            gid = int(group)
        else:
            gid = grp.getgrnam(group)[2]
        os.setegid(gid)
    if user is not None:
        assert grp is not None, "Can't change user on %r platform." % sys.platform
        if user.isdigit():
            uid = int(user)
        else:
            uid = pwd.getpwnam(user)[2]
        os.seteuid(uid)

#### DAEMON FUNCTIONS ####
def daemon(pidfile):
    if os.path.exists(pidfile):
        pid = read_pidfile(pidfile)
        if process_exists(pid):
            msg = "daemon already running (PID %s, file %s)" % (pid, pidfile)
            sys.exit(msg)
    daemonize()
    os._urandomfd = None  # @@MO: Workaround for Mac OS X (see module docstring).
    write_pidfile(pidfile)

def stop(pidfile):
    pid = read_pidfile(pidfile)
    try:
        os.kill(pid, signal.SIGTERM)
    except OSError, e:
        if e.errno == errno.ESRCH: # No such process.
            sys.exit(1)   
        else:
            raise
    # Signal delivered, delete pidfile and exit program.
    os.remove(pidfile)
    sys.exit(0)     

#### DAEMON UTILITY FUNCTIONS ####
def read_pidfile(pidfile):
    f = open(pidfile, 'r')
    pid = f.read()
    f.close()
    try:
        pid = int(pid)
    except ValueError:
        msg = "corrupt PID file %s (content not numeric)" % pidfile
        raise RuntimeError(message)
    return pid
        
def write_pidfile(pidfile):
    pid = os.getpid()
    f = open(pidfile, 'w')
    print >>f, pid
    f.close()

def process_exists(pid):
    try:
        os.kill(pid, 0)
    except OSError, e:
        if e.errno == errno.ESRCH:
            return False
        else:
            raise
    return True

def daemonize():
    """Detach a process from the controlling terminal and run it in the
       background as a daemon.  A Python Cookbook recipe by Chad J Schroeder.
       http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/278731
       Original documentation follows.

       Disk And Execution MONitor (Daemon)
   
       Default daemon behaviors (they can be modified):
          1.) Ignore SIGHUP signals.
          2.) Default current working directory to the "/" directory.
       #  3.) Set the current file creation mode mask to 0.
          4.) Close all open files (0 to [SC_OPEN_MAX or 256]).
          5.) Redirect standard I/O streams to "/dev/null".
   
       [# indicates behaviors disabled in this version.]

       Failed fork() calls will return a tuple: (errno, strerror).  This
       behavior can be modified to meet your program's needs.
   
       Resources:
          Advanced Programming in the Unix Environment: W. Richard Stevens
          Unix Network Programming (Volume 1): W. Richard Stevens
          http://www.erlenstar.demon.co.uk/unix/faq_2.html#SEC16
    """
    try:
       # Fork a child process so the parent can exit.  This will return control
       # to the command line or shell.  This is required so that the new process
       # is guaranteed not to be a process group leader.  We have this guarantee
       # because the process GID of the parent is inherited by the child, but
       # the child gets a new PID, making it impossible for its PID to equal its
       # PGID.
       pid = os.fork()
    except OSError, e:
       return((e.errno, e.strerror))     # ERROR (return a tuple)
 
    if (pid == 0):       # The first child.
 
       # Next we call os.setsid() to become the session leader of this new
       # session.  The process also becomes the process group leader of the
       # new process group.  Since a controlling terminal is associated with a
       # session, and this new session has not yet acquired a controlling
       # terminal our process now has no controlling terminal.  This shouldn't
       # fail, since we're guaranteed that the child is not a process group
       # leader.
       os.setsid()
 
       # When the first child terminates, all processes in the second child
       # are sent a SIGHUP, so it's ignored.
       signal.signal(signal.SIGHUP, signal.SIG_IGN)
 
       try:
          # Fork a second child to prevent zombies.  Since the first child is
          # a session leader without a controlling terminal, it's possible for
          # it to acquire one by opening a terminal in the future.  This second
          # fork guarantees that the child is no longer a session leader, thus
          # preventing the daemon from ever acquiring a controlling terminal.
          pid = os.fork()        # Fork a second child.
       except OSError, e:
          return((e.errno, e.strerror))  # ERROR (return a tuple)
 
       if (pid == 0):      # The second child.
          # Ensure that the daemon doesn't keep any directory in use.  Failure
          # to do this could make a filesystem unmountable.
          os.chdir("/")
          # Give the child complete control over permissions.
          os.umask(0)
       else:
          os._exit(0)      # Exit parent (the first child) of the second child.
    else:
       os._exit(0)         # Exit parent of the first child.
 
    # Close all open files.  Try the system configuration variable, SC_OPEN_MAX,
    # for the maximum number of open files to close.  If it doesn't exist, use
    # the default value (configurable).
    try:
       maxfd = os.sysconf("SC_OPEN_MAX")
    except (AttributeError, ValueError):
       maxfd = 256       # default maximum
 
    for fd in range(0, maxfd):
       try:
          os.close(fd)
       except OSError:   # ERROR (ignore)
          pass
 
    # Redirect the standard file descriptors to /dev/null.
    os.open("/dev/null", os.O_RDONLY)    # standard input (0)
    os.open("/dev/null", os.O_RDWR)       # standard output (1)
    os.open("/dev/null", os.O_RDWR)       # standard error (2)
 
    return(0)