Re: Pool Module Question (Now Proxy Server Question)

"Ben Ringold" <ben-3GbDDtM/[email protected]>
Newsgroups gmane.comp.python.spyce.general
Message-ID <[email protected]>
Thanks Conan and Jonathan for all the help.

Updating to Spyce 2.0.3 and installing SQLAlchemy required updating Python and MySQL and loading a few more libraries.  It all worked except I managed to destroy mod_python in the process.  I'm working on that problem, but in the meantime I've switched over to the proxy server method.

I've used the scripts Conan suggested with a few path changes (see attached modified files).  The performance seems great, considerably faster than the previous installation I had (spyce 1.3, mod_python with poorly pooled db connections).  However, the proxy server dies off and the daemon disappears every two hours or so.

I don't see anything useful in usiecr2.err, and usiecr2.log just has a few generic exception messages before the daemon disappears:

Exception happened during processing of request from ('XXX.XXX.XXX.XXX', YYYY)

I'm not sure if there are other logs that may have pertinent information, or perhaps there are changes I could make to those daemonization scripts to make it more robust.


Regards.

Ben Ringold
Hypothetical Software


---------- Original Message ----------------------------------
From: "Jonathan Ellis" <[email protected]>
Date:  Sat, 15 Apr 2006 09:51:29 -0700

>On Sat, 15 Apr 2006 08:16:29 -0600, "Conan C. Albrecht"
><conan-RXmsyT/Szrj2fBVCVOL8/[email protected]> said:
>> The "multiple instances of spyce, hence multiple instances of the  
>> pool"  problem is related to mod_python, not spyce.  Mod_python runs  
>> multiple instances of itself in an effort to increase load.  The side  
>> effect to this is you get multiple pools.
>
>With the 2.x versions, Spyce will pickle objects to disk (with locking)
>under mod_python to avoid multiple pools.  So you still get correct
>behavior; it's just slow. :)
>
>The advice to use a proxy server setup when using Pool is still good.
>
>-Jonathan
>-- 
>C++ is history repeated as tragedy. Java is history repeated as farce.  --Scott McKay
>
>
 




________________________________________________________________
Sent via the WebMail system at hypotheticalsoftware.com
run_daemon.py (text/plain, 841 B)
#!/usr/local/bin/python

import daemonize
import os, os.path, sys

LOG_DIR = '/var/log/usiecr2'
LOG_FILE = os.path.join(LOG_DIR, 'usiecr2.log')
ERR_FILE = os.path.join(LOG_DIR, 'usiecr2.err')
PID_FILE = '/var/run/usiecr2.pid'
CONF_FILE = '/usr/share/spyce/spyceconf.py'  # this dir CANNOT have a space in it or the split will fail below
SPYCE_DIR = '/usr/share/spyce'

# ensure the correct directories exist
if not os.path.exists(LOG_DIR):
  os.makedirs(LOG_DIR)

# first fork this process to a daemon
daemonize.startstop(stdout=LOG_FILE, stderr=ERR_FILE, pidfile=PID_FILE)

# change to the spyce directory
sys.path.append(SPYCE_DIR)
os.chdir(SPYCE_DIR)

# next call the spyce web server
# change argv to the new command we normally call to start the server
sys.argv = ('spyceCmd.py -l --conf ' + CONF_FILE).split(' ')
execfile(sys.argv[0])
daemonize.py (text/plain, 4.7 KB)
#!/usr/local/bin/python

'''
    This module is used to fork the current process into a daemon.
    Almost none of this is necessary (or advisable) if your daemon 
    is being started by inetd. In that case, stdin, stdout and stderr are 
    all set up for you to refer to the network connection, and the fork()s 
    and session manipulation should not be done (to avoid confusing inetd). 
    Only the chdir() and umask() steps remain as useful.
    References:
        UNIX Programming FAQ
            1.7 How do I get my program to act like a daemon?
                http://www.erlenstar.demon.co.uk/unix/faq_2.html#SEC16
        Advanced Programming in the Unix Environment
            W. Richard Stevens, 1992, Addison-Wesley, ISBN 0-201-56317-7.

    History:
      2001/07/10 by Jurgen Hermann
      2002/08/28 by Noah Spurrier
      2003/02/24 by Clark Evans
      
      http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66012
'''
import sys, os, time
from signal import SIGTERM

def deamonize(stdout='/dev/null', stderr=None, stdin='/dev/null',
              pidfile=None, startmsg = 'started with pid %s' ):
    '''
        This forks the current process into a daemon.
        The stdin, stdout, and stderr arguments are file names that
        will be opened and be used to replace the standard file descriptors
        in sys.stdin, sys.stdout, and sys.stderr.
        These arguments are optional and default to /dev/null.
        Note that stderr is opened unbuffered, so
        if it shares a file with stdout then interleaved output
        may not appear in the order that you expect.
    '''
    # Do first fork.
    try: 
        pid = os.fork() 
        if pid > 0: sys.exit(0) # Exit first parent.
    except OSError, e: 
        sys.stderr.write("fork #1 failed: (%d) %s\n" % (e.errno, e.strerror))
        sys.exit(1)
        
    # Decouple from parent environment.
    os.chdir("/") 
    os.umask(0) 
    os.setsid() 
    
    # Do second fork.
    try: 
        pid = os.fork() 
        if pid > 0: sys.exit(0) # Exit second parent.
    except OSError, e: 
        sys.stderr.write("fork #2 failed: (%d) %s\n" % (e.errno, e.strerror))
        sys.exit(1)
    
    # Open file descriptors and print start message
    if not stderr: stderr = stdout
    si = file(stdin, 'r')
    so = file(stdout, 'a+')
    se = file(stderr, 'a+', 0)
    pid = str(os.getpid())
    sys.stderr.write("\n%s\n" % startmsg % pid)
    sys.stderr.flush()
    if pidfile: file(pidfile,'w+').write("%s\n" % pid)
    
    # Redirect standard file descriptors.
    os.dup2(si.fileno(), sys.stdin.fileno())
    os.dup2(so.fileno(), sys.stdout.fileno())
    os.dup2(se.fileno(), sys.stderr.fileno())

def startstop(stdout='/dev/null', stderr=None, stdin='/dev/null',
              pidfile='pid.txt', startmsg = 'started with pid %s' ):
    if len(sys.argv) > 1:
        action = sys.argv[1]
        try:
            pf  = file(pidfile,'r')
            pid = int(pf.read().strip())
            pf.close()
        except IOError:
            pid = None
        if 'stop' == action or 'restart' == action:
            if not pid:
                mess = "Could not stop, pid file '%s' missing.\n"
                sys.stderr.write(mess % pidfile)
                if 'stop' == action:
                    sys.exit(1)
                action = 'start'
                pid = None
            else:
               try:
                  while 1:
                      os.kill(pid,SIGTERM)
                      time.sleep(1)
               except OSError, err:
                  err = str(err)
                  if err.find("No such process") > 0:
                      os.remove(pidfile)
                      if 'stop' == action:
                          sys.exit(0)
                      action = 'start'
                      pid = None
                  else:
                      print str(err)
                      sys.exit(1)
            time.sleep(5)

        if 'start' == action:
            if pid:
                mess = "Start aborded since pid file '%s' exists.\n"
                sys.stderr.write(mess % pidfile)
                sys.exit(1)
            deamonize(stdout,stderr,stdin,pidfile,startmsg)
            return
    print "usage: %s start|stop|restart" % sys.argv[0]
    sys.exit(2)

def test():
    '''
        This is an example main function run by the daemon.
        This prints a count and timestamp once per second.
    '''
    sys.stdout.write ('Message to stdout...')
    sys.stderr.write ('Message to stderr...')
    c = 0
    while 1:
        sys.stdout.write ('%d: %s\n' % (c, time.ctime(time.time())) )
        sys.stdout.flush()
        c = c + 1
        time.sleep(1)

if __name__ == "__main__":
    startstop(stdout='/tmp/deamonize.log',
              pidfile='/tmp/deamonize.pid')
    test()
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.