Re: redirect.internal with parameters?

#!/usr/bin/python
#
# session2.py
# Version 1.2
#
# Copyright (c) 2006 Conan C. Albrecht
#
# Permission is hereby granted, free of charge, to any person obtaining a copy 
# of this software and associated documentation files (the "Software"), to deal 
# in the Software without restriction, including without limitation the rights 
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 
# copies of the Software, and to permit persons to whom the Software is furnished 
# to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all 
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, 
# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR 
# PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE 
# FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR 
# OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 
# DEALINGS IN THE SOFTWARE.

__doc__ = '''
This module automates sessioning for a Spyce web site.  It emulates
a dictionary specific to each user (really each browser) accessing
your web site.  You simply use session2 as if it were a dictionary
variable, and its contents automatically change depending upon the
user calling the page.

Following is an example:
[[.import name=session2]]
[[\
  session2['username'] = 'BartSimpson'
  print session2['username'] # prints BartSimpson
]]

In the example above, the 'username' key would now be valid for
all pages on your site, until the session expires.  This is 
considerably easier than the session module that ships with Spyce.

Why Use session2?
===================
While Spyce comes with a default session module, it has limitations that
this module overcomes.  This module is a complete replacement for it.
In particular, this module has the following advantages:

1. It allows you to use a session across your entire site, not just to 
   a single page.  The built-in session module doesn't allow you access
   to the cookie module, so its session actually only works on the same
   page it was set on.  This module defaults session information to
   be accessible across your entire site.

2. It automates the creation of sessions -- you simply act as though 
   every user on your site has a session, and let the module efficiently 
   handle the internals for you.  

3. It provides a more dictionary-like interface.  You can pretend that
   the session2 module is a dictionary variable specific to each
   user.  It will automatically change for each user.

4. It limits the number of session files on disk to 256, regardless 
   of how many sessions are being managed.  

5. It allows you to use the session in the *same* request in which
   it was created.  This is not possible with the session module
   that ships with Spyce.

6. It efficiently clears out old sessions, keeping the cache files as 
   small as possible.  This cleaning happens automatically.

The module has a few limitations.  First, it always saves to
a set of dbm files on disk.  It does not do a memory cache.  This is
especially useful when running Spyce under mod_python, fastcgi, or cgi --
where multiple versions of Spyce might be running.  If you are using
the web-server installation of Spyce, this will slow things down since
you could be doing it in memory.

Second, although it allows you to use the session in the same request
as it is created in, a redirect.internal call makes the session invalid.
This is because the request and response objects are recreated on an
internal redirect.  So if you set a value in the session, then call
redirect.internal, whatever you set is lost.  The best way around this
right now is to use redirect.external when you need to save things in
the session before a redirect.

Module Setup
============

The module is initialized as any other module, such as:
[[.import name=session2]]
For most users, this is probably all you need to know
about initializing the module.

The module has several options that can be set in spyceconf.py
or in the import call.  I recommend setting any option
customizations in spyceconf.py so it goes across all pages
on your site automatically.  The parameters are as follows:

 * method:    Whether to keep sessions in memory on in dbm
              files on the disk.  The default is dbm files
              since this is the only way to do sessions in 
              mod_python, cgi, and fastcgi installations of
              Spyce (which I think is by far the most common).
              In-memory sessioning is faster and can be used
              in web-server installations of Spyce.
              
              Set method='dbm' to use a disk cache or 
              method='memory' to keeps sessions in memory.
              
 * directory: If you are using dbm method of sessioning, this
              sets the directory to save the sessions in.  It
              defaults to /tmp on Unix and c:\temp on Windows.
              
 * path:      The path to attach the session to.  This has reference
              to cookie semantics.  For example, if the path is 
              /myapp, the session will only be valid under /myapp
              pages (and below).  The default is '/', which means
              the session is valid site-wide.
              
 * expire:    The number of seconds the session is good for.
              The default is None, meaning the session becomes 
              invalid when the browser is closed.  If negative
              or zero, cookies are not used at all.  Instead, 
              URL-pass-through is required (see below).
              
The following command stores sessions in the '/mydir' directory,
has an expiration of 60 seconds, and is valid only in the '/myapp'
path subtree:

[[.import name=session2 args="method='dbm', directory='/mydir', path='/myapp', expire=60"]]

If you want to set these options from spyceconf.py, the variables
become session2_method, session2_directory, session2_path, and
session2_expire.  For example, the import options above can be 
set in spyceconf.py by adding:

session2_method    = 'dbm'
session2_directory = '/mydir'
session2_path      = '/myapp'
session2_expire    = 60

If you put the above variable options in spyceconf.py and also put
them in the [[.import]] call, the [[.import]] call wins.

Module Use
==========

Use the session by calling the get(), put(), and delete() methods. The
following example stores, uses, and then deletes the username:
[[.import name=session2]]
[[\
  session2['username'] = 'Bart Simpson'
  print session2.get('username')        # prints Bart Simpson
  print session2['username']            # prints Bart Simpson
  
  del session2['username']              # deletes the key/value from the session
  #print session2['username']           # throws a KeyError (so commented out)
  print session2.get('username')        # prints None
  print session2.get('username', 'Me')  # prints Me since it was the default

  session2['visited'] = session2.get('visited', 0) + 1
  print 'You visited us %s times.' % session2['visited']
]]

In other words, it is very similar to a dictionary.  You can pretend the session2
module is just a dictionary specific to the user accessing the page.  The session2
"dictionary" contents will change automatically to match the user's values
for the browser accessing your page.

Cookies vs. URL Pass Through: By default, the module sends a cookie to the
client browser.  More specifically, cookies are used if expire is None (default)
or greater than 0.  Cookies provide automatic management of the session id. 
You don't have to do anything in particular to make them work.

If you give a negative value to the cookie parameter (either
in spyceconf.py or in the [[.import]] statement, no cookies are sent.
Instead, you'll have to send the _session2_id parameter in *every*
form and URL link.  This is done as follows:

<f:form>
<f:hidden name="_session2_id" value="[[=session2.id()]]"/>
</f:form>

or

<a href="/mypage.spy?_session2_id=[[=session2.id()]]">Link</a>

While this is a lot more work than simply allowing a cookie to
be passed automatically by the module and browser, it supports
those situations where cookies are not desirable or possible.

IMPORTANT NOTE 1:
You should only put small, temporary objects into the session.  While 
the session will take anything that can be pickled, remember that
it must be stored in a dbm hash.  Large objects will slow things down
considerably; avoid them.  Store strings, ints, and simple objects.

For example, if you want to store a User object, store the primary 
key (user id) to the User object.  On each request, reload the User
object from your database using the user id stored in the session.

IMPORTANT NOTE 2:

This module adds a _lastaccess key to each session.  It uses this 
value to know when the session needs to be cleaned out.  Don't 
use this key name.  In other words, don't call 
session2['_lastaccess'] = anything.

IMPORTANT NOTE 3:

If you are running Spyce in mod_python, cgi, or fastcgi installation
modes, you *should not* use the memory option.  Use the dbm method
for these installation modes.  While dbm is slower and takes disk
space, an in-memory cache will cause update errors since you'll
get multiple caches!  This is because mod_python, cgi, and  fastcgi
create multiple instances of your application -- you'll have an
cache in *each* instance.

In addition, dbm-method sessions survive a server reboot, memory-method
sessions do not.


The Internals
=============
For those interested, the session id is a GUID (see my post on
ActiveState).  The module grabs two hex characters from the GUID
(from the random part) and saves the session object under a
file named by these two characters.  This ensures that 1) sessions
are randomly spread around the files, and 2) a maximum of 256
(16^2) files are used.

If you are running a very high transaction site, you may want to
increase the number of files used.  Do this by changing the 
NUM_FILES_EXPONENT constant.  You should not change this value
unless you read the warnings and really know what you are doing.

Sessions are regular Python dictionaries.  The standard python 
"shelve" module is used to save these dictionaries to disk.
Since sessions are temporary and shouldn't be moved from installation
to installation, the module uses the highest possible pickle
protocol for speed.

Whenever a session is saved to a file, the last checked
timestamp is used to see if the file should be cleaned.
If enough time has passed, all old sessions are deleted
from the shelf.

The module cleans every CLEAN_INTERVAL seconds (set to every
24 hours right now).  If you want a different CLEAN_INTERVAL,
just change this constant at the top of the file.  I didn't make
this accessible publicly because I don't think most users
care how often it cleans sessions out.  Note that files are
not actually ever cleaned until something is set within them.

Sessions are never actually created until your program puts
data in them.  In other words, they are created just in time.
Since many site visitors may never get session data set, this
provides a great efficiency since no disk access is required
for them.  Your program will think they have a session, but
as long as you don't call put(), they won't get a session
internally.

The module is thread-safe.  A series of file locks is used 
to lock shelves when they are being used or cleaned.  However,
if multiple instances of Spyce are created, it is remotely
possible that sessions might blast each other.  I'm not 
sure how to solve this without additional disk access.  If anyone
wants to help here please do.  (note that the regular session
module has this problem, too.)

Since cookies aren't accessible until the *next* request,
the module saves the sessionid in response._session_dbm_id
when a sessionid is created.  This allows it to be used in
the current request, even if it hasn't come back to us as
a cookie yet.
'''

from spyceModule import spyceModule
import spyce
import types, sys, shelve, pickle, os, os.path, threading

#########################################################
###   Constants 

# the default directory to store sessions in on Unix
DEFAULT_UNIX_DIRECTORY = '/tmp'

# the default directory to store sessions in on Windows
DEFAULT_WIN_DIRECTORY = 'c:\\temp'

# how often to check for and clean old sessions
CLEAN_INTERVAL = 60 * 60 * 24  # clean every 24 hours

# the exponent used for the number of files to store sessions
# in.  The default value of 2 means 16^2, or 256 files.  Setting
# this value to 3 means 16^3, or 4096 files.  The only reason
# you'd ever want to change this is if you have a really
# high traffic Spyce site and you want sessions to be
# spread across more files.  Note that different filesystems
# (ext3, NTFS, etc.) have limitations on the number of files
# that can be in a given directory, so setting this number too 
# high may put too many files in a directory and cause all kinds
# of fun things to occur!
NUM_FILES_EXPONENT = 2


########################################################
###   Locks for multi-threading safety

program_lock = threading.RLock()
file_locks = {}

  
######################################################## 
###   In-memory cache if it's being used

memory_cache = {}

  
########################################################
###   The session module

class session2(spyceModule):
  '''Manages the sessions of the web site in a dbm (shelve) file.'''
  
  def init(self, *args, **kargs):
    '''Initializes the module.  Spyce calls this automatically.
       Don't call directly.'''
    # little function to check kargs first, then spyceconf, then default
    def get_option(name, default):
      if kargs.has_key(name):                                      return kargs[name]
      elif hasattr(spyce.getServer().config, 'session2_' + name):  return getattr(spyce.getServer().config, 'session2_' + name)
      else:                                                        return default
       
    # get the options
    self.directory = get_option('directory', (os.name == 'win32' or os.name == 'nt') and DEFAULT_WIN_DIRECTORY or DEFAULT_UNIX_DIRECTORY)
    if not os.path.exists(self.directory):          
      os.makedirs(self.directory)
    self.path = get_option('path', '/')
    self.expire = get_option('expire', None)
    self.method = get_option('method', 'dbm').lower()


  ################################################
  ###   Main interface methods

  def get(self, key, default=None):
    '''Retrieves the value saved under this key, or the default if the key doesn't exist'''
    session = self._get_session()
    if session.has_key(key):
      return session[key]
    return default
    
  
  def put(self, key, value):
    '''Puts the given value under the key in the session'''
    session = self._get_session()
    session[key] = value
    self._save_session(session)
    
    
  def delete(self, key):
    '''Deletes the session under the given key'''
    session = self._get_session()
    if session.has_key(key):
      del session[key]
    self._save_session(session)
    
  
  def id(self):
    '''Returns the session id of this user'''
    # this method looks for the session id in several places, then
    # creates it on the fly if needed.  It *always* returns an id.
    
    # check the request parameters (to see if it was sent via URL)
    sessionid = None
    if self.expire <= 0:
      if self._api.getModule('request').get('_session2_id'):
        sessionid = self._api.getModule('request').get('_session2_id')
    
    # check the response (to see if we created one)
    if hasattr(self._api.getModule('response'), '_session2_id'):  # allows session to be accessed upon first save to dbm -- cookie won't come back until the *next* request when it is set
      sessionid = self._api.getModule('response')._session2_id
      
    # get it from the cookie
    else:
      sessionid = self._api.getModule('cookie').get('_session2_id')
      
    # if we couldn't find a sessionid, create one 
    if sessionid == None or len(sessionid) != 40:
      sessionid = generate()
      self._api.getModule('response')._session2_id = sessionid   # so it can be used later in this request

    # refresh the cookie in the browser (restarts expiration timer)
    self._api.getModule('cookie').set('_session2_id', sessionid, expire=self.expire, path=self.path)
    
    # return the id
    return sessionid
    
    
  ###################################################
  ###   Dictionary-like methods (for convenience)
    
  def __setitem__(self, key, value):  self.put(key, value)  # use our method so it gets saved
  def __delitem__(self, key):         self.delete(key)      # use our method so it gets saved
  def __getitem__(self, key):         return self._get_session()[key]
  def __len__(self):                  return len(self._get_session())
  def __iter__(self):                 return iter(self._get_session())
  def has_key(self, key):             return self._get_session().has_key(key)
  def keys(self):                     return self._get_session().keys()
  def values(self):                   return self._get_session().values()
  def items(self):                    return self._get_session().items()
  def __contains__(self, key):        return key in self._get_session()
    
    
  #####################################################
  ###   Private methods
   
  def _get_session(self):
    '''Returns the user's session, or an empty dict {} if it doesn't exist.'''
    sessionid = self.id()
    if sessionid and len(sessionid) == 40:
      if self.method == 'memory':  # memory method
        self._lock('memory')
        try:
          if memory_cache.has_key(sessionid):
            return memory_cache[sessionid]
        finally:
          self._unlock('memory')
          
      else:  # dbm method
        filename = os.path.join(self.directory, 'spyce-' + sessionid[16:16 + NUM_FILES_EXPONENT] + '.sessions')
        self._lock(filename)
        try:
          s = shelve.open(filename, protocol=pickle.HIGHEST_PROTOCOL)  # highest protocol for speed
          try:
            if s.has_key(sessionid):
              session = s[sessionid]
              if session.has_key('_lastaccess') and session['_lastaccess'] < time.time() - 60: # only save lastaccess once a minute so we don't save unnecessarily
                session['_lastaccess'] = time.time()
                self._save_session(session)
              self._save_session(session)
              return session
          finally:
            s.close()
        finally:
          self._unlock(filename)
    return {}
    
  
  def _save_session(self, session):
    '''Saves a session object (a dictionary)'''
    # get or create the session id
    sessionid = self.id()

    # save the object
    if self.method == 'memory':  # memory method
      self._lock('memory')
      try:
        session['_lastaccess'] = time.time()
        memory_cache[sessionid] = session
        self._clean(memory_cache)
      finally:
        self._unlock('memory')
    
    else:  # dbm method
      filename = os.path.join(self.directory, 'spyce-' + sessionid[16:16 + NUM_FILES_EXPONENT] + '.sessions')
      self._lock(filename)
      try:
        s = shelve.open(filename, protocol=pickle.HIGHEST_PROTOCOL)  # highest protocol for speed
        session['_lastaccess'] = time.time()
        s[sessionid] = session
        self._clean(s)
        s.close()
      finally:
        self._unlock(filename)


  def _clean(self, s):
    '''Cleans a given shelf or dictionary, if needed'''
    # this method is not thread-safe, but it is called from _save_session (which is already thread safe)
    if not s.has_key('lastclean') or s['lastclean'] < time.time() - CLEAN_INTERVAL:
      oldtime = time.time() - CLEAN_INTERVAL
      for key, value in s.items():
        if isinstance(value, types.DictType):
          if not value.has_key('_lastaccess') or value['_lastaccess'] < oldtime:
            del s[key]
      s['lastclean'] = time.time()
  

  def _lock(self, filename):
    '''Locks the given filename'''
    program_lock.acquire()
    try:
      if not file_locks.has_key(filename):
        file_locks[filename] = threading.RLock()
      file_locks[filename].acquire()
    finally:
      program_lock.release()
    
    
  def _unlock(self, filename):
    '''Unlocks the given filename'''
    program_lock.acquire()
    try:
      if not file_locks.has_key(filename):
        file_locks[filename] = threading.RLock()
      file_locks[filename].release()
    finally:
      program_lock.release()
    
    
    
    

############################################################################
###   This is a copy of my GUID code on ActiveState's cookbook.
###   GUIDS are used for session ids.   I've copied the file here
###   so it won't have any external dependencies

import math
import socket
import random
import sys
import time
import threading

#############################
###   global module variables

#Makes a hex IP from a decimal dot-separated ip (eg: 127.0.0.1)
make_hexip = lambda ip: ''.join(["%04x" % long(i) for i in ip.split('.')]) # leave space for ip v6 (65K in each sub)
  
MAX_COUNTER = 0xfffffffe
counter = 0L
firstcounter = MAX_COUNTER
lasttime = 0
ip = ''
lock = threading.RLock()
try:  # only need to get the IP addresss once
  ip = socket.getaddrinfo(socket.gethostname(),0)[-1][-1][0]
  hexip = make_hexip(ip)
except: # if we don't have an ip, default to someting in the 10.x.x.x private range
  ip = '10'
  rand = random.Random()
  for i in range(3):
    ip += '.' + str(rand.randrange(1, 0xffff))  # might as well use IPv6 range if we're making it up
  hexip = make_hexip(ip)

  
  
#################################
###   Public module functions

def generate(ip=None):
  '''Generates a new guid.  A guid is unique in space and time because it combines
     the machine IP with the current time in milliseconds.  Be careful about sending in
     a specified IP address because the ip makes it unique in space.  You could send in
     the same IP address that is created on another machine.
  '''
  global counter, firstcounter, lasttime
  lock.acquire() # can't generate two guids at the same time
  try:
    parts = []

    # do we need to wait for the next millisecond (are we out of counters?)
    now = long(time.time() * 1000)
    while lasttime == now and counter == firstcounter: 
      time.sleep(.01)
      now = long(time.time() * 1000)

    # time part
    parts.append("%016x" % now)

    # counter part
    if lasttime != now:  # time to start counter over since we have a different millisecond
      firstcounter = random.randint(1, MAX_COUNTER)  # start at random position
      counter = firstcounter
    counter += 1
    if counter > MAX_COUNTER:
      counter = 0
    lasttime = now
    parts.append("%08x" % (counter)) 

    # ip part
    parts.append(hexip)

    # put them all together
    return ''.join(parts)
  finally:
    lock.release()