SF.net SVN: tmda: [2066] trunk/tmda
[email protected] Wed, 04 Oct 2006 14:18:16 -0700
| Newsgroups | gmane.mail.spam.tmda.cvs |
|---|---|
| Message-ID | <[email protected]> |
Revision: 2066
http://svn.sourceforge.net/tmda/?rev=2066&view=rev
Author: jasonrm
Date: 2006-10-04 14:18:11 -0700 (Wed, 04 Oct 2006)
Log Message:
-----------
First cut at Maildir format support for the pending queue.
Modified Paths:
--------------
trunk/tmda/NEWS
trunk/tmda/TMDA/ChangeLog
trunk/tmda/TMDA/Defaults.py
trunk/tmda/TMDA/Queue/ChangeLog
trunk/tmda/TMDA/Queue/Queue.py
trunk/tmda/htdocs/config-vars.html
Added Paths:
-----------
trunk/tmda/TMDA/Queue/MaildirQueue.py
Modified: trunk/tmda/NEWS
===================================================================
--- trunk/tmda/NEWS 2006-10-04 18:10:53 UTC (rev 2065)
+++ trunk/tmda/NEWS 2006-10-04 21:18:11 UTC (rev 2066)
@@ -5,6 +5,38 @@
======================================================================
+TMDA 1.1.7 "Hazelburn" --
+
+* TMDA can now optionally store unconfirmed messages in a "Maildir"
+ rather than in TMDA's custom pending queue format. This allows
+ you to use any mail reading program that supports Maildir to browse
+ your pending queue, rather than the 'tmda-pending' and 'tmda-cgi'
+ programs, though the latter will still work. For more information
+ on Maildir, see http://en.wikipedia.org/wiki/Maildir
+
+ Maildir support should be considered "experimental" in this release
+ only because it's not had significant exposure in live environments,
+ but seems to work well in testing. If you wish to try it, you must
+ set the following variable in your config:
+
+ PENDING_QUEUE_FORMAT = 'maildir'
+
+ See http://wiki.tmda.net/ConfigurationVariables#PENDING_QUEUE_FORMAT
+
+ Future releases will provide a tool to handle conversion for you,
+ but for now, you must also convert your current pending queue to a
+ Maildir by hand, which is not difficult since the formats are fairly
+ similar:
+
+ % chmod +t $HOME
+ % cd ~/.tmda/
+ % mv pending pending.old
+ % mkdir -m 0700 -p pending/{new,cur,tmp}
+ % cp pending.old/*.msg pending/new
+ % chmod -t $HOME
+
+======================================================================
+
TMDA 1.1.6 "Glen Albyn" --
* tmda-filter will defer incoming deliveries if the sticky bit is set
Modified: trunk/tmda/TMDA/ChangeLog
===================================================================
--- trunk/tmda/TMDA/ChangeLog 2006-10-04 18:10:53 UTC (rev 2065)
+++ trunk/tmda/TMDA/ChangeLog 2006-10-04 21:18:11 UTC (rev 2066)
@@ -1,3 +1,7 @@
+2006-10-04 Jason R. Mastaler <[email protected]>
+
+ * Defaults.py (PENDING_QUEUE_FORMAT): Add 'maildir'.
+
2006-09-28 Jason R. Mastaler <[email protected]>
* Defaults.py (PENDING_QUEUE_FORMAT): New variable.
Modified: trunk/tmda/TMDA/Defaults.py
===================================================================
--- trunk/tmda/TMDA/Defaults.py 2006-10-04 18:10:53 UTC (rev 2065)
+++ trunk/tmda/TMDA/Defaults.py 2006-10-04 21:18:11 UTC (rev 2066)
@@ -1060,6 +1060,12 @@
# directory (PENDING_DIR). Offers high performance, but can only
# be browsed with TMDA tools like 'tmda-pending' and 'tmda-cgi'.
#
+# "maildir"
+# Maildir is a specific one-file-per-message organization that
+# was introduced with the qmail system by D.J. Bernstein. For
+# more information, see http://en.wikipedia.org/wiki/Maildir
+# NOTE: Maildir support is currently considered experimental.
+#
# Default is "original".
if not vars().has_key('PENDING_QUEUE_FORMAT'):
PENDING_QUEUE_FORMAT = 'original'
Modified: trunk/tmda/TMDA/Queue/ChangeLog
===================================================================
--- trunk/tmda/TMDA/Queue/ChangeLog 2006-10-04 18:10:53 UTC (rev 2065)
+++ trunk/tmda/TMDA/Queue/ChangeLog 2006-10-04 21:18:11 UTC (rev 2066)
@@ -1,3 +1,11 @@
+2006-10-04 Jason R. Mastaler <[email protected]>
+
+ * MaildirQueue.py: new file. Maildir format support.
+
+2006-10-04 Jason R. Mastaler <[email protected]>
+
+ * Queue.py (Queue.init): Add 'maildir'.
+
2006-09-27 Jason R. Mastaler <[email protected]>
* Queue.py: new file.
Added: trunk/tmda/TMDA/Queue/MaildirQueue.py
===================================================================
--- trunk/tmda/TMDA/Queue/MaildirQueue.py (rev 0)
+++ trunk/tmda/TMDA/Queue/MaildirQueue.py 2006-10-04 21:18:11 UTC (rev 2066)
@@ -0,0 +1,278 @@
+# -*- python -*-
+#
+# Copyright (C) 2001,2002,2003,2004,2005,2006 Jason R. Mastaler <[email protected]>
+#
+# This file is part of TMDA.
+#
+# TMDA is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version. A copy of this license should
+# be included in the file COPYING.
+#
+# TMDA is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+# for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with TMDA; if not, write to the Free Software Foundation, Inc.,
+# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+"""Maildir compatible pending queue format.
+
+http://en.wikipedia.org/wiki/Maildir
+"""
+
+
+from email.Utils import parseaddr
+
+
+import fcntl
+import glob
+import os
+import signal
+import socket
+import stat
+import time
+
+
+from TMDA import Defaults
+from TMDA import Errors
+from TMDA import Util
+from TMDA.Queue.Queue import Queue
+
+
+def alarm_handler(signum, frame):
+ """Handle an alarm."""
+ print 'Signal handler called with signal', signum
+ raise IOError, "Couldn't open device!"
+
+def lock_file(fp):
+ """Do fcntl file locking."""
+ fcntl.flock(fp.fileno(), fcntl.LOCK_EX)
+
+def unlock_file(fp):
+ """Do fcntl file unlocking."""
+ fcntl.flock(fp.fileno(), fcntl.LOCK_UN)
+
+
+
+class MaildirQueue(Queue):
+ def __init__(self):
+ Queue.__init__(self)
+ self.format = "maildir"
+
+
+ def exists(self):
+ if os.path.exists(Defaults.PENDING_DIR):
+ return True
+ else:
+ return False
+
+
+ def _create(self):
+ if not self.exists():
+ dirpath = Defaults.PENDING_DIR
+ os.makedirs(os.path.join(dirpath, 'cur'), 0700)
+ os.mkdir(os.path.join(dirpath, 'new'), 0700)
+ os.mkdir(os.path.join(dirpath, 'tmp'), 0700)
+
+
+ def _convert(self):
+ pass
+
+
+ def cleanup(self):
+ if not self.exists():
+ return
+
+ lifetimesecs = Util.seconds(Defaults.PENDING_LIFETIME)
+ cwd = os.getcwd()
+ os.chdir(os.path.join(Defaults.PENDING_DIR, 'new'))
+ new_msgs = glob.glob('[0-9]*.[0-9]*.*')
+ os.chdir(os.path.join(Defaults.PENDING_DIR, 'cur'))
+ cur_msgs = glob.glob('[0-9]*.[0-9]*.*')
+ os.chdir(cwd)
+ msgs = new_msgs + cur_msgs
+
+ for msg in msgs:
+ now = '%d' % time.time()
+ min_time = int(now) - int(lifetimesecs)
+ msg_time = int(msg.split('.')[0])
+ if msg_time > min_time:
+ # skip this message
+ continue
+ # delete this message
+ for f in (os.path.join(Defaults.PENDING_DIR, 'new', msg),
+ os.path.join(Defaults.PENDING_DIR, 'cur', msg)):
+ if os.path.exists(f):
+ fpath = f
+ break
+ if Defaults.PENDING_DELETE_APPEND:
+ try:
+ msgobj = Util.msg_from_file(open(fpath, 'r'))
+ except IOError:
+ # in case of concurrent cleanups
+ pass
+ else:
+ rp = parseaddr(msgobj.get('return-path'))[1]
+ Util.append_to_file(rp, Defaults.PENDING_DELETE_APPEND)
+ try:
+ os.unlink(fpath)
+ except OSError:
+ # in case of concurrent cleanups
+ pass
+
+
+ def fetch_ids(self):
+ cwd = os.getcwd()
+ os.chdir(os.path.join(Defaults.PENDING_DIR, 'new'))
+ new_msgs = glob.glob('[0-9]*.[0-9]*.*')
+ os.chdir(os.path.join(Defaults.PENDING_DIR, 'cur'))
+ cur_msgs = glob.glob('[0-9]*.[0-9]*.*')
+ ids = ['.'.join(i.split('.')[:2])
+ for i in new_msgs + cur_msgs]
+ os.chdir(cwd)
+ return ids
+
+
+ def insert_message(self, msg, mailid, recipient):
+ # Create the Maildir if necessary.
+ self._create()
+ # X-TMDA-Recipient is used by release_pending()
+ del msg['X-TMDA-Recipient']
+ msg['X-TMDA-Recipient'] = recipient
+ # Write message
+ time, pid = mailid.split('.')
+ self.__deliver_maildir(Util.msg_as_string(msg), time, pid,
+ Defaults.PENDING_DIR)
+ del msg['X-TMDA-Recipient']
+
+
+ def fetch_message(self, mailid, fullParse=False):
+ msgs = (glob.glob(os.path.join(Defaults.PENDING_DIR, 'cur/')
+ + '[0-9]*.[0-9]*.*')) + \
+ (glob.glob(os.path.join(Defaults.PENDING_DIR, 'new/')
+ + '[0-9]*.[0-9]*.*'))
+ for m in msgs:
+ if mailid in m:
+ msg = Util.msg_from_file(file(m, 'r'),fullParse=fullParse)
+ return msg
+ else:
+ # couldn't find message, defer and retry until we find it
+ raise IOError, "couldn't locate %s, will retry" % m
+
+
+ def delete_message(self, mailid):
+ msgs = (glob.glob(os.path.join(Defaults.PENDING_DIR, 'cur/')
+ + '[0-9]*.[0-9]*.*')) + \
+ (glob.glob(os.path.join(Defaults.PENDING_DIR, 'new/')
+ + '[0-9]*.[0-9]*.*'))
+ for m in msgs:
+ if mailid in m:
+ os.unlink(m)
+
+
+ def find_message(self, mailid):
+ cwd = os.getcwd()
+ os.chdir(Defaults.PENDING_DIR)
+ msgs = glob.glob('cur/[0-9]*.[0-9]*.*') + \
+ glob.glob('new/[0-9]*.[0-9]*.*')
+
+ for i in range(5):
+ for m in msgs:
+ if mailid in m:
+ os.chdir(cwd)
+ return True
+ else:
+ # retry several times in case a MUA moved/renamed the
+ # message to cur/
+ time.sleep(1)
+ msgs = glob.glob('cur/[0-9]*.[0-9]*.*') + \
+ glob.glob('new/[0-9]*.[0-9]*.*')
+ continue
+ os.chdir(cwd)
+ # give up; message is not there
+ return False
+
+
+ def __deliver_maildir(self, message, time, pid, maildir):
+ """Reliably deliver a mail message into a Maildir.
+
+ Implementation differs slightly from the one in TMDA.Deliver()
+ since we need to maintain the time and pid in the file's name.
+
+ message is the mail message as a string.
+
+ time and pid come from the mailid.
+
+ maildir is the destination Maildir.
+
+ Based on code from getmail
+ Copyright (C) 2001 Charles Cazabon, and licensed under the GNU
+ General Public License version 2.
+ """
+ # e.g, 1014754642.51195.aguirre.la.mastaler.com
+ filename = '%s.%s.%s' % (time, pid, socket.gethostname())
+ # Set a 24-hour alarm for this delivery.
+ signal.signal(signal.SIGALRM, alarm_handler)
+ signal.alarm(24 * 60 * 60)
+
+ dir_tmp = os.path.join(maildir, 'tmp')
+ dir_cur = os.path.join(maildir, 'cur')
+ dir_new = os.path.join(maildir, 'new')
+ if not (os.path.isdir(dir_tmp) and
+ os.path.isdir(dir_cur) and
+ os.path.isdir(dir_new)):
+ raise Errors.DeliveryError, 'not a Maildir! (%s)' % maildir
+
+ fname_tmp = os.path.join(dir_tmp, filename)
+ fname_new = os.path.join(dir_new, filename)
+
+ # File must not already exist.
+ if os.path.exists(fname_tmp):
+ raise Errors.DeliveryError, fname_tmp + 'already exists!'
+ if os.path.exists(fname_new):
+ raise Errors.DeliveryError, fname_new + 'already exists!'
+
+ # Get user & group of maildir.
+ s_maildir = os.stat(maildir)
+ maildir_owner = s_maildir[stat.ST_UID]
+ maildir_group = s_maildir[stat.ST_GID]
+
+ # Open file to write.
+ try:
+ fd = os.open(fname_tmp, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0600)
+ fp = os.fdopen(fd, 'wb', 4096)
+ os.chmod(fname_tmp, 0600)
+ try:
+ os.chown(fname_tmp, maildir_owner, maildir_group)
+ except OSError:
+ # Not running as root, can't chown file.
+ pass
+ fp.write(message)
+ fp.flush()
+ os.fsync(fp.fileno())
+ fp.close()
+ except (OSError, IOError), o:
+ signal.alarm(0)
+ raise Errors.DeliveryError, \
+ 'Failure writing file %s (%s)' % (fname_tmp, o)
+
+ # Move message file from Maildir/tmp to Maildir/new
+ try:
+ os.link(fname_tmp, fname_new)
+ os.unlink(fname_tmp)
+ except OSError:
+ signal.alarm(0)
+ try:
+ os.unlink(fname_tmp)
+ except:
+ pass
+ raise Errors.DeliveryError, 'failure renaming "%s" to "%s"' \
+ % (fname_tmp, fname_new)
+
+ # Cancel the alarm.
+ signal.alarm(0)
+ signal.signal(signal.SIGALRM, signal.SIG_DFL)
Modified: trunk/tmda/TMDA/Queue/Queue.py
===================================================================
--- trunk/tmda/TMDA/Queue/Queue.py 2006-10-04 18:10:53 UTC (rev 2065)
+++ trunk/tmda/TMDA/Queue/Queue.py 2006-10-04 21:18:11 UTC (rev 2066)
@@ -120,9 +120,9 @@
if qformat.lower() == 'original':
from OriginalQueue import OriginalQueue
return OriginalQueue()
- #if qformat.lower() == 'maildir':
- # from MaildirQueue import MaildirQueue
- # return MaildirQueue()
+ if qformat.lower() == 'maildir':
+ from MaildirQueue import MaildirQueue
+ return MaildirQueue()
else:
raise Errors.ConfigError, \
"Unknown PENDING_QUEUE_FORMAT: " + '"%s"' % qformat
Modified: trunk/tmda/htdocs/config-vars.html
===================================================================
--- trunk/tmda/htdocs/config-vars.html 2006-10-04 18:10:53 UTC (rev 2065)
+++ trunk/tmda/htdocs/config-vars.html 2006-10-04 21:18:11 UTC (rev 2066)
@@ -1170,6 +1170,14 @@
be browsed with TMDA tools like 'tmda-pending' and 'tmda-cgi'.
</blockquote>
<br>
+"<code>maildir</code>"
+<blockquote>
+ Maildir is a specific one-file-per-message organization that
+ was introduced with the qmail system by D.J. Bernstein. For
+ more information, see http://en.wikipedia.org/wiki/Maildir
+ NOTE: Maildir support is currently considered experimental.
+</blockquote>
+<br>
Default is "original".
<dt><hr>
<a name="PENDING_RELEASE_APPEND"><h4>PENDING_RELEASE_APPEND</h4></a>
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.