[PEAR-BUG] Bug #2050 [Com]: Messages get sent multiple times

Edit report at http://pear.php.net/bugs/bug.php?id=2050&edit=1

 ID:               2050
 Comment by:       mscarda
 Reported By:      mscarda at gmail dot com
 Summary:          Messages get sent multiple times
 Status:           Wont fix
 Type:             Bug
 Package:          Mail_Queue
 Operating System: Linux
 PHP Version:      4.3.8
 Assigned To:      quipo
 New Comment:

Hi everybody,
I'm having this issue: sometime my clients gets double email, sometimes
not. What it can be? How can I fix it?
Can someone please help me?

Thanks

Mike


Previous Comments:
------------------------------------------------------------------------

[2007-01-20 06:04:57] quipo

Hi, 
while this is not strictly a Mail_Queue bug, but rather an unfortunate
timing issue with cron and the smtp queue, I agree that Mail_Queue
doesn't help in preventing this problem.

The current Mail_Queue v.1.x can't be fixed since it would require a BC
break, and that's not allowed by PEAR rules for 'stable' packages. A new
Mail_Queue2 is in the works, though, and it will provide a sort of
row-level locking to avoid concurrency problems like this one.

------------------------------------------------------------------------

[2006-09-08 19:21:43] ben at nlcweb dot com

Can an admin verify the above posted method?  I'm having the same
troubles with a client's customers receiving up to 10 duplicate copies
of the same email due to this problem.  I would like to hear an update
on people who have tried it or if something is in the works for the Mail
Queue package...

------------------------------------------------------------------------

[2006-03-09 16:53:19] ben dot litton at gmail dot com

I may have a solution. I opted for a redundant solution. First, in your
cron script, try this...

#!/usr/bin/php
<?php
$filename = '/whatever/path/you/want/mail.lock';
$fp = @fopen($filename, 'x');
if (!$fp)
     exit;
//run mail daemon here.
     fclose($fp);
unlink($filename);
?>

If the lockfile is absent, it runs, if it's there, it's already running,
and it aborts. This begs the question, what if the cron script dies
during the run and the lockfile remains? 

I modified the PEAR code. 

1. add a column called ptime to your table, default to null, it's a
datetime. 

2. We use db as our container, so I added these to the container db.php
file. 

In the constructor, ~line 65

    /**
     * @var string one day ago, this is compared to current processing
time to prevent duplicate e-mails
     */
    var $olddate;

    /**
     * @var string this is the timestamp of when this class was
initialized. It is inserted into the database under ptime as the process
time
     */
    var $ptime;

The _preload function ~line 125:

 function _preload()
    {
        $query = sprintf("SELECT * FROM %s WHERE sent_time IS NULL
                            AND (ptime IS NULL or ptime < %s) 
                            AND try_sent < %d
                            AND %s > time_to_send
                            ORDER BY time_to_send",
                         $this->mail_table,
                         $this->db->quote($this->olddate),
                         $this->try,
                         $this->db->quote(date("Y-m-d H:i:s"))
                         );
        $query = $this->db->modifyLimitQuery($query, $this->offset,
$this->limit);
        if (DB::isError($query)) {
            return new Mail_Queue_Error(MAILQUEUE_ERROR_QUERY_FAILED,
                $this->pearErrorMode, E_USER_ERROR, __FILE__, __LINE__,
                'DB::modifyLimitQuery failed - '.$query->toString());
        }
        $res = $this->db->query($query);
        if (DB::isError($res)) {
            return new Mail_Queue_Error(MAILQUEUE_ERROR_QUERY_FAILED,
                $this->pearErrorMode, E_USER_ERROR, __FILE__, __LINE__,
                'DB::query failed - "'.$query.'" - '.$res->toString());
        }

        $this->_last_item = 0;
        $this->queue_data = array(); //reset buffer
        $id_increment = 0;
        $id_list = array();

        while ($row = $res->fetchRow(DB_FETCHMODE_ASSOC)) {
            if (!is_array($row)) {
                return new
Mail_Queue_Error(MAILQUEUE_ERROR_QUERY_FAILED,
                    $this->pearErrorMode, E_USER_ERROR, __FILE__,
__LINE__,
                    'DB::query failed - "'.$query.'" -
'.$res->toString());
            }
            $this->queue_data[$this->_last_item] = new Mail_Queue_Body(
                $row['id'],
                $row['create_time'],
                $row['time_to_send'],
                $row['sent_time'],
                $row['id_user'],
                $row['ip'],
                $row['sender'],
                $row['recipient'],
                unserialize($row['headers']),
                unserialize($row['body']),
                $row['delete_after_send'],
                $row['try_sent']
            );
            $id_list[$id_increment++] = $row['id'];
            $this->_last_item++;
        }

        if (count($id_list) > 0)
          {
            $query = sprintf("update %s  set ptime = %s where id in
(%s)", $this->mail_table,  $this->db->quote($this->ptime),
implode(',',$id_list));
            $res = $this->db->query($query);
            if (DB::isError($res)) {
              return new Mail_Queue_Error(MAILQUEUE_ERROR_QUERY_FAILED,
                                          $this->pearErrorMode,
E_USER_ERROR, __FILE__, __LINE__,
                                          'DB::query failed -
"'.$query.'" - '.$res->toString());
            }
          }

        return true;
    }

------------------------------------------------------------------------

[2006-03-08 09:14:39] benny dot butler at americanfamilyfunds dot com

Don't blame pear for this one.  I have a a mailing script that sends out
14k emails in the morning.  It just uses mail() to do it, and it has
sent up to 30 to the same person (always if the server load gets to
high.)

I originally had it going as fast as it could, but that created a lot of
dupes, then I had it throttled back to do batches, but some people would
still get 2-3.  Since then I have created another table in mysql to keep
up with that day's log.  I select my email out of one table, then delete
it out of that table, then check my log table to see if it's already
been sent, and if it hasn't send it.  If it has, move on. The the loop
is while(1){mailscript;wait(1)}

This REALLY slows down how I can process, especially since I have over a
million rows in the log, but no dupes since I did this.  Well, there was
one other step.  I also told it, each time it loops, to check the 5min
server load.  If it's over 4, I set the wait to be 20... that keeps the
server load down.

THe biggest cause that I've found for the whole thing is DNS queries
taking to long.

------------------------------------------------------------------------

[2006-03-03 10:20:30] pedro dot vera at gmail dot com

We just got burned by this bug. Ours was set to send 2500 messages every
5 minutes. We sent 500 on the first batch, then about 1200 in the
second. A bunch of people got three messages instead of one. Argh.

------------------------------------------------------------------------

The remainder of the comments for this report are too long. To view
the rest of the comments, please view the bug report online at
    http://pear.php.net/bugs/bug.php?id=2050

-- 
Edit this bug report at http://pear.php.net/bugs/bug.php?id=2050&edit=1