Re: tmda-ofmipd: the dreaded TypeError: Expected list, got <type 'str'>

Stephen Warren wrote:
> Dave Steinberg wrote:
>> GNUs (as I understand it) is a non-smtp MUA.  So it goes like this:
>>
>> GNUs (via sendmail binary) -> postfix -> tmda-ofmipd
> 
> OK. Interesting. I doubt it's the same issue I was thinking of then - 
> it's extremely unlikely that postfix is not "Seeing" tmda-ofmipd just 
> drop a connection.
> 
> To confirm this, please look at the either/both:
> 
> * postfix logs for when it delivers the message to tmda-ofmipd.

Here are the customer's postfix logs, you can see it gets a 250 from 
tmda-ofmipd:

Dec  5 12:00:41 pascal postfix/pickup[5888]: 72211300B6: uid=1000
from=<REDACTED1>
Dec  5 12:00:41 pascal postfix/cleanup[12187]: 72211300B6:
message-id=<[email protected]>
Dec  5 12:00:41 pascal postfix/qmgr[5889]: 72211300B6: from=<REDACTED1>,
size=1711, nrcpt=2 (queue active)
Dec  5 12:00:41 pascal postfix/local[12189]: 72211300B6:
to=<[email protected]>, orig_to=<nils@pascal>, relay=local, delay=0.1,
delays=0.06/0.01/0/0.03, dsn=2.0.0, status=sent (delivered to mailbox)
Dec  5 12:00:46 pascal postfix/smtp[12190]: 72211300B6: to=<REDACTED2>,
relay=mail.geekisp.com[216.168.135.169]:8025, delay=4.7, 
delays=0.06/0.02/2.4/2.2,
dsn=2.0.0, status=sent (250 Ok)
Dec  5 12:00:46 pascal postfix/qmgr[5889]: 72211300B6: removed

> * Run tmda-ofmipd in debug mode and you should also see all the protocol 
> responses that get sent back.

Switched.  I'll ask the customer to send the mail again.

> Given the flow you mentioned, I expect it to log a that tmda-ofmipd 
> reported a 250 message indicating successful forwarding of the message 
> to the configured destination.

That's indeed what the postfix logs say.  I'll confirm when I have more 
from the tmda-ofmipd logs.

>> Error: failure delivering message to command 
>> "['/usr/local/bin/tmda-inject', 'REDACTED2']" (comman
>> d "['/usr/local/bin/tmda-inject', 'REDACTED2']" exited 1  (Traceback 
>> (most recent call last):
>> File "/usr/local/bin/tmda-inject", line 116, in ?
>> orig_msgout_as_string = Util.msg_as_string(msgout) 
>> File "./TMDA/Util.py", line 552, in msg_as_string
>> File "./TMDA/pythonlib/email/generator.py", line 84, in flatten
>> File "./TMDA/pythonlib/email/generator.py", line 109, in _write
>> File "./TMDA/pythonlib/email/generator.py", line 135, in _dispatch
>> File "./TMDA/pythonlib/email/generator.py", line 266, in _handle_message
>> File "./TMDA/pythonlib/email/message.py", line 185, in get_payload
>> TypeError: Expected list, got <type 'str'>))
> 
> This exception is inside tmda-inject. It would be interesting to know 
> why it's happening, but it's probably best to concentrate first on why 
> the error isn't propagated to tmda-ofmipd and hence back to postfix.
> 
> Looking at the code, I honestly can't see why this is happening. I'll 
> need to think about it a little. Sorry.

Fair enough, and no need to apologize - I really appreciate your 
attention to my problem!

Regarding the tmda-ofmipd change you suggested - the code in question, 
in my version, is almost exactly as you suggested I make it.  Here's 
what I did:

Original (slightly reformatted for readability):

if not opts.throttlescript or not os.system("%s %s" % (opts.throttlescript,
   self.__auth_username)):
try:
     status = self.__server.process_message(self.__peer,
					   self.__mailfrom,
					   self.__rcpttos,
					   self.__data,
					   self.__auth_username)
except:
     print >>DEBUGSTREAM, "process_message raised an exception:"
     import traceback
     traceback.print_exc(DEBUGSTREAM)
     raise
else:
   status = self.push('450 Outgoing mail quota exceeded')

Mine:

if not opts.throttlescript or not os.system("%s %s" % (opts.throttlescript,
   self.__auth_username)):
try:
     status = self.__server.process_message(self.__peer,
					   self.__mailfrom,
					   self.__rcpttos,
					   self.__data,
					   self.__auth_username)
except:
     status = self.push('451 Internal exception in tmda-ofmipd')
     print >>DEBUGSTREAM, "process_message raised an exception:"
     import traceback
     traceback.print_exc(DEBUGSTREAM)
     raise
else:
   status = self.push('450 Outgoing mail quota exceeded')

... all I did was add the "status = self.push('451..." line.

Clearly the 'raise' would be called in this case, so perhaps we're 
looking at the wrong code path?

>> PS - I'll try your patch for tmda-ofmipd in the morning.  Any chance it 
>> could make its way into the trunk?
> 
> There is some disagreement whether it's the right thing to do; see the 
> archives...

Looks like most of it got implemented, just short of the setting the status.

Per your 2nd mail, other details:

I am running tmda-ofmipd as the vpopmail user, and I am using vpopmail 
virtual domains.  I'm running it via svscan, and the run script is:

#!/bin/sh
PATH=$PATH:/usr/local/bin
QMAILQUEUE="bin/qmail-dk"
RELAYCLIENT=""
DKSIGN="/etc/domainkeys/%/default"

export QMAILQUEUE RELAYCLIENT DKSIGN

exec 2>&1
exec /usr/local/bin/setuidgid vpopmail \
         /usr/local/bin/tmda-ofmipd -f -p localhost:7025 \
                 -A "/home/vpopmail/bin/vchkpw /bin/true" \
                 -S /usr/local/tmda/contrib/vpopmail-vdir.sh \
                 --pure-proxy --debug

I believe tmda-inject is called directly.  I wasn't aware of any hooks 
that would let me call something else, even if I wanted to!

Thanks again for your help,
-- 
Dave Steinberg
http://www.geekisp.com/
http://www.steinbergcomputing.com/