Advice Required - for avoiding conflict betwen qmail-smtpd, rblsmtpd loaded as shared objects in tcpserver

Manvendra Bhangui <[email protected]> Fri, 31 Mar 2017 22:35:23 +0530
Newsgroups gmane.mail.qmail.general
Message-ID <CAOqj+1PQEV0FxHvEi4J=tT-QZ16uyrE4yiyKHnHUkWMsf56HLg@mail.gmail.com>
In recent past I worked on an idea that I had
to eliminate the frequent fork() / exec()
calls made by tcpserver for incoming emails

Summary
-------------
1. the entire smtpd code was written as a function smtp_init()
    and compiled as /usr/lib/indimail/plugins/qmail_smtpd.so
2. the entire rblsmtpd code was written as a function rblsmtpd()
   and compiled as /usr/lib/indimail/plugins/rblsmtpd.so
3. wrote a function load_shared() for tcpserver which is just
    a 40 line function using dlopen(). To avoid messing up
   this email link to this function is here
   https://sourceforge.net/p/indimail/code/ci/master/tree/ucspi-tcp-0.88/load_shared.c

How it works
------------------
You declare environment variables PLUGIN0, PLUGIN1, ... to load
the plugins using dlopen. Normally tcpserver will load a function with
the basename of the plugin after removing ".so" from the name.
One can fine tune the behaviour for each
plugin by defining additional variables (but not necessary) like
PLUGIN0_init, PLUGIN0_dir
The value of PLUGIN0, PLUGIN1, etc variables should
point to the path. e.g.

PLUGIN0=/usr/lib/indimail/plugins/qmail_smtpd.so
PLUGIN1=/usr/lib/indimail/plugins/rblsmtpd.so
PLUGIN0_init=smtp_init
PLUGIN0_dir=/etc/indimail/control

The above causes tcpserver to load qmail_smtpd.so
and execute rblsmtpd() function for rbl and smtp_init() function
for each and every connection. BTW my supervise run file
for qmail-smtpd is like this
QMAILDUID=`/usr/bin/id -u indimail`
NOFILESGID=`/usr/bin/id -g indimail`
ME=`head -1 /etc/indimail/control/me`
HOSTNAME=`uname -n`

if [ -z "$QMAILDUID" -o -z "$NOFILESGID" -o -z "$ME" ]; then
    echo QMAILDUID, NOFILESGID, or ME is unset in
    echo /service/qmail-smtpd.25/run
    sleep 5
    exit 1
fi
if [ ! -f /etc/indimail/control/rcpthosts ]; then
    echo "No /etc/indimail/control/rcpthosts!"
    echo "Refusing to start SMTP listener because it'll create an open relay"
    sleep 5
    exit 1
fi

exec /usr/bin/envdir /service/qmail-smtpd.25/variables sh -c "
exec /usr/bin/softlimit -m \$SOFT_MEM -o 1024 \
/usr/bin/tcpserver -v -h -R -l \$LOCALIP \
-x /etc/indimail/tcp.smtp.cdb \
-c /service/qmail-smtpd.25/variables/MAXDAEMONS -o -b \$MAXDAEMONS \
-u $QMAILDUID -g $NOFILESGID \$LOCALIP \$PORT \$RBLCOMMAND \
/usr/lib/indimail/plugins/qmail_smtpd.so $HOSTNAME  \$AUTHMODULES
/bin/false 2>&1"

Problem Statement
===============
1. There are duplicate functions between the shared
     library of qmail_smtpd.so and rblsmtpd.so. This should
     not have been an issue as I am using the flags
     RTLD_NOW|RTLD_LOCAL|RTLD_NODELETE
     I had hoped that RTLD_LOCAL will ensure that symbols
     local to qmail_smtpd.so will not be used by rblsmtpd()
     and vice versa.

2. However my tcpserver code had a control_readline()
    function which was slightly different than the
    control_readline() function used in qmail_smptd.so.
    On CentOS7 I found that, smtp_init() function loaded
    from qmail_smtpd.so started using the tcpserver's
    control_readlne() function and it resulted in a BUG

3. To resolve this issue, I made both the functions same

4. I did objdump on qmail_smtpd.so and rblsmtpd.so and
    found additional duplicate functions, which I believe
    shoudn't be a problem ???  I used the command
    $ objdump -T rblsmtpd.so |grep ".text" |awk '{print $7}' > /tmp/rbl
    $ objdump -T qmail_smtpd.so |grep ".text"|awk '{print $7}' > /tmp/smtp
    $ cat /tmp/smtp /tmp/rbl |sort |uniq -c|sort -k1|grep ".* 2"
      2 alloc
      2 alloc_free
      2 alloc_re
      2 byte_copy
      2 byte_copyr
      2 byte_diff
      2 byte_zero
      2 case_diffb
      2 case_diffs
      2 commands
      2 env_get
      2 error_str
      2 fmt_ulong
      2 ndelay_on
      2 open_read
      2 scan_ulong
      2 scan_xlong
      2 sig_catch
      2 stralloc_append
      2 stralloc_cat
      2 stralloc_catb
      2 stralloc_cats
      2 stralloc_copy
      2 stralloc_copyb
      2 stralloc_copys
      2 stralloc_ready
      2 stralloc_readyplus
      2 str_chr
      2 str_diff
      2 str_len
      2 str_start

5.  I need to do the same for global variables
     but stuck because of lack of knowledge

6.  I have done some deep reading on dlopen()
     and have additionally added the flag
     RTLD_DEEPBIND available since
     glibc 2.3.4. Removing the duplicate
     functions and RTLD_DEEPBIND has
     fixed the bug i encountered on CentOS7

7.  I have saved on the exec call and copying
     of program space when exec happens
     but loading of control files still happens
     in the child after fork(). My idea is to
     write a separate shared object just
     to load all the control files. But it involves
     some tedious work due to so many global
     variables, especially the stralloc variables

Worries
======
1.  Maybe I have done something stupid here
     eventhough it works for me

2.  I don't have the full knowledge of how
     the ld loader works and half baked
    knowledge is not good.

3.  I believe this will require community effort,
     and I am ready to work under someone's
     guidance. I believe this can be done
     for qmail/netqmail as indimail is basically
     DJB's qmail with some meddling of my own.
     I do believe that this (along with loading
     of control files in the parent) will result
     in some good performance improvement.
     Kai Peter did reach out to me and Erwin
     But somehow, we never got in touch as
     I have to figure out how to use IRC. But
     I am ready to play a developer role if
     someone steps in to revive qmail. I have
     looked at sendmail/postfix code and
     feel it would be a shame have a far
     superior code lose out.

Copying Kai and Eh in this email as I find them still
interested in working to improve qmail and I do once
in a while look at Eh's code. If I borrow his code, I do mention
it in the indimail CREDITS file

-- 
Regards Manvendra - http://www.indimail.org
GPG Pub Key
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC7CBC760014D250C