Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd

On 4 April 2017 at 20:20, Malte Tancred <[email protected]> wrote:
> On 4 Apr 2017, at 16:11, Manvendra Bhangui wrote:
>
>> On 2 April 2017 at 20:47, Charles Cazabon
>> <[email protected]> wrote:
>
> ...
>>>
>>> As djb himself says, measure, don't speculate.
>
>
>> So finally I measured and now I have data, and it so consistent, that
>> I too am surprised. Using dlopen() to load the control files in
>> tcpserver gives a huge, huge gain if the control files are large.
>
>
> Nice work.
>
> Just a thought though. It seems to me that control files weren't designed
> to handle huge lists of this sort and that a better approach would be to
> move to a more efficient file format or database, say, cdb, sqlite, LMDB
> or postgresql.

Yes. I use mysql. So if a control file is named as badmailfrom.sql,
then qmail-smtpd looks for a table named badmailfrom and reads
badmailfrom.sql for the user, password, database and port/socket.
Advantage of postgresql/mysql is that just one control file like
badmailfrom, etc need to be maintained for all your qmail relay
servers running in your network. I deliberately used a 25 Mib file to
expose the performance issue that can occur in a fork() --> exec
design. My quest is to find an answer for a private namespace for the
loaded shared libraries so that a malicious code be prevented from
modifying tcpserver's global variables or exposing functions with the
same name in tcpserver. The man page for dlmopen suggests that dlmopen
is the answer. However, I have not been able to make dlmopen() work.
tcpserver runs with 100% cpu and even strace doesn't work. Sometimes
technology can be so difficult :(