Re: Slow process-quarantine

To close this, thanks for the comparative stats.  I was able to fix the 
performance on the process-quarantine with some mysql tuning on the i/o 
parameters (innodb settings).  Brought the performance up to about 3-4 
messages/sec.

-Alan

On 5/3/2019 1:03 PM, Alan Sparks wrote:
> A question for you all as well - are you using the "original" script 
> from http://www.maiamailguard.com/maia/wiki (e.g., 1.0.2 or earlier), 
> or the later forked ones from 
> https://github.com/technion/maia_mailguard ("1.0.4 unofficial")?
>
> I am using the latter, I note Ed's below is logging the "-sub" part 
> that seems to be removed from this.  The fork seems to be different, 
> not sure if that makes a difference.
>
> Are those of you with fast performance using an SQL database, or files?
>
> -Alan
>
> On 5/3/2019 12:51 PM, Ed wrote:
>> Hello,
>>
>> 2019-05-03 14:40:07 Maia: [process-quarantine-sub] Learned mail item 
>> 25296296 as spam and reported it
>> 2019-05-03 14:40:10 Maia: [process-quarantine-sub] Learned mail item 
>> 25296657 as spam and reported it
>> 2019-05-03 14:40:10 Maia: [process-quarantine-sub] 2 spam items 
>> processed (2 learned, 2 reported)
>> 2019-05-03 14:40:10 Maia: [process-quarantine-sub] 0 non-spam items 
>> processed (0 learned)
>> 2019-05-03 14:40:10 Maia: [process-quarantine-sub] 0 spam items 
>> processed (0 learned, 0 reported)
>> 2019-05-03 14:40:10 Maia: [process-quarantine-sub] 0 non-spam items 
>> processed (0 learned)
>> 2019-05-03 14:40:11 Maia: [process-quarantine] Processing completed: 
>> 2 items processed in 1 run.
>>
>>
>> Looks like processed 2 messages in < 4 seconds so about 2 sec per 
>> message or about 3x faster than yours.
>>
>> I should get a bigger run in the morning, we'll where that goes.
>>
>> --Ed
>>
>> On 5/3/19 11:42 AM, Alan Sparks wrote:
>>> Is that from a different process? expire-quarantine-cache.pl maybe?
>>>
>>> Those stats are not shown in my process-quarantine output.
>>>
>>> -Alan
>>>
>>> On 5/3/2019 9:25 AM, Ed wrote:
>>>> Hi,
>>>>
>>>> expired old bayes database entries in 2 seconds
>>>> 118933 entries kept, 76770 deleted
>>>> token frequency: 1-occurrence tokens: 59.76%
>>>> token frequency: less than 8 occurrences: 17.77%
>>>>
>>>> Mine runs with RHEL6 and mysql as well p;retty much the same setup 
>>>> as you.  Dual cpu 8 total cores.  I don't think it takes advantage 
>>>> of parallel processing however.
>>>>
>>>> --Ed
>>>>
>>>> On 5/3/19 11:11 AM, Alan Sparks wrote:
>>>>> I have maia 1.0.4 set up on a machine (CentOS 6, RAID1, i3-4130T 
>>>>> processor).  Works fine.
>>>>>
>>>>> Except the process quarantine stuff is pretty slow.  I do have the 
>>>>> bayes stuff (SA 3.3.1 from EPEL6) set up to use MySQL database, 
>>>>> and a average result in the process-quarantine.log looks something 
>>>>> like:
>>>>>
>>>>>
>>>>> 2019-05-03 09:05:33  53 of 53 emails processed (100%) : 51 learned 
>>>>> ham, 2 learned spam, 0 reported spam, 0 skipped
>>>>> 2019-05-03 09:05:33  Processed 53 emails in 331 seconds (0.16 
>>>>> emails/sec)
>>>>> 2019-05-03 09:05:33  Shutting down
>>>>>
>>>>> Is this normal or average for people?  Seems like it should learn 
>>>>> faster than that.  Checked MySQL slow logs, and see no queries 
>>>>> running long (at least not longer than 10 seconds).
>>>>>
>>>>> -Alan
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Maia-users mailing list
>>>>> [email protected]
>>>>> http://www.renaissoft.com/cgi-bin/mailman/listinfo/maia-users
>>>>
>>>
>>> _______________________________________________
>>> Maia-users mailing list
>>> [email protected]
>>> http://www.renaissoft.com/cgi-bin/mailman/listinfo/maia-users
>>
>
> _______________________________________________
> Maia-users mailing list
> [email protected]
> http://www.renaissoft.com/cgi-bin/mailman/listinfo/maia-users

_______________________________________________
Maia-users mailing list
[email protected]
http://www.renaissoft.com/cgi-bin/mailman/listinfo/maia-users