Next steps? Occasional hangs - new info and ideas

I'm trying to track down what I think are multiple, but related issues  
here.  First, I "think" (but am clearly not sure) that  
lbm_run_filters_on_reception_idle_cb is being called too often, which  
then calls libbalsa_mailbox_message_match, which calls  
message_match_real, which calls  libbalsa_utf8_strst, which seems to be  
what is actually taking most of the time.  Separately, I'm still not  
sure about is which of those functions is actually blocking anything  
else from happening.  (Could there be a threading problem?)  Anyway, I  
want to add some debug output to those functions, but in as  
non-intrusive a way as possible.  I'm open to suggestions about what  
functions might be most appropriate.  I suppose I could just output to  
stderr, which would avoid opening/closing a specific debug file for the  
purpose.  Would there be any point in creating a new G_MESSAGES_DEBUG  
category, such as "filtering" ?

Thanks for any suggestions or pointers on some appropriate reading.

Jack


On 2018.10.29 18:07, Jack wrote:
> Trimming down, leaving only enough to perhaps tickle some memories:
> 
> On 2018.09.11 21:00, Jack wrote:
>>> On 2018.08.28 15:13, Albrecht Dreß wrote:
>>>> Am 28.08.18 20:57 schrieb(en) Jack via balsa-list:
>>> [snip other issues...]
>>>>> 3) For the past several weeks, I have been noticing an  
>>>>> occasional, unpredictable hang/pause in Balsa when downloading  
>>>>> messages.  […]  I'm looking for any thoughts on how to  
>>>>> troubleshoot this better.
> 
>>>> You might want to run Balsa with all debug messages  
>>>> (G_MESSAGES_DEBUG=all) or only low-level networking messages  
>>>> (G_MESSAGES_DEBUG=libnetclient) enabled and check if anything  
>>>> unusual happens with the problematic mailbox.  Remember that  
>>>> passwords will be printed in the output, so please sanitise any  
>>>> files before posting them.
>>> I've done this, but seen no messages that would indicate any  
>>> particular problem.  However, the hangs mostly seem to be after  
>>> Balsa finished downloading a message (often the last downloaded  
>>> line in the DEBUG is a '.')  so I'm now wondering if the hang might  
>>> be caused by checking gpg signatures - and like magic, I have had  
>>> this issue in the past, and there was a thread in February about  
>>> it.  You provided some notes on checking gpg configuration, which  
>>> I'll have to read and follow up on again.  I have 504 "pub" keys  
>>> (from "gpg -k") and I wouldn't consider that a particularly large  
>>> keyring - or am I wrong?  (Browsing through the list of keys, I  
>>> suspect most got there by due to messages from various mailing  
>>> lists.)
>>> 
>>> However, two more questions:  First, as I said in February, why  
>>> would this check (or whatever is currently happening) completely  
>>> block all Balsa activity?  Second, in my Inbox, I do have "Decrypt  
>>> and check signatures automatically" set to "Never" so it's  
>>> obviously not exactly the same problem I was having back them.
>>> 
>>> Anyway - I've updated several bits of my gpg configuration, and I  
>>> know I've got lots to go, but even if I'm imagining it, I seem to  
>>> be having less of the hangs, but not totally gone yet.
>> OK, I've now run with G_MESSAGES_DEBUG=crypto and seen the hangs  
>> with no crypto related output at all to console.  Does that  
>> exonerate any gpg stuff?  If so, where do I look next?
> Right after my last reply, I cleaned up several issues with my gpg  
> config, so I'm now more convinced that is not the problem.  I'm now  
> wondering if it might be related to scanning an applying filters.   
> However, I recompiled balsa with profiling support, and have the  
> grpof output of one very long run (several hundred messages -  
> returning from almost a week travel).  I'm not at all sure I'm  
> correctly interpreting the output, but it looks like  
> libbalsa_utf8_strstr is taking the most time, and it is called by  
> message_match_real, called by libbalsa_mailbox_message_match, called  
> by lbm_run_filters_on_reception_idle_cb.
> 
> So, I have lines of questions.  First, does anybody have more  
> experience interpreting the gprof output?  If so, the gprof output is  
> about 5000 lines, so probably not suitable for posting to a message  
> here (does the mailing list allow attachments?).  I'll be glad to  
> post here or two send individually.
> 
> Second, I assume "cb" is for callback - which implies that the  
> filters are called every time a message is added to the mailbox.  If  
> this tries to filter every message in the mailbox, instead of just  
> the new one, that might be something to consider changing, although I  
> can imagine times it would be appropriate to filter the entire  
> mailbox.  The other option would be to only call the filters after  
> all new messages have been added.  I know if I'm retrieving lots of  
> messages, I sometimes read some in the mailbox to which they have  
> been filtered, before all messages are retrieved, but I think I'd be  
> willing to wait (or just read in the inbox) if it would stop these  
> freezes.  As part of this question, though, it seems to be  
> specifically libbalsa_utf8_strstr taking all the time - so I wonder  
> if either it could be optimised, or if there is something funny about  
> my configuration which is causing a problem others do not see.
> 
> For some additional info about the freezes - when it happens, the  
> balsa itself is totally unresponsive.  I can adjust the window size  
> (which I assume is done by the window manager, not balsa itself) but  
> balse itself waits another many seconds before redrawing the window  
> to reflect the new size, or to reflect if I've clicked on another  
> mailbox or another message.  Even the "Checking Mail..." freezes - I  
> cannot move it, and the sliding bars under each server being checked  
> stop moving.  These freezes can last up to a minute or so, and seem  
> to restart with each retrieved message, which seems to support my  
> theory above.
> 
> Sorry to be so long winded, but I'm really hoping to provide enough  
> information to either identify the problem, or to let someone give me  
> some ideas on further troubleshooting.
> 
> Thanks.
> 
> Jack
_______________________________________________
balsa-list mailing list
[email protected]
https://mail.gnome.org/mailman/listinfo/balsa-list