Re: How to disable mail indexing with Akonadi search

Am Donnerstag, 11. Februar 2016, 12:12:11 CET schrieb Martin Steigerwald:
> Am Montag, 8. Februar 2016, 10:15:55 CET schrieb Daniel Vrátil:
> > On Saturday, February 6, 2016 7:35:17 PM CET Martin Steigerwald wrote:
> > > Back in Baloo times it was easy to disable mail indexing. But with
> > > Akonadi
> > > Search which as I understand is basically a renamed Baloo for mail stuff
> > > I
> > > didn´t find a way yet. Can one disable it?
> > > 
> > > I searched for an option in akonadi_indexing_agent settings, but didn´t
> > > find one.
> > > 
> > > I excluded the biggest folders yet it still crawls Akonadi to a halt
> > > with
> > > at least 90-100% CPU load on mysqld for tenth of minutes or more up to
> > > the point where something happens that doesn´t happen with Akonadi +
> > > KDEPIM from master otherwise since all the performance improvements:
> > > KMail looses connection to Akonadi and is unusable unless I restart it.
> > > 
> > > Also currently the mail index would grow to big that BTRFS gets into
> > > trouble with free space, up to kernel 4.3 even freezing up the whole
> > > machine with a kworker thread searching for free space (known issue in
> > > BTRFS, that may or may not be fixed in 4.4 or later).
> > > 
> > > For now at least for the storage thing I did:
> > > 
> > > touch ~/.local/share/akonadi/search_db
> > > 
> > > then akonadi indexing agent does not create a directory there and no
> > > space
> > > is used up.
> > > 
> > > As you know my mail setup is a bit crazy, so I know the index is big,
> > > but
> > > still after 5 GiB I keep wondering whether it needs to be that big,
> > > especially as I excluded at least the folders with LKML mails.
> > > 
> > > For now until I have more free SSD space I just want to disable it.
> > > 
> > > I help myself with repeating
> > > 
> > > killall /home/kde/install/bin/akonadi_indexing_agent
> > > 
> > > until Akonadi gives up starting the indexing agent after each time I
> > > start
> > > Akonadi by logging into a Plasma session or restarting it after a new
> > > compile from master.
> > 
> > You can prevent the Indexer from starting by modifying /usr/share/akonadi/
> > agents/akonadindexingagent.desktop, unsetting "AutoStart" capability.
> > 
> > 
> > You use maildir, so the main suspect why the IO goes crazy is that when
> > indexing is triggered, we are forced to upload all the mails to Akonadi
> > and
> > index it, then Akonadi expires the data again and they are removed from
> > the
> > database - so you get maildir READ + DB write, then DB read, then Xapian
> > write (super-IO-intensive) and again DB write (delete) - I suppose this
> > together adds up to some insane IO numbers.
> 
> I thought about this again after having had the massive reduction of db size
> on akonadi maintenance with akonadictl fsck ; akonadictl vacuum.
> 
> I missed one start of the indexing agent and it did some work (despite it
> could save the Xapian database cause of my touch ~/.local/share/akonadi/
> search_db and I think its better off with not doing any work at all if it
> can not save it).
> 
> And after that db was about 1,4 GiB larger again.
> 
> Then I wondered I still have SizeThreshold=16768 in my akonadiserverrc back
> from time where Akonadi massively leaked files in file_db_data and then it
> didn´t have the multi leveled cache. But when I understand you correctly on
> indexing it would stuff all mails below 16 KiB into the database and as
> MySQL AFAIK does not auto-vacuum they stay there.
> 
> So I will attempt now to let the indexer run, but without my special
> SizeThreshold setting, so it takes the default of 4 KiB again. Maybe this
> way it will complete a full indexing run without blocking up Akonadi
> completely.

Jup, I think this is it. I removed the setting and tada:

martin@merkaba:~/.local/share/akonadi> date ; find file_db_data | wc -l
Do 11. Feb 12:29:12 CET 2016
8180
martin@merkaba:~/.local/share/akonadi> date ; find file_db_data | wc -l
Do 11. Feb 12:29:19 CET 2016
8864
martin@merkaba:~/.local/share/akonadi> date ; find file_db_data | wc -l
Do 11. Feb 12:29:29 CET 2016
9826

So still extremely inefficient, yet according to atop much less load on MySQL 
database. Around 20-50% cpu load from some quick glances instead of around 
100%.

I/O load is actually quite light around 5-10% on each of the SSDs.

Lets see how it goes.

Thanks,
-- 
Martin
_______________________________________________
KDE PIM users mailing list
Subscription management: https://mail.kde.org/mailman/listinfo/kdepim-users