Re: How to disable mail indexing with Akonadi search

Am Montag, 8. Februar 2016, 10:15:55 CET schrieb Daniel Vrátil:
> On Saturday, February 6, 2016 7:35:17 PM CET Martin Steigerwald wrote:
> > Back in Baloo times it was easy to disable mail indexing. But with Akonadi
> > Search which as I understand is basically a renamed Baloo for mail stuff I
> > didn´t find a way yet. Can one disable it?
> > 
> > I searched for an option in akonadi_indexing_agent settings, but didn´t
> > find one.
> > 
> > I excluded the biggest folders yet it still crawls Akonadi to a halt with
> > at least 90-100% CPU load on mysqld for tenth of minutes or more up to
> > the point where something happens that doesn´t happen with Akonadi +
> > KDEPIM from master otherwise since all the performance improvements:
> > KMail looses connection to Akonadi and is unusable unless I restart it.
> > 
> > Also currently the mail index would grow to big that BTRFS gets into
> > trouble with free space, up to kernel 4.3 even freezing up the whole
> > machine with a kworker thread searching for free space (known issue in
> > BTRFS, that may or may not be fixed in 4.4 or later).
> > 
> > For now at least for the storage thing I did:
> > 
> > touch ~/.local/share/akonadi/search_db
> > 
> > then akonadi indexing agent does not create a directory there and no space
> > is used up.
> > 
> > As you know my mail setup is a bit crazy, so I know the index is big, but
> > still after 5 GiB I keep wondering whether it needs to be that big,
> > especially as I excluded at least the folders with LKML mails.
> > 
> > For now until I have more free SSD space I just want to disable it.
> > 
> > I help myself with repeating
> > 
> > killall /home/kde/install/bin/akonadi_indexing_agent
> > 
> > until Akonadi gives up starting the indexing agent after each time I start
> > Akonadi by logging into a Plasma session or restarting it after a new
> > compile from master.
> 
> You can prevent the Indexer from starting by modifying /usr/share/akonadi/
> agents/akonadindexingagent.desktop, unsetting "AutoStart" capability.
> 
> 
> You use maildir, so the main suspect why the IO goes crazy is that when
> indexing is triggered, we are forced to upload all the mails to Akonadi and
> index it, then Akonadi expires the data again and they are removed from the
> database - so you get maildir READ + DB write, then DB read, then Xapian
> write (super-IO-intensive) and again DB write (delete) - I suppose this
> together adds up to some insane IO numbers.

I thought about this again after having had the massive reduction of db size 
on akonadi maintenance with akonadictl fsck ; akonadictl vacuum.

I missed one start of the indexing agent and it did some work (despite it 
could save the Xapian database cause of my touch ~/.local/share/akonadi/
search_db and I think its better off with not doing any work at all if it can 
not save it).

And after that db was about 1,4 GiB larger again.

Then I wondered I still have SizeThreshold=16768 in my akonadiserverrc back 
from time where Akonadi massively leaked files in file_db_data and then it 
didn´t have the multi leveled cache. But when I understand you correctly on 
indexing it would stuff all mails below 16 KiB into the database and as MySQL 
AFAIK does not auto-vacuum they stay there.

So I will attempt now to let the indexer run, but without my special 
SizeThreshold setting, so it takes the default of 4 KiB again. Maybe this way 
it will complete a full indexing run without blocking up Akonadi completely.

> I am aware that this is extremely inefficient and I have a plan how to
> improve performance for local email massively by allowing direct access to
> the local remote storage by resources and applications with Akonadi only
> storing the usual metadata and path to the actual end-storage files
> (instead of the actual data). Unfortunately I don't know when I'll get
> around to start working on it. If you have some time and feel like helping,
> just get in touch ;)

As written in the other mails I think its pretty complex, but if you have a 
junior job sized chunk on what I could work, feel free to share.

Thanks,
-- 
Martin
_______________________________________________
KDE PIM users mailing list
Subscription management: https://mail.kde.org/mailman/listinfo/kdepim-users