Re: Work-around to issues with Akonadi file based caching (was: Re: rant)

Hi Martin,

Thank you for your email. Unfortunately, after rebuilding akonadi database, 
upgrading to the akonadi version you mentioned, and increasing the file 
threshold to 32768, I still have to watch for the size of the folder 
file_db_data. I checked it for the first time in about 2 weeks yesterday, after 
your email, and found out that it grew to a size of 25 GB, sigh. After 
akonadictl fsck, it went down to its "normal" size of 4.5 GB, and the rest 
ended up in the lost+found folder, which was eventually deleted. I am not sure 
why this is happening and I don't have the expertise to find out. I truly 
believe that one of the best pieces of KDE suite is KDEPIM, and I would love 
to see it working more reasonably soon.

Thank you again for all your help and guidance.

Hakan

On Thursday, March 12, 2015 02:07:43 PM Martin Steigerwald wrote:
> Am Freitag, 27. Februar 2015, 09:58:04 schrieb Martin Steigerwald:
> > Hakan,
> > 
> > Am Sonntag, 15. Februar 2015, 17:19:45 schrieb E. Hakan Duran:
> > > On Friday, February 13, 2015 10:34:28 AM Martin Steigerwald wrote:
> > > > ...
> > > > The SizeThreshold thing helped me with that. But have caution: This
> > > > is
> > > > no official recommendation of Akonadi developers. Still, it gave me
> > > > a
> > > > good improvement with performance and latency *and* seemed to have
> > > > solved the insane amount of file and data in file_db_data issue for
> > > > me. Or at least it worked around it successfully.
> > > >
> > > >...
> > > 
> > > Thank you once again Martin. I made the changes you recommended. I
> > > think it feels a little more responsive now, but I have to use it a
> > > few more days to make a more objective call.
> > > 
> > > I still notice a growing file_db_data folder, proportional to the time
> > > akonadi is left on. I am considering to delete everything and start
> > > from scratch but I have so many accounts (e-mail, personal and work
> > > calendars, etc.) it feels like a waste of my time at the moment.
> > > Instead, I will keep turning akonadi off as soon as I check and reply
> > > to my emails, and then run fsck afterwards to get rid of the mess it
> > > generated until I have more time for the revamp or I am truly pissed
> > > 
> > > :).
> > 
> > so far this change works for me.
> > 
> > Several POP3 accounts, one very large, all in one maildir, and a 30-day
> > limited IMAP account:
> > 
> > martin@merkaba:~/.local/share/akonadi> find file_db_data | wc -l
> > 32
> > 
> > martin@merkaba:~/.local/share/akonadi#1> du -sh file_db_data
> > 4,3M    file_db_data
> > 
> > Very large IMAP account from work:
> > 
> > ms@merkaba:~/.local/share/akonadi> find file_db_data | wc -l
> > 2376
> > ms@merkaba:~/.local/share/akonadi> du -sh file_db_data
> > 663M    file_db_data
> > 
> > 
> > Hmmm, yikes. Its large in there, but at least not very many files. And
> > well, it seems that are mails with larger attachment that Akonadi caches
> > there:
> > 
> > ms@merkaba:~/.local/share/akonadi> du -sh file_db_data/* | sort -rh |
> > head 11M     file_db_data/4735240_r1
> > 11M     file_db_data/4735240_r0
> > 11M     file_db_data/4735239_r1
> > 11M     file_db_data/4735239_r0
> > 9,4M    file_db_data/4734016_r1
> > 9,4M    file_db_data/4734016_r0
> > 9,3M    file_db_data/4731257_r0
> > 8,6M    file_db_data/4731853_r1
> > 8,6M    file_db_data/4731853_r0
> > 7,2M    file_db_data/4732214_r1
> > 
> > And as I think I set the account to offline caching that even makes
> > sense.
> > 
> > 
> > Well all it comes down to is: If you have lots of mails that are larger
> > than 32 KiB the change won´t help. You may try with an even larger
> > SizeThreshhold, but as I did never test it and no of no one else who
> > did, I can´t say whether it would work. Also I don´t know the maximum
> > limit for SizeThreshold, I don´t know the limit of MySQL for storing
> > these blobs in the database.
> > 
> > All I see here is that SizeThreshold=32768 works okay here.
> > 
> > 
> > These figures are weeks after the change and I didn´t do any manual
> > optimization aka akonadictl fsck or akonadictl vacuum. I may try it on
> > the work account next week to see whether Akonadi again lost track of
> > some files.
> 
> This still seems to work for me:
> 
> Since my work-around with SizeThreshold=32768 on my private setup:
> 
> martin@merkaba:~/.local/share/akonadi> find file_db_data | wc -l
> 33
> martin@merkaba:~/.local/share/akonadi> du -sh file_db_data
> 4,4M    file_db_data
> martin@merkaba:~/.local/share/akonadi> du -sh file_db_data/* | sort -rh |
> head -10
> 896K    file_db_data/2815963_r0
> 584K    file_db_data/2630687_r0
> 452K    file_db_data/2630658_r0
> 448K    file_db_data/2630655_r0
> 368K    file_db_data/2488539_r0
> 164K    file_db_data/2758167_r0
> 152K    file_db_data/2488220_r0
> 120K    file_db_data/2488152_r0
> 104K    file_db_data/2565672_r0
> 100K    file_db_data/2943194_r0
> 
> 
> And on my laptop work setup:
> 
> ms@merkaba:~/.local/share/akonadi> find file_db_data | wc -l
> 2442
> 
> ms@merkaba:~/.local/share/akonadi> du -sh file_db_data
> 703M    file_db_data
> 
> still quite much, but I set the account to offline caching and I think it
> cached mails with larger attachments:
> 
> ms@merkaba:~/.local/share/akonadi> du -sh file_db_data/* | sort -rh | head
> -10
> 11M     file_db_data/4735240_r1
> 11M     file_db_data/4735240_r0
> 11M     file_db_data/4735239_r1
> 11M     file_db_data/4735239_r0
> 9,4M    file_db_data/4734016_r1
> 9,4M    file_db_data/4734016_r0
> 9,3M    file_db_data/4731257_r0
> 8,6M    file_db_data/4731853_r2
> 8,6M    file_db_data/4731853_r1
> 8,6M    file_db_data/4731853_r0
> 
> Ciao,

_______________________________________________
KDE PIM users mailing list
Subscription management: https://mail.kde.org/mailman/listinfo/kdepim-users

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEABECAAYFAlUDGjQACgkQu5WqKQPUC0suZgCgsIPu66aVzoyMErW0GBKwp2wh
h2cAnjtr0Te7k1EwP+DBewQT5T4C6vrz
=Mz+G
-----END PGP SIGNATURE-----