Re: [Probably found] Re: Please check your maildir stores for 1-byte sized files
Martin Steigerwald <[email protected]>
| Newsgroups | gmane.comp.kde.users.pim |
|---|---|
| Message-ID | <2014270.LMSP8qNMs1@merkaba> |
Am Samstag, 4. Mai 2013, 12:55:45 schrieb Martin Steigerwald: > Am Freitag, 3. Mai 2013, 09:30:15 schrieb O. Sinclair: > > On 02/05/2013 20:47, Martin Steigerwald wrote: > > > Am Donnerstag, 2. Mai 2013, 17:34:05 schrieb Martin Steigerwald: > > >> Hi! > > >> > > >> I stumpled over what I think is a possible data loss again with > > >> my most recent, I think pretty simple setup with KMail and filter > > >> configuration from scratch: > > >> > > >> Bug 319226 - produces 1-byte-sized files on failed move attempts > > >> while filtering http://bugs.kde.org/319226 > > >> > > >> I saw it produce three more of these 1-byte-sized files > > > > > > […] > > > > > >> I still hope its just happening for me and its somehow related to > > >> filter rules, but please sanity check your maildir stores. > > > > > > I suspect its really something with my filter rules, particularily > > > the CRM114 spam filtering integration rules I made. > > > > > > So it may really just happen to me. > > > > > > I removed those CRM114 rules now and will see what happens. > > > > > > Still it would be reassuring to hear from some POP3 users that they > > > do not have one byte sized files in their local maildirs while > > > using (local) filtering themselves. > > > > I find none and I have been using CRM114 though it seems to be botched > > in Kubuntu 13.04 (not first time, worked nicely in 12.10 though). Have > > also removed CRM114 as it simply does not seem to work.. > > Thanks! > > Can you recall the exact filter fules you used? > > I think it would be helpful to compare them with the ones I have posted to > the bug report above. > > After having switched of my CRM114 spam filter rules I did not see further > one- byte sized files. So I´d like to find out whats different with your > rules. > > I am quite sure my CRM114 on Debian Sid with still crm114 package version > 20100106-3 works just fine. It did so a week ago with KMail 1. It never > produced one-byte sized files or other erratic results as far as I can see. > > > For comparison on list here are my CRM114 filter rules. On looking at the > filter rules while pasting them here, I found a possible issue with them > explaining what I was seeing. But still, I´d like to compare with your > filter rules, before trying again, as its quite tedious to restore crunched > mails from POP3 server. > > > > Pipe through crm114 to find if its spam: > > [Filter #0] > Applicability=1 > AutomaticName=false > ConfigureShortcut=false > ConfigureToolbar=false > Enabled=true > Icon=system-run > StopProcessingHere=false > ToolbarName=crm114: Auf Spam prüfen > accounts-set=akonadi_pop3_resource_0 > action-args-0=crm -u ~/.crm114 mailreaver.crm > action-name-0=filter app > actions=1 > apply-on=check-mail,manual-filtering > contentsA=262144 > fieldA=<size> > funcA=less-or-equal > identifier=BfESlQkyajcYUgNg > name=crm114: Auf Spam prüfen > operator=or > rules=1 > > > Set status of mails crm114 marked in rule #0 to spam and filter them to spam > folder: > > [Filter #1] > Applicability=0 > AutomaticName=false > ConfigureShortcut=true > ConfigureToolbar=false > Enabled=true > Icon=system-run > StopProcessingHere=true > ToolbarName=crm114: Spam ausfiltern > action-args-0=Lokale Ordner/Papierkorb/Spam > action-args-1=P > action-name-0=transfer > action-name-1=unset status > actions=2 > apply-on=check-mail,manual-filtering > contentsA=SPAM.* > contentsB=^[2345] > fieldA=X-CRM114-Status > fieldB=X-GMX-Antispam > funcA=regexp > funcB=regexp > identifier=5t65LS2QA9PYTGUe > name=crm114: Spam ausfiltern > operator=or > rules=2 > > > Filter mails crm114 marked as unsure to unsure folder: > > [Filter #2] > Applicability=0 > AutomaticName=false > ConfigureShortcut=false > ConfigureToolbar=false > Enabled=true > Icon=system-run > StopProcessingHere=true > ToolbarName=crm114: Unsicher ausfiltern > action-args-0=Lokale Ordner/Papierkorb/Unsicher > action-name-0=transfer > actions=1 > apply-on=check-mail,manual-filtering > contentsA=UNSURE > fieldA=X-CRM114-Status > funcA=contains > identifier=ui7f6pceYEB2uWgx > name=crm114: Unsicher ausfiltern > operator=or > rules=1 > > > > And add the send manual filters for telling crm114 whether a mail is spam or > good. I had them at the end cause with apply-on=manual-filtering and > StopProcessingHere=true any Ctrl+J ended on them and other mailing list > based filters after it did not work. Now, after having looked at my > original KMail-1 based filters again, I think I´d just omit uncheck > "apply-on=manual-filtering" option. > > [Filter #39] > Applicability=0 > AutomaticName=false > ConfigureShortcut=true > ConfigureToolbar=true > Enabled=true > Icon=mail-mark-notjunk > StopProcessingHere=true > ToolbarName=crm114: Als Gut einstufen > accounts-set=akonadi_pop3_resource_0,akonadi_maildir_resource_0 > action-args-0=H > action-args-1=crm -u ~/.crm114 mailreaver.crm --good > action-args-2=rm -u ~/.crm114 mailreaver.crm > > GOSH! That rm -u should be a "crm", a copy and paste error. I think I > noticed an error like this and fixed it. > > That might explain what I am seeing. > > > action-name-0=unset status > action-name-1=execute > action-name-2=filter app > actions=3 > apply-on=manual-filtering > identifier=XOzgjB9dqty8V2Zn > name=crm114: Als Gut einstufen > operator=all > rules=0 > > > Still that filter was after all mailing list based filters and the one-byte > sized files where sorted into mailing lists by previous filters which have > StopProcessingHere. > > I will look at newer revisions of my filter rules. Seems also to be broken in that way, so I may have not spotted this error. > And well, I also had one-byte sized files with imported filter rules from > KDEPIM-1, which were not having this "rm" in it. Just checked it. > > But still: > > merkaba:/tmp> echo "test" > test > merkaba:/tmp> LANG=C rm -u test > rm: invalid option -- 'u' > Try 'rm --help' for more information. > merkaba:/tmp#1> cat test > test > merkaba:/tmp> > > > So if Akonadi pipes the mail through rm -u, it should get back "rm: invalid > option -- 'u' > Try 'rm --help' for more information." > > or something like that. > > Well, that could somehow have lead to a one byte sized file. I tried just this misconfigured filter rule a ton of time with Ctrl-J and triggering via filter actions menu on two test mails. These are still there. I thought they would be gone. Maybe Akonadi doesn´t accept the pipe result, if the command returns an error code? I don´t even see the rm to be called, but may be to quick to notice. While thats no prove that the rule did not cause any havoc, I have an idea on how to test crm114 filter rules for one-byte sized files (with crm 114 rules enabled) safely: 1) create a pop3 test account 2) stuff a lots of mails to this mail account via a for loop and mail command. 3) retrieve mails only from that test account. I think I will try this way. I found another mistake with the mark as good rule: It should not be called on manual filtering, only by being called from filter action menu or icon. While this should not cause one-byte sized files, it might have contributed to several filters being called quickly one after another and possible race conditions relating that. martin@merkaba:~> while true; do ps aux | grep rm | egrep -v "(grep|btrfs| dirmngr)" ; sleep 0.01 ; done martin 6208 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agentD29042.tmp martin 6209 0.0 0.0 89148 1480 ? S 13:11 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 6263 0.0 0.0 4180 580 ? S 13:11 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agentL29042.tmp martin 6264 0.0 0.0 89144 1468 ? S 13:11 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 6311 0.0 0.0 4180 580 ? S 13:11 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agentF29042.tmp martin 6312 0.0 0.0 89016 1416 ? R 13:11 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 6311 0.0 0.0 4180 580 ? S 13:11 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agentF29042.tmp martin 6312 0.0 0.0 89148 1476 ? R 13:11 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 6361 0.0 0.0 4180 580 ? S 13:11 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agentj29042.tmp martin 6362 0.0 0.0 89152 1480 ? S 13:11 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 6416 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agentg29042.tmp martin 6417 0.0 0.0 89152 1480 ? S 13:11 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 6464 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agenta29042.tmp martin 6465 0.0 0.0 89152 1480 ? S 13:11 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 6515 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agents29042.tmp martin 6516 0.0 0.0 89020 1408 ? S 13:11 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 6515 0.0 0.0 4180 576 ? S 13:11 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agents29042.tmp martin 6516 0.0 0.0 89152 1472 ? S 13:11 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 6533 0.0 0.0 89152 648 ? S 13:11 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 6534 0.0 0.0 4180 576 ? S 13:11 0:00 sh -c . \/mailtrainer.crm --fileprefix= -- good=reaver_cache/known_good/20130504_131157_862193_C8DEACBE -- spam=reaver_cache/empty/ martin 6535 0.0 0.0 93208 1420 ? R 13:11 0:00 /usr/bin/crm ./mailtrainer.crm --fileprefix= -- good=reaver_cache/known_good/20130504_131157_862193_C8DEACBE -- spam=reaver_cache/empty/ martin 7390 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agentQ29042.tmp martin 7392 0.0 0.0 89152 1428 ? S 13:12 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 7390 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agentQ29042.tmp martin 7392 0.0 0.0 89152 1488 ? R 13:12 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 7445 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agenta29042.tmp martin 7447 0.0 0.0 89152 1480 ? S 13:12 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 7445 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agenta29042.tmp martin 7447 0.0 0.0 0 0 ? R 13:12 0:00 [crm] martin 7496 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agente29042.tmp martin 7498 0.0 0.0 89148 1420 ? S 13:12 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 7548 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agentA29042.tmp martin 7549 0.0 0.0 89016 1292 ? R 13:12 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 7548 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agentA29042.tmp martin 7549 0.0 0.0 89148 1472 ? S 13:12 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 7566 0.0 0.0 89148 652 ? S 13:12 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 7567 0.0 0.0 4180 580 ? S 13:12 0:00 sh -c . \/mailtrainer.crm --fileprefix= -- good=reaver_cache/known_good/20130504_131206_383516_C8DEACBE -- spam=reaver_cache/empty/ martin 7568 0.0 0.0 122148 4948 ? R 13:12 0:00 /usr/bin/crm ./mailtrainer.crm --fileprefix= -- good=reaver_cache/known_good/20130504_131206_383516_C8DEACBE -- spam=reaver_cache/empty/ martin 7603 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agentb29042.tmp martin 7604 0.0 0.0 89016 1408 ? S 13:12 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 7603 0.0 0.0 4180 580 ? S 13:12 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agentb29042.tmp martin 7604 0.0 0.0 89148 1472 ? S 13:12 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 7621 0.0 0.0 89148 652 ? S 13:12 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good martin 7622 0.0 0.0 4180 576 ? S 13:12 0:00 sh -c . \/mailtrainer.crm --fileprefix= -- good=reaver_cache/known_good/20130504_131206_586995_C8DEACBE -- spam=reaver_cache/empty/ martin 7623 0.0 0.0 93208 1444 ? R 13:12 0:00 /usr/bin/crm ./mailtrainer.crm --fileprefix= -- good=reaver_cache/known_good/20130504_131206_586995_C8DEACBE -- spam=reaver_cache/empty/ martin 7653 0.0 0.0 4180 576 ? S 13:12 0:00 /bin/sh -c (crm -u ~/.crm114 mailreaver.crm --good) </tmp/kde- martin/akonadi_mailfilter_agents29042.tmp martin 7655 0.0 0.0 89148 1472 ? S 13:12 0:00 crm -u /home/martin/.crm114 mailreaver.crm --good -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 _______________________________________________ KDE PIM users mailing list Subscription management: https://mail.kde.org/mailman/listinfo/kdepim-users