Re: IMAP issue with CONDSTORE

David Rebarchik <[email protected]> Fri, 30 Apr 2021 04:05:02 -0500
Newsgroups gmane.mail.imap.dbmail
Message-ID <[email protected]>
This is a multi-part message in MIME format.
--===============7828463842687973162==
Content-Type: multipart/alternative;
 boundary="------------B986562D8378F0D0C3AE8D85"
Content-Language: en-US

This is a multi-part message in MIME format.
--------------B986562D8378F0D0C3AE8D85
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit

Alan,

After one day it isn't working. Same symptom as before. I've not put my 
"modified" version back yet, but probably will soon.

When I tested it right after installing the fix, it seemed to work. I 
sent several test messages and they appeared on the android phone and in 
Thunderbird as expected. When I "read" them, that updated between 
devices as expected. But, I noticed at the end of the day that my phone 
stopped working.

The symptom is that my cellphone (using Samsung Mail) stops updating. 
The only way I've gotten it to update is to remove the mail account and 
re-add it.  But that then only downloads the mail at that point.  It 
still doesn't update.  I've not figured out why exactly, but if I modify 
the advertised features of the IMAP server, it works.  I use Thunderbird 
on my Windows desktop and that has always mostly worked, with my change 
in the advertised features it sometimes stops updating, but I can close 
and reopen it and it starts working again.

I'm willing to test and provide log files. Just tell me what you need.

Thank you for your help.

Thanks,
Dave

On 4/29/2021 9:48 AM, David Rebarchik wrote:
> Alan,
>
> Thank you for the notice that a fix was pushed.  I just downloaded and 
> complied it. My initial testing shows it is fixed.  I've installed it 
> on my production server and will continue to watch. If there is 
> anything seeming wrong, I'll let you know.
>
> Thank you very much for spending time on this project. I really love it.
>
> Thanks,
> Dave
>
> On 4/29/2021 6:02 AM, Alan Hicks wrote:
>>
>> Hi,
>>
>> Is this still an issue? A fix for MODSEQ related to CONDSTORE was 
>> pushed a few days ago that should have resolved this one.
>>
>> Kind regards,
>> Alan
>>
>> On 03/04/2021 23:43, David Rebarchik wrote:
>>> Bobby,
>>>
>>> The reason for finally upgrading to a newer version was the TLS 
>>> support in the newer version.  With 2.2 I was able to run things 
>>> through stunnel, but many clients, Outlook being one, started 
>>> defaulting to STARTTLS instead of using the SSL port. This caused 
>>> many of my users trouble setting up accounts on their computers. So, 
>>> I decided to upgrade so STARTTLS would work.
>>>
>>> Another reason for upgrading is that I has having trouble compiling 
>>> the older 2.2 version on a newer OS. The older code needed older 
>>> libraries and it was getting hard to get them all to work, while 
>>> upgrading to newer code was easier.
>>>
>>> When doing the upgrade, the most amount of time was spent doing the 
>>> header conversion.  I recommend adding the entry 
>>> "header_cache_readonly = yes" to dbmail.conf after you have the 
>>> needed header name values. On my hardware with the number of 
>>> messages I had it took over 4 months to do the conversion. (2.6 
>>> million messages.) If I had put that line in, it would have taken 
>>> only a few weeks.
>>>
>>> During the conversion I had two databases setup, dbmail22 and 
>>> dbmail3, then added triggers to insert and delete rows when things 
>>> changed in dbmail22 so the dbmail3 database had all the same 
>>> messages. This way all my users could still use IMAP and when the 
>>> conversion was done and tested, I just moved the IP from my old mail 
>>> server to the new, running the new version, and there was only about 
>>> half hour of downtime.
>>>
>>> Other than this issue with CONDSTORE, which happened several weeks 
>>> after I started using 3.2, I have had little trouble with the basic 
>>> use of DBMail 3.2. I wish that dbmail-timsieved also supported 
>>> STARTTLS, but I see that as optional and haven't opened that issue 
>>> yet. I've also noticed that it takes 3 to 5 seconds to insert a 
>>> message, which is tolerable for mail delivery, but when an IMAP 
>>> client is moving messages to the server, it takes a LONG time, but 
>>> this doesn't happen often.
>>>
>>> I hope that some of these things help when you decide to upgrade to 
>>> 3.2.
>>>
>>> I still need help getting CONDSTORE working again!!!
>>>
>>> Thanks,
>>> Dave
>>>
>>> On 4/2/2021 10:46 AM, Bobby wrote:
>>>> Re: [Dbmail] IMAP issue with CONDSTORE Hello Dave,
>>>>
>>>> I am still able to use DBMail 2.2.18 and I must say it's been maybe 
>>>> even 20 years since the first install.
>>>> So, thank you for sharing this info, as I am planning an upgrade 
>>>> too, and because I have a similar problem with Outlook Mobile since 
>>>> a 2-3 months (don't remember exactly).
>>>> But I am not sure if the reason is not TLS related.
>>>>
>>>> Please share details once you know more, if they are not provided 
>>>> in the list. Thank you!
>>>>
>>>>
>>>> Kind regards,
>>>> Bobby
>>>>
>>>> Friday, March 19, 2021, 7:51:00 AM, you wrote:
>>>>
>>>> *> I'm hoping that this list is still active. I've been using 
>>>> DBMail for 18
>>>> > years, since version 2.0.
>>>>
>>>> > I've had very little issues with it over the years, but I've 
>>>> recently
>>>> > upgraded from 2.2 to 3.2.5-beta.  Everything was working for a few
>>>> > weeks, until last Friday. My cell phone stopped getting updates, 
>>>> while
>>>> > Thunderbird was still working fine. I found that when I clear out 
>>>> the
>>>> > account and re-add it I was able to download mail, but it would 
>>>> never
>>>> > update again after that. Similar behavior was happening with the 
>>>> IMAP
>>>> > web client (IMP) I am using.
>>>>
>>>> > With a lot of digging through detailed log files, I found that it 
>>>> was
>>>> > due to CONDSTORE, so I removed it from the line in dbmail.h #define
>>>> > IMAP_CAPABILITY_STRING. After that both the web client and the 
>>>> mobile
>>>> > device seem to "work" but now Thunderbird is acting strange, it 
>>>> updates
>>>> > most of the time, but not all the time. When it stops updating, I 
>>>> close
>>>> > and re-open it, then it refreshes and works for a while.
>>>>
>>>> > I don't want to leave the system crippled like this. I want it to 
>>>> work
>>>> > efficiently. What could cause CONDSTORE to stop working?  It was 
>>>> fine
>>>> > for several weeks.  I'm willing to do tests and post log files, 
>>>> but I'm
>>>> > not sure what is needed.
>>>>
>>>> > Thanks,
>>>> > Dave
>>>> > _______________________________________________
>>>> > DBmail mailing list
>>>> *> [email protected] <mailto:[email protected]>
>>>> > https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail 
>>>> <https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Best regards,
>>>> Bobby
>>>> [email protected] <mailto:[email protected]>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> DBmail mailing list
>>>> [email protected]
>>>> https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail
>>>
>>>
>>> _______________________________________________
>>> DBmail mailing list
>>> [email protected]
>>> https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail
>>
>>
>> _______________________________________________
>> DBmail mailing list
>> [email protected]
>> https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail
>


--------------B986562D8378F0D0C3AE8D85
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <font face="monospace">Alan,<br>
      <br>
      After one day it isn't working. Same symptom as before. I've not
      put my "modified" version back yet, but probably will soon.<br>
      <br>
      When I tested it right after installing the fix, it seemed to
      work. I sent several test messages and they appeared on the
      android phone and in Thunderbird as expected. When I "read" them,
      that updated between devices as expected. But, I noticed at the
      end of the day that my phone stopped working.<br>
      <br>
      The symptom is that my cellphone (using Samsung Mail) stops
      updating. The only way I've gotten it to update is to remove the
      mail account and re-add it.  But that then only downloads the mail
      at that point.  It still doesn't update.  I've not figured out why
      exactly, but if I modify the advertised features of the IMAP
      server, it works.  I use Thunderbird on my Windows desktop and
      that has always mostly worked, with my change in the advertised
      features it sometimes stops updating, but I can close and reopen
      it and it starts working again.<br>
      <br>
      I'm willing to test and provide log files. Just tell me what you
      need.<br>
      <br>
      Thank you for your help.<br>
      <br>
      Thanks,<br>
      Dave<br>
    </font><br>
    <div class="moz-cite-prefix">On 4/29/2021 9:48 AM, David Rebarchik
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:[email protected]">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <font face="monospace">Alan,<br>
        <br>
        Thank you for the notice that a fix was pushed.  I just
        downloaded and complied it. My initial testing shows it is
        fixed.  I've installed it on my production server and will
        continue to watch. If there is anything seeming wrong, I'll let
        you know.<br>
        <br>
        Thank you very much for spending time on this project. I really
        love it.<br>
        <br>
        Thanks,<br>
        Dave<br>
      </font><br>
      <div class="moz-cite-prefix">On 4/29/2021 6:02 AM, Alan Hicks
        wrote:<br>
      </div>
      <blockquote type="cite"
        cite="mid:[email protected]">
        <meta http-equiv="Content-Type" content="text/html;
          charset=UTF-8">
        <p>Hi,</p>
        <p> Is this still an issue? A fix for MODSEQ related to
          CONDSTORE was pushed a few days ago that should have resolved
          this one.</p>
        <p>Kind regards,<br>
          Alan<br>
        </p>
        <div class="moz-cite-prefix">On 03/04/2021 23:43, David
          Rebarchik wrote:<br>
        </div>
        <blockquote type="cite"
cite="mid:[email protected]">
          <meta http-equiv="Content-Type" content="text/html;
            charset=UTF-8">
          <font face="monospace">Bobby,<br>
            <br>
            The reason for finally upgrading to a newer version was the
            TLS support in the newer version.  With 2.2 I was able to
            run things through stunnel, but many clients, Outlook being
            one, started defaulting to STARTTLS instead of using the SSL
            port. This caused many of my users trouble setting up
            accounts on their computers. So, I decided to upgrade so
            STARTTLS would work.<br>
            <br>
            Another reason for upgrading is that I has having trouble
            compiling the older 2.2 version on a newer OS. The older
            code needed older libraries and it was getting hard to get
            them all to work, while upgrading to newer code was easier.<br>
            <br>
            When doing the upgrade, the most amount of time was spent
            doing the header conversion.  I recommend adding the entry
            "header_cache_readonly = yes" to dbmail.conf after you have
            the needed header name values. On my hardware with the
            number of messages I had it took over 4 months to do the
            conversion. (2.6 million messages.) If I had put that line
            in, it would have taken only a few weeks.<br>
            <br>
            During the conversion I had two databases setup, dbmail22
            and dbmail3, then added triggers to insert and delete rows
            when things changed in dbmail22 so the dbmail3 database had
            all the same messages. This way all my users could still use
            IMAP and when the conversion was done and tested, I just
            moved the IP from my old mail server to the new, running the
            new version, and there was only about half hour of downtime.<br>
            <br>
            Other than this issue with CONDSTORE, which happened several
            weeks after I started using 3.2, I have had little trouble
            with the basic use of DBMail 3.2. I wish that
            dbmail-timsieved also supported STARTTLS, but I see that as
            optional and haven't opened that issue yet. I've also
            noticed that it takes 3 to 5 seconds to insert a message,
            which is tolerable for mail delivery, but when an IMAP
            client is moving messages to the server, it takes a LONG
            time, but this doesn't happen often.<br>
          </font><font face="monospace"><font face="monospace"><br>
              I hope that some of these things help when you decide to
              upgrade to 3.2.  <br>
              <br>
              I still need help getting CONDSTORE working again!!!<br>
              <br>
            </font>Thanks,<br>
            Dave<br>
          </font><br>
          <div class="moz-cite-prefix">On 4/2/2021 10:46 AM, Bobby
            wrote:<br>
          </div>
          <blockquote type="cite"
            cite="mid:[email protected]">
            <title>Re: [Dbmail] IMAP issue with CONDSTORE</title>
            <meta http-equiv="Content-Type" content="text/html;
              charset=UTF-8">
            <span style=" font-family:'Calibri'; font-size: 11pt;">Hello
              Dave,<br>
              <br>
              I am still able to use DBMail 2.2.18 and I must say it's
              been maybe even 20 years since the first install. <br>
              So, thank you for sharing this info, as I am planning an
              upgrade too, and because I have a similar problem with
              Outlook Mobile since a 2-3 months (don't remember
              exactly).<br>
              But I am not sure if the reason is not TLS related.<br>
              <br>
              Please share details once you know more, if they are not
              provided in the list. Thank you!<br>
              <br>
              <br>
              Kind regards,<br>
              Bobby<br>
              <br>
              Friday, March 19, 2021, 7:51:00 AM, you wrote:<br>
              <br>
              <span style=" color: #800000;"><b>&gt; I'm hoping that
                  this list is still active. I've been using DBMail for
                  18 <br>
                  &gt; years, since version 2.0.<br>
                  <br>
                  &gt; I've had very little issues with it over the
                  years, but I've recently <br>
                  &gt; upgraded from 2.2 to 3.2.5-beta.  Everything was
                  working for a few <br>
                  &gt; weeks, until last Friday. My cell phone stopped
                  getting updates, while <br>
                  &gt; Thunderbird was still working fine. I found that
                  when I clear out the <br>
                  &gt; account and re-add it I was able to download
                  mail, but it would never <br>
                  &gt; update again after that. Similar behavior was
                  happening with the IMAP <br>
                  &gt; web client (IMP) I am using.<br>
                  <br>
                  &gt; With a lot of digging through detailed log files,
                  I found that it was <br>
                  &gt; due to CONDSTORE, so I removed it from the line
                  in dbmail.h #define <br>
                  &gt; IMAP_CAPABILITY_STRING. After that both the web
                  client and the mobile <br>
                  &gt; device seem to "work" but now Thunderbird is
                  acting strange, it updates <br>
                  &gt; most of the time, but not all the time. When it
                  stops updating, I close <br>
                  &gt; and re-open it, then it refreshes and works for a
                  while.<br>
                  <br>
                  &gt; I don't want to leave the system crippled like
                  this. I want it to work <br>
                  &gt; efficiently. What could cause CONDSTORE to stop
                  working?  It was fine <br>
                  &gt; for several weeks.  I'm willing to do tests and
                  post log files, but I'm <br>
                  &gt; not sure what is needed.<br>
                  <br>
                  &gt; Thanks,<br>
                  &gt; Dave<br>
                  &gt; _______________________________________________<br>
                  &gt; DBmail mailing list<br>
                </b></span></span><a style=" font-family:'calibri';
              font-size: 11pt;" href="mailto:[email protected]"
              moz-do-not-send="true">&gt; [email protected]</a><br>
            <a style=" font-family:'calibri'; font-size: 11pt;"
              href="https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail"
              moz-do-not-send="true">&gt;
              https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail</a><br>
            <br>
            <br>
            <br>
            <br>
            <br>
            <span style=" font-family:'Calibri'; font-size: 11pt;">Best
              regards,<br>
              Bobby<br>
            </span><a style=" font-family:'calibri'; font-size: 11pt;"
              href="mailto:[email protected]" moz-do-not-send="true">[email protected]</a><br>
            <br>
            <br>
            <br>
            <br>
            <fieldset class="mimeAttachmentHeader"></fieldset>
            <pre class="moz-quote-pre" wrap="">_______________________________________________
DBmail mailing list
<a class="moz-txt-link-abbreviated" href="mailto:[email protected]" moz-do-not-send="true">[email protected]</a>
<a class="moz-txt-link-freetext" href="https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail" moz-do-not-send="true">https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail</a>
</pre>
          </blockquote>
          <br>
          <br>
          <fieldset class="mimeAttachmentHeader"></fieldset>
          <pre class="moz-quote-pre" wrap="">_______________________________________________
DBmail mailing list
<a class="moz-txt-link-abbreviated" href="mailto:[email protected]" moz-do-not-send="true">[email protected]</a>
<a class="moz-txt-link-freetext" href="https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail" moz-do-not-send="true">https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail</a>
</pre>
        </blockquote>
        <div class="moz-signature"><br>
        </div>
        <br>
        <fieldset class="mimeAttachmentHeader"></fieldset>
        <pre class="moz-quote-pre" wrap="">_______________________________________________
DBmail mailing list
<a class="moz-txt-link-abbreviated" href="mailto:[email protected]" moz-do-not-send="true">[email protected]</a>
<a class="moz-txt-link-freetext" href="https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail" moz-do-not-send="true">https://lists.nfg.nl/cgi-bin/mailman/listinfo/dbmail</a>
</pre>
      </blockquote>
      <br>
    </blockquote>
    <br>
  </body>
</html>

--------------B986562D8378F0D0C3AE8D85--

--===============7828463842687973162==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: inline

X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KREJtYWlsIG1h
aWxpbmcgbGlzdApEQm1haWxAbGlzdHMubmZnLm5sCmh0dHBzOi8vbGlzdHMubmZnLm5sL2NnaS1i
aW4vbWFpbG1hbi9saXN0aW5mby9kYm1haWwK

--===============7828463842687973162==--