Re: Long delays between spooling and delivering

--0000000000001984aa063b8782ef
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Quan

Yes I agree it's the default date when messages are first added to the
sendQueue, but to my mind that should trigger immediate processing to
MailDelivrerToHost, whereas these messages were sat with that timestamp for
over 48 hours.
When messages are retried the next delivery date gets updated to the next
try time, so they weren't in retry state.
Restarting the server caused them to be dequeued and delivered immediately.

My RemoteDelivery mailet config below.

        <processor state=3D"relay" enableJmx=3D"true">
            <mailet match=3D"All" class=3D"RemoteDelivery">
                <outgoingQueue>outgoing</outgoingQueue>
                <delayTime>
1*10 minute,
2*30 minute,
4*1 hour,
8*2 hour,
4*6 hour
</delayTime>
                <maxRetries>20</maxRetries>
                <maxDnsProblemRetries>5</maxDnsProblemRetries>
                <deliveryThreads>20</deliveryThreads>
                <sendpartial>true</sendpartial>
                <bounceProcessor>bounces</bounceProcessor>
<debug>true</debug>
<startTLS>true</startTLS> <!-- added by MP 2025-07-17-->
<mail.smtp.starttls.required>false</mail.smtp.starttls.required>
<mail.smtp.ssl.protocols>TLSv1.3 TLSv1.2 TLSv1.1 TLSv1 SSLv3
SSLv2Hello</mail.smtp.ssl.protocols>

            </mailet>
        </processor>


Kind regards
Matt Pryor
Research and Development Manager

The International Presence Group of Companies
EMAIL: [email protected]
URL: www.International-presence.com




On Mon, 4 Aug 2025 at 11:03, Quan tran hong <[email protected]>
wrote:

> Hi,
>
> IMO, there is nothing harmful about the delivery date of 1969.
>
> I guess you get it from the mail queue route. It was likely the result
> of Unix epoch plus  -1 millisecond (the -1 default delay, which means the=
re
> is no delay) cf
>
> https://github.com/apache/james-project/blob/master/server/queue/queue-jm=
s/src/main/java/org/apache/james/queue/jms/JMSCacheableMailQueue.java#L705
>
> As long as the delivery time is prior to the current system time, then th=
e
> email should be dequeued and delivered. cf
>
> https://github.com/apache/james-project/blob/master/server/queue/queue-jm=
s/src/main/java/org/apache/james/queue/jms/JMSCacheableMailQueue.java#L529
>
> Who knows, something is wrong with your time system?
>
> Otherwise, I am still curious about your remote delivery metrics. I doubt
> remote delivery could fail, so it takes time for the mail to be retried. =
If
> you are too, you can try to turn on the metrics webadmin routes:
>
> https://github.com/apache/james-project/blob/master/server/grafana-report=
ing/prometheus-datasource/README.md#expose-metrics-for-prometheus-collectio=
n
>
> Also, sharing the RemoteDelivery mailet configuration (especially
> *delayTime*,* maxRetries*...) could help too.
>
> Quan
>
>
> On Mon, Aug 4, 2025 at 4:19=E2=80=AFPM Matt Pryor <
> [email protected]>
> wrote:
>
> > Hi Quan, that webadmin feature doesn't seem to exist:
> >
> > {"statusCode":404,"type":"notFound","message":"GET /metrics can not be
> > found","details":null}
> >
> > As an aside after restarting James those messages with delivery date of
> > 1969 were immediately picked up and sent, but looking through the logs =
it
> > seems they were queued on Friday 1st August at 16:44.
> >
> > Kind regards
> > Matt Pryor
> > Research and Development Manager
> >
> > The International Presence Group of Companies
> > EMAIL: [email protected]
> > URL: www.International-presence.com
> >
> >
> >
> >
> > On Mon, 4 Aug 2025 at 10:02, Quan tran hong <[email protected]=
m
> >
> > wrote:
> >
> > > Hi Matt,
> > >
> > > Can you share the remote delivery timer metric?
> > > *curl **localhost:8000/metrics*, then you can search for the
> > > *RemoteDeliveryTrial* metrics.
> > >
> > > Quan
> > >
> > > On Mon, Aug 4, 2025 at 3:45=E2=80=AFPM Matt Pryor <
> > > [email protected]>
> > > wrote:
> > >
> > > > Thanks Quan.
> > > >
> > > > The delays I'm seeing are between initial spooling and the very fir=
st
> > > > delivery attempt.
> > > >
> > > > Currently seeing a number of messages in the outgoing mailQueue wit=
h
> > next
> > > > delivery of
> > > >
> > > > 1969-12-31T23:59:59.999Z
> > > > They do not seem to be progressing.
> > > > I've noticed that when a message is first queued it has this date b=
ut
> > > only
> > > > for a very short period until it's picked up by the deliverer.
> > > > Suggests to me that some thread has died?
> > > > A call to the healthcheck suggests everything is fine:
> > > >
> > > > {"status":"healthy","checks":[{"componentName":"Guice application
> > > >
> > > >
> > >
> >
> lifecycle","escapedComponentName":"Guice%20application%20lifecycle","stat=
us":"healthy","cause":null},{"componentName":"MailReceptionCheck","escapedC=
omponentName":"MailReceptionCheck","status":"healthy","cause":null},{"compo=
nentName":"EventDeadLettersHealthCheck","escapedComponentName":"EventDeadLe=
ttersHealthCheck","status":"healthy","cause":null},{"componentName":"EmptyE=
rrorMailRepository","escapedComponentName":"EmptyErrorMailRepository","stat=
us":"healthy","cause":null},{"componentName":"Embedded
> > > >
> > > >
> > >
> >
> ActiveMQ","escapedComponentName":"Embedded%20ActiveMQ","status":"healthy"=
,"cause":null}]}
> > > >
> > > >
> > > >
> > > > Kind regards
> > > > Matt Pryor
> > > > Research and Development Manager
> > > >
> > > > The International Presence Group of Companies
> > > > EMAIL: [email protected]
> > > > URL: www.International-presence.com
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, 1 Aug 2025 at 18:14, Quan tran hong <
> > [email protected]
> > > >
> > > > wrote:
> > > >
> > > > > Hi Matt,
> > > > >
> > > > > > What could be the cause of this I wonder? The server doesn't se=
em
> > > > > massively
> > > > > busy in between those times.
> > > > >
> > > > > In my opinion, something could be wrong with your SMTP remote
> gateway
> > > > that
> > > > > makes James (here RemoteDelivery mailet) can not send mail outbou=
nd
> > > right
> > > > > away. Then, RemoteDelivery would delay some time before retrying
> the
> > > > > next remote delivery, which could explain the delay time you
> > observed.
> > > > >
> > > > > You can send an inbound email to verify the theory. If inbound is
> > fast,
> > > > > then the problem is likely at the SMTP remote gateway.
> > > > >
> > > > > Quan
> > > > >
> > > > > On Fri, Aug 1, 2025 at 8:33=E2=80=AFPM Matt Pryor <
> > > > > [email protected]>
> > > > > wrote:
> > > > >
> > > > > > I'm seeing long periods where MailDelivrerToHost isn't mentione=
d
> in
> > > the
> > > > > > logs even though there are plenty of emails being spooled.
> > > > > >
> > > > > > How can I find out what's going on?
> > > > > >
> > > > > > I've increased the delivery threads to 20 and the spool threads
> to
> > > 200
> > > > > but
> > > > > > if anything it's made it worse.
> > > > > >
> > > > > > Customer now screaming at me.
> > > > > >
> > > > > > E.g.
> > > > > > 2025-08-01 13:11:41.199 [DEBUG] o.a.j.m.i.JamesMailSpooler - =
=3D=3D=3D=3D
> > End
> > > > > > processing mail
> > > > > > Mail1754053901091-441b1ec5-bce3-40ee-af08-94d5a109240e-
> > to-gmail.com
> > > > =3D=3D=3D=3D
> > > > > >
> > > > > > Then... nothing.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Kind regards
> > > > > > Matt Pryor
> > > > > > Research and Development Manager
> > > > > >
> > > > > > The International Presence Group of Companies
> > > > > > EMAIL: [email protected]
> > > > > > URL: www.International-presence.com
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Fri, 1 Aug 2025 at 10:59, Matt Pryor <
> > > > > [email protected]>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi, I'm running Apache James 3.82.
> > > > > > >
> > > > > > > I'm seeing long delays between spooling messages and then
> > > delivering
> > > > > > them.
> > > > > > > For example:
> > > > > > >
> > > > > > >  2025-08-01 04:22:22.253 [DEBUG] o.a.j.m.i.JamesMailSpooler -
> > =3D=3D=3D=3D
> > > > End
> > > > > > > processing mail
> > > > > > > Mail1754022142176-e4670a57-8cf4-4412-934e-f0981e559015-
> > > to-xxxxx.com
> > > > > =3D=3D=3D=3D
> > > > > > >
> > > > > > >  And then.... 2025-08-01 04:41:00.785 [DEBUG]
> > > > > > > o.a.j.t.m.r.d.DeliveryRunnable - will process mail
> > > > > > > Mail1754022142176-e4670a57-8cf4-4412-934e-f0981e559015-
> > > to-xxxxx.com
> > > > > > >
> > > > > > > What could be the cause of this I wonder? The server doesn't
> seem
> > > > > > > massively busy in between those times.
> > > > > > >
> > > > > > > I upgraded from 3.80 to 3.82 in the hope this would fix the
> > problem
> > > > but
> > > > > > it
> > > > > > > doesn't seem to have.
> > > > > > >
> > > > > > > Kind regards
> > > > > > > Matt Pryor
> > > > > > > Research and Development Manager
> > > > > > >
> > > > > > > The International Presence Group of Companies
> > > > > > > EMAIL: [email protected]
> > > > > > > URL: www.International-presence.com
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

--0000000000001984aa063b8782ef--