Re: Apache server returns early before process is complete

--0000000000008f47a70633767287
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Timeout happens because of how we handle the request. Timeout is basically
no response came back. Why that happens is because we think we want to have
a correct response. Unfortunately for long running requests the correct
response shouldn't be via http response code or we face situations like
this. Instead reply with a 200 OK immediately and then provide correct
status in the message body. Once a response code/header has been sent
timeout won't trigger and you could potentially hold the connection for
hours without a problem.

On Wed, Apr 23, 2025, 9:32=E2=80=AFAM Andreas Mock <[email protected]> wr=
ote:

> Hi Joseph,
>
> your description is very vague, so can only answer on some assumptions:
>
> It sounds like a timeout is fired somewhere.
>
> Best advice in these situations: Log as many steps as you can. Keep your
> eyes open on TCP/IP and higher level timeouts.
>
> Declare only ONE instance responsible for a retry: Either the app server
> calling the dispatcher with several tries or the dispatcher trying for
> himself. Not both.
>
> Best regards
> Andreas
>
>
> Am 23.04.2025 um 16:21 schrieb Joseph He:
> > All, good day.
> >
> > Here is the issue I have.
> > My entire application is running on ModPerl/Apache environment.
> > I send Http::Request with data load from my App server to a dispatch
> > server thru LWP::UserAgent, I set the timeout 600 seconds.
> >
> > The dispatch server is supposed to manipulate the data and send the
> > data to an external SFTP server. Because the SFTP can fail, it will
> > keep trying up to 4 times with 30 seconds sleep in case that SFTP
> > connection fails.
> >
> > Recently, I found that I uploaded the file twice sometimes. I figured
> > out the root cause is that my Dispatch server returns 'failure' at 6
> > minutes while it keeps trying to do the SFTP. The App server
> > received HTTP::Response with error status so it issued another call to
> > send data. It turns out I uploaded the identified file twice.
> >
> > Anybody has this sort of experience? Why does the dispatch server
> > return 'error' while it still processes the data?
> >
> > Thanks a lot,
> > Joseph
> >
>

--0000000000008f47a70633767287
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto">Timeout happens because of how we handle the request. Tim=
eout is basically no response came back. Why that happens is because we thi=
nk we want to have a correct response. Unfortunately for long running reque=
sts the correct response shouldn&#39;t be via http response code or we face=
 situations like this. Instead reply with a 200 OK immediately and then pro=
vide correct status in the message body. Once a response code/header has be=
en sent timeout won&#39;t trigger and you could potentially hold the connec=
tion for hours without a problem.</div><br><div class=3D"gmail_quote gmail_=
quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Apr 23, 2025=
, 9:32=E2=80=AFAM Andreas Mock &lt;<a href=3D"mailto:[email protected]">a=
[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote=
" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);=
padding-left:1ex">Hi Joseph,<br>
<br>
your description is very vague, so can only answer on some assumptions:<br>
<br>
It sounds like a timeout is fired somewhere.<br>
<br>
Best advice in these situations: Log as many steps as you can. Keep your <b=
r>
eyes open on TCP/IP and higher level timeouts.<br>
<br>
Declare only ONE instance responsible for a retry: Either the app server <b=
r>
calling the dispatcher with several tries or the dispatcher trying for <br>
himself. Not both.<br>
<br>
Best regards<br>
Andreas<br>
<br>
<br>
Am 23.04.2025 um 16:21 schrieb Joseph He:<br>
&gt; All, good day.<br>
&gt;<br>
&gt; Here is the issue I have.<br>
&gt; My entire application is running on ModPerl/Apache environment.<br>
&gt; I send Http::Request with data load from my App server to a dispatch <=
br>
&gt; server thru LWP::UserAgent, I set the timeout 600 seconds.<br>
&gt;<br>
&gt; The dispatch server is supposed to manipulate the data and send the <b=
r>
&gt; data to an external SFTP server. Because the SFTP can fail, it will <b=
r>
&gt; keep trying up to 4 times with 30 seconds sleep in case that SFTP <br>
&gt; connection fails.<br>
&gt;<br>
&gt; Recently, I found that I uploaded the file twice sometimes. I figured =
<br>
&gt; out the root cause is that my Dispatch server returns &#39;failure&#39=
; at 6 <br>
&gt; minutes while it keeps trying to do the SFTP. The App server <br>
&gt; received=C2=A0HTTP::Response with error status so it issued another ca=
ll to <br>
&gt; send data. It turns out I uploaded the identified file twice.<br>
&gt;<br>
&gt; Anybody has this sort of experience? Why does the dispatch=C2=A0server=
 <br>
&gt; return &#39;error&#39; while it still processes the data?<br>
&gt;<br>
&gt; Thanks a lot,<br>
&gt; Joseph<br>
&gt;<br>
</blockquote></div>

--0000000000008f47a70633767287--