Re: Apache server returns early before process is complete
Mithun Bhattacharya <[email protected]> Wed, 23 Apr 2025 17:17:16 -0500
| Newsgroups | gmane.comp.apache.mod-perl |
|---|---|
| Message-ID | <CAHTPV9JjKEcGPQjwe78xuZH5838yZiJsCxOe71r-fLD_8Ltvdw@mail.gmail.com> |
--00000000000062e9b6063379775e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Okay agreed that is a valid time out basically it is saying that a client has established tcp/ip connection but has not put its request either a get put or a post On Wed, Apr 23, 2025, 3:38=E2=80=AFPM Joseph He <[email protected]> = wrote: > On Apache2 doc, I found this. How does this timeout work? It looks like i= t > can only wait for 300 seconds before failing a request. > > https://httpd.apache.org/docs/2.0/mod/core.html#timeout > Description: > <https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Description> A= mount > of time the server will wait for certain events before failing a request > Syntax: <https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Syntax= > > TimeOut seconds > Default: > <https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Default> TimeO= ut > 300 > Context: > <https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Context> serve= r > config, virtual host > Status: <https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Status= > > Core > Module: <https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Module= > > core > > The TimeOut directive currently defines the amount of time Apache will > wait for three things: > > 1. The total amount of time it takes to receive a GET request. > 2. The amount of time between receipt of TCP packets on a POST or PUT > request. > 3. The amount of time between ACKs on transmissions of TCP packets in > responses. > > We plan on making these separately configurable at some point down the > road. The timer used to default to 1200 before 1.2, but has been lowered = to > 300 which is still far more than necessary in most situations. It is not > set any lower by default because there may still be odd places in the cod= e > where the timer is not reset when a packet is sent. > > On Wed, Apr 23, 2025 at 3:07=E2=80=AFPM Mithun Bhattacharya <mithnb@gmail= .com> > wrote: > >> You configure timeout at the client side. Apache is at the server side. >> Server doesn't have a concept of time it could take days to run and not >> care. >> >> mod_perl code is where you are sending the http return status to make >> sure the client doesn't timeout waiting for the server to respond. >> >> On Wed, Apr 23, 2025, 2:19=E2=80=AFPM Joseph He <[email protected]= m> wrote: >> >>> Thanks, all. >>> Is that Apache timeout controlled by its configuration "Timeout"? >>> I don't think it has anything to do with modPerl. Am I missing somethin= g? >>> Thanks. >>> >>> On Wed, Apr 23, 2025 at 1:41=E2=80=AFPM Mithun Bhattacharya <mithnb@gma= il.com> >>> wrote: >>> >>>> Timeout happens because of how we handle the request. Timeout is >>>> basically no response came back. Why that happens is because we think = we >>>> want to have a correct response. Unfortunately for long running reques= ts >>>> the correct response shouldn't be via http response code or we face >>>> situations like this. Instead reply with a 200 OK immediately and then >>>> provide correct status in the message body. Once a response code/heade= r has >>>> been sent timeout won't trigger and you could potentially hold the >>>> connection for hours without a problem. >>>> >>>> On Wed, Apr 23, 2025, 9:32=E2=80=AFAM Andreas Mock <[email protected]= e> wrote: >>>> >>>>> Hi Joseph, >>>>> >>>>> your description is very vague, so can only answer on some assumption= s: >>>>> >>>>> It sounds like a timeout is fired somewhere. >>>>> >>>>> Best advice in these situations: Log as many steps as you can. Keep >>>>> your >>>>> eyes open on TCP/IP and higher level timeouts. >>>>> >>>>> Declare only ONE instance responsible for a retry: Either the app >>>>> server >>>>> calling the dispatcher with several tries or the dispatcher trying fo= r >>>>> himself. Not both. >>>>> >>>>> Best regards >>>>> Andreas >>>>> >>>>> >>>>> Am 23.04.2025 um 16:21 schrieb Joseph He: >>>>> > All, good day. >>>>> > >>>>> > Here is the issue I have. >>>>> > My entire application is running on ModPerl/Apache environment. >>>>> > I send Http::Request with data load from my App server to a dispatc= h >>>>> > server thru LWP::UserAgent, I set the timeout 600 seconds. >>>>> > >>>>> > The dispatch server is supposed to manipulate the data and send the >>>>> > data to an external SFTP server. Because the SFTP can fail, it will >>>>> > keep trying up to 4 times with 30 seconds sleep in case that SFTP >>>>> > connection fails. >>>>> > >>>>> > Recently, I found that I uploaded the file twice sometimes. I >>>>> figured >>>>> > out the root cause is that my Dispatch server returns 'failure' at = 6 >>>>> > minutes while it keeps trying to do the SFTP. The App server >>>>> > received HTTP::Response with error status so it issued another call >>>>> to >>>>> > send data. It turns out I uploaded the identified file twice. >>>>> > >>>>> > Anybody has this sort of experience? Why does the dispatch server >>>>> > return 'error' while it still processes the data? >>>>> > >>>>> > Thanks a lot, >>>>> > Joseph >>>>> > >>>>> >>>> --00000000000062e9b6063379775e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto">Okay agreed that is a valid time out basically it is sayi= ng that a client has established tcp/ip connection but has not put its requ= est either a get put or a post</div><br><div class=3D"gmail_quote gmail_quo= te_container"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Apr 23, 2025, 3= :38=E2=80=AFPM Joseph He <<a href=3D"mailto:[email protected]">jo= [email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_q= uote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,2= 04);padding-left:1ex"><div dir=3D"ltr">On Apache2 doc, I found this. How do= es this timeout work? It looks like it can only wait for 300 seconds before= failing a request.=C2=A0<div><br></div><div><a href=3D"https://httpd.apach= e.org/docs/2.0/mod/core.html#timeout" target=3D"_blank" rel=3D"noreferrer">= https://httpd.apache.org/docs/2.0/mod/core.html#timeout</a><br><div><table = style=3D"font-size:14px;border:1px solid rgb(170,170,170);border-collapse:c= ollapse;padding:2px;margin-top:0.5em;margin-bottom:1em;color:rgb(0,51,102)"= ><tbody><tr><th style=3D"empty-cells:show;padding:0.1em 0.2em;vertical-alig= n:top;text-align:left;line-height:1.3em"><a href=3D"https://httpd.apache.or= g/docs/2.0/mod/directive-dict.html#Description" style=3D"color:rgb(0,115,19= 9);background-color:inherit" target=3D"_blank" rel=3D"noreferrer">Descripti= on:</a></th><td style=3D"empty-cells:show;padding:0.1em 0.2em;vertical-alig= n:top;line-height:1.3em">Amount of time the server will wait for certain ev= ents before failing a request</td></tr><tr><th style=3D"empty-cells:show;pa= dding:0.1em 0.2em;vertical-align:top;text-align:left;line-height:1.3em"><a = href=3D"https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Syntax" s= tyle=3D"color:rgb(0,115,199);background-color:inherit" target=3D"_blank" re= l=3D"noreferrer">Syntax:</a></th><td style=3D"empty-cells:show;padding:0.1e= m 0.2em;vertical-align:top;line-height:1.3em"><code style=3D"font-family:&q= uot;Courier New",Courier,monospace;font-size:1em">TimeOut=C2=A0<var>se= conds</var></code></td></tr><tr><th style=3D"empty-cells:show;padding:0.1em= 0.2em;vertical-align:top;text-align:left;line-height:1.3em"><a href=3D"htt= ps://httpd.apache.org/docs/2.0/mod/directive-dict.html#Default" style=3D"co= lor:rgb(0,115,199);background-color:inherit" target=3D"_blank" rel=3D"noref= errer">Default:</a></th><td style=3D"empty-cells:show;padding:0.1em 0.2em;v= ertical-align:top;line-height:1.3em"><code style=3D"font-family:"Couri= er New",Courier,monospace;font-size:1em">TimeOut 300</code></td></tr><= tr><th style=3D"empty-cells:show;padding:0.1em 0.2em;vertical-align:top;tex= t-align:left;line-height:1.3em"><a href=3D"https://httpd.apache.org/docs/2.= 0/mod/directive-dict.html#Context" style=3D"color:rgb(0,115,199);background= -color:inherit" target=3D"_blank" rel=3D"noreferrer">Context:</a></th><td s= tyle=3D"empty-cells:show;padding:0.1em 0.2em;vertical-align:top;line-height= :1.3em">server config, virtual host</td></tr><tr><th style=3D"empty-cells:s= how;padding:0.1em 0.2em;vertical-align:top;text-align:left;line-height:1.3e= m"><a href=3D"https://httpd.apache.org/docs/2.0/mod/directive-dict.html#Sta= tus" style=3D"color:rgb(0,115,199);background-color:inherit" target=3D"_bla= nk" rel=3D"noreferrer">Status:</a></th><td style=3D"empty-cells:show;paddin= g:0.1em 0.2em;vertical-align:top;line-height:1.3em">Core</td></tr><tr><th s= tyle=3D"empty-cells:show;padding:0.1em 0.2em;vertical-align:top;text-align:= left;line-height:1.3em"><a href=3D"https://httpd.apache.org/docs/2.0/mod/di= rective-dict.html#Module" style=3D"color:rgb(0,115,199);background-color:in= herit" target=3D"_blank" rel=3D"noreferrer">Module:</a></th><td style=3D"em= pty-cells:show;padding:0.1em 0.2em;vertical-align:top;line-height:1.3em">co= re</td></tr></tbody></table><p style=3D"line-height:1.3em;margin:0px 0px 1e= m;padding:0px;color:rgb(0,51,102);font-size:14px">The=C2=A0<code style=3D"f= ont-family:"Courier New",Courier,monospace;font-size:1em;color:rg= b(40,127,0);background-color:inherit">TimeOut</code>=C2=A0directive current= ly defines the amount of time Apache will wait for three things:</p><ol sty= le=3D"color:rgb(0,51,102);font-size:14px"><li style=3D"line-height:1.3em;ma= rgin-top:0.5em">The total amount of time it takes to receive a GET request.= </li><li style=3D"line-height:1.3em;margin-top:0.5em">The amount of time be= tween receipt of TCP packets on a POST or PUT request.</li><li style=3D"lin= e-height:1.3em;margin-top:0.5em">The amount of time between ACKs on transmi= ssions of TCP packets in responses.</li></ol><p style=3D"line-height:1.3em;= margin:0px 0px 1em;padding:0px;color:rgb(0,51,102);font-size:14px">We plan = on making these separately configurable at some point down the road. The ti= mer used to default to 1200 before 1.2, but has been lowered to 300 which i= s still far more than necessary in most situations. It is not set any lower= by default because there may still be odd places in the code where the tim= er is not reset when a packet is sent.</p></div></div></div><br><div class= =3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Apr 23, 2025= at 3:07=E2=80=AFPM Mithun Bhattacharya <<a href=3D"mailto:mithnb@gmail.= com" target=3D"_blank" rel=3D"noreferrer">[email protected]</a>> wrote:<b= r></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex= ;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"auto"= ><p dir=3D"ltr">You configure timeout at the client side. Apache is at the = server side. Server doesn't have a concept of time it could take days t= o run and not care.</p><p dir=3D"ltr">mod_perl code is where you are sendin= g the http return status to make sure the client doesn't timeout waitin= g for the server to respond.</p></div><br><div class=3D"gmail_quote"><div d= ir=3D"ltr" class=3D"gmail_attr">On Wed, Apr 23, 2025, 2:19=E2=80=AFPM Josep= h He <<a href=3D"mailto:[email protected]" target=3D"_blank" rel= =3D"noreferrer">[email protected]</a>> wrote:<br></div><blockquot= e class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px s= olid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr">Thanks, all.<div>I= s that Apache timeout controlled by its configuration=C2=A0"Timeout&qu= ot;?=C2=A0</div><div>I don't think it has anything to do with modPerl. = Am I=C2=A0missing something?</div><div>Thanks.</div></div><br><div class=3D= "gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Apr 23, 2025 at= 1:41=E2=80=AFPM Mithun Bhattacharya <<a href=3D"mailto:[email protected]= " rel=3D"noreferrer noreferrer" target=3D"_blank">[email protected]</a>> = wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0= px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir= =3D"auto">Timeout happens because of how we handle the request. Timeout is = basically no response came back. Why that happens is because we think we wa= nt to have a correct response. Unfortunately for long running requests the = correct response shouldn't be via http response code or we face situati= ons like this. Instead reply with a 200 OK immediately and then provide cor= rect status in the message body. Once a response code/header has been sent = timeout won't trigger and you could potentially hold the connection for= hours without a problem.</div><br><div class=3D"gmail_quote"><div dir=3D"l= tr" class=3D"gmail_attr">On Wed, Apr 23, 2025, 9:32=E2=80=AFAM Andreas Mock= <<a href=3D"mailto:[email protected]" rel=3D"noreferrer noreferrer" t= arget=3D"_blank">[email protected]</a>> wrote:<br></div><blockquote cl= ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid= rgb(204,204,204);padding-left:1ex">Hi Joseph,<br> <br> your description is very vague, so can only answer on some assumptions:<br> <br> It sounds like a timeout is fired somewhere.<br> <br> Best advice in these situations: Log as many steps as you can. Keep your <b= r> eyes open on TCP/IP and higher level timeouts.<br> <br> Declare only ONE instance responsible for a retry: Either the app server <b= r> calling the dispatcher with several tries or the dispatcher trying for <br> himself. Not both.<br> <br> Best regards<br> Andreas<br> <br> <br> Am 23.04.2025 um 16:21 schrieb Joseph He:<br> > All, good day.<br> ><br> > Here is the issue I have.<br> > My entire application is running on ModPerl/Apache environment.<br> > I send Http::Request with data load from my App server to a dispatch <= br> > server thru LWP::UserAgent, I set the timeout 600 seconds.<br> ><br> > The dispatch server is supposed to manipulate the data and send the <b= r> > data to an external SFTP server. Because the SFTP can fail, it will <b= r> > keep trying up to 4 times with 30 seconds sleep in case that SFTP <br> > connection fails.<br> ><br> > Recently, I found that I uploaded the file twice sometimes. I figured = <br> > out the root cause is that my Dispatch server returns 'failure'= ; at 6 <br> > minutes while it keeps trying to do the SFTP. The App server <br> > received=C2=A0HTTP::Response with error status so it issued another ca= ll to <br> > send data. It turns out I uploaded the identified file twice.<br> ><br> > Anybody has this sort of experience? Why does the dispatch=C2=A0server= <br> > return 'error' while it still processes the data?<br> ><br> > Thanks a lot,<br> > Joseph<br> ><br> </blockquote></div> </blockquote></div> </blockquote></div> </blockquote></div> </blockquote></div> --00000000000062e9b6063379775e--