Re: is IO:Socket:INET thread safe?
[email protected] (Jesper Persson) Wed, 29 May 2013 11:06:56 +0200
| Newsgroups | perl.libwww |
|---|---|
| Message-ID | <CA+uC3V0+BoB_y8hacU1oKRohJfrfn=_nUO+3fS-YUa9Htd6nUw@mail.gmail.com> |
--089e013c6a9051788f04ddd7b512 Content-Type: text/plain; charset=ISO-8859-1 The program has been running for more than 10 years without this beeing an issue - at least I have never been aware of this kind of problem. Perhaps I only first see it now on the new server because the server is a virtual machine and maybe doesn't have a dedicate cpu and more often will get interrupts on the cpu. The reason that the program uses processes is that is has a mother process fetching work from the database and communicating the work/result of the work to/from the threads via Thread::Queue. Am I the first one using threads doing http requests in perl on linux having this problem? Seems odd. If I would change my program to using processes instead of threads, would gethostbyname then be process safe? It looks as if gethostbyname has a version that is reentrant: gethostbyname_r. Any reason that perl doesn't use that version? Regards On 29 May 2013 10:12, Francesco Nidito <[email protected]> wrote: > Unfortunately, gethostbyname is not reentrant. > > (maybe a stupid question) Is there any particular reason why you cannot > use multiple processes instead? > > Typed on a very small keyboard... > ------------------------------ > From: Jesper Persson <[email protected]> > Sent: 29/05/2013 07:17 > To: libwww <[email protected]> > Subject: is IO:Socket:INET thread safe? > > Hi everybody, > > I have a multithreaded program that monitors the content of web pages. > On a newly installed CentOS server I have seen strange behaviour where I > get 404 because http request is sent to a wrong ip address not maching the > domain that was requested. > > I have inserted a print statement in IO:Socket:INET.pm in the procedure > _get_addr > to see what ip address the domain is translated to: > > sub _get_addr { > my($sock,$addr_str, $multi) = @_; > my @addr; > if ($multi && $addr_str !~ /^\d+(?:\.\d+){3}$/) { > (undef, undef, undef, undef, @addr) = gethostbyname($addr_str); > } else { > my $h = inet_aton($addr_str); > push(@addr, $h) if defined $h; > } > open(FH, ">>/tmp/socket_get_addr.log"); > print FH localtime().": addr_str: $addr_str, addr: > ".inet_ntoa($addr[0])."\n"; > close(FH); > @addr; > } > > From the log file: > Wed May 29 07:38:09 2013: addr_str: dk1.siteimprove.com, addr: > 93.160.60.57 > Wed May 29 07:38:09 2013: addr_str: siteimprove.dk, addr: 93.160.60.57 > > the ip address of siteimprove.dk is wrong and at the same second just > before a lookup of dk1.siteimprove.com gets the same ip address and that > is correct. > > 7 minutes later in the log file the siteimprove.dk domain is looked up > correctly: > Wed May 29 07:45:11 2013: addr_str: siteimprove.dk, addr: 93.160.60.87 > > Manually looking up the ip on then server gets the right ip address. > > this entry is in the hosts file: > 93.160.60.87 siteimprove.dk > > CentOS release 6.3 (Final) > Summary of my perl5 (revision 5 version 10 subversion 1) configuration: > IO:Socket:INET version: $VERSION = "1.31"; > > Any help would be greatly appreciated. > > Regards > Jesper Persson > --089e013c6a9051788f04ddd7b512 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">The program has been running for more than 10 years withou= t this beeing an issue - at least I have never been aware of this kind of p= roblem.<div style>Perhaps I only first see it now on the new server because= the server is a virtual machine and maybe doesn't have a dedicate cpu = and more often will get interrupts on the cpu.</div> <div style><br></div><div style>The reason that the program uses processes = is that is has a mother process fetching work from the database and communi= cating the work/result of the work to/from the threads via=A0Thread::Queue.= </div> <div style><br></div><div style>Am I the first one using threads doing http= requests in perl on linux having this problem? Seems odd.<br></div><div st= yle><br></div><div style>If I would change my program to using processes in= stead of threads, would gethostbyname then be process safe?</div> <div style><br></div><div style>It looks as if gethostbyname has a version = that is reentrant: gethostbyname_r.=A0</div><div style>Any reason that perl= doesn't use that version?</div><div style><br></div><div style>Regards= </div> </div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On 29 M= ay 2013 10:12, Francesco Nidito <span dir=3D"ltr"><<a href=3D"mailto:fra= [email protected]" target=3D"_blank">[email protected]</a>&g= t;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> <div> <div> <div style=3D"FONT-SIZE:11pt;FONT-FAMILY:Calibri,sans-serif">Unfortunately,= gethostbyname is not reentrant. <br><br>(maybe a stupid question) Is there= any particular reason why you cannot use multiple processes instead?<br> <br>Typed on a very small keyboard...</div></div> <div dir=3D"ltr"> <hr> <span style=3D"FONT-SIZE:11pt;FONT-FAMILY:Calibri,sans-serif;FONT-WEIGHT:bo= ld">From: </span><span style=3D"FONT-SIZE:11pt;FONT-FAMILY:Calibri,sans-ser= if"><a href=3D"mailto:[email protected]" target=3D"_blank">Jesper = Persson</a></span><br> <span style=3D"FONT-SIZE:11pt;FONT-FAMILY:Calibri,sans-serif;FONT-WEIGHT:bo= ld">Sent: </span><span style=3D"FONT-SIZE:11pt;FONT-FAMILY:Calibri,sans-ser= if">29/05/2013 07:17</span><br><span style=3D"FONT-SIZE:11pt;FONT-FAMILY:Ca= libri,sans-serif;FONT-WEIGHT:bold">To: </span><span style=3D"FONT-SIZE:11pt= ;FONT-FAMILY:Calibri,sans-serif"><a href=3D"mailto:[email protected]" target= =3D"_blank">libwww</a></span><br> <span style=3D"FONT-SIZE:11pt;FONT-FAMILY:Calibri,sans-serif;FONT-WEIGHT:bo= ld">Subject: </span><span style=3D"FONT-SIZE:11pt;FONT-FAMILY:Calibri,sans-= serif">is IO:Socket:INET thread safe?</span><br><br></div></div><div class= =3D"HOEnZb"> <div class=3D"h5"><div dir=3D"ltr"> Hi everybody,<div><br></div><div>I have a multithreaded program that monito= rs the content of web pages.</div><div>On a newly installed CentOS server I= have seen strange behaviour where I get 404 because http request is sent t= o a wrong ip address not maching the domain that was requested.</div> <div><br></div><div>I have inserted a print statement in IO:Socket:INET.pm = in the procedure _get_addr</div><div><div>to see what ip address the domain= is translated to:</div><div><br></div><div> sub _get_addr {</div><div>=A0 =A0 my($sock,$addr_str, $multi) =3D @_;</div>= <div>=A0 =A0 my @addr;</div><div>=A0 =A0 if ($multi && $addr_str !~= /^\d+(?:\.\d+){3}$/) {</div><div>=A0 =A0 =A0 =A0 (undef, undef, undef, und= ef, @addr) =3D gethostbyname($addr_str);</div> <div>=A0 =A0 } else {</div><div>=A0 =A0 =A0 =A0 my $h =3D inet_aton($addr_s= tr);</div><div>=A0 =A0 =A0 =A0 push(@addr, $h) if defined $h;</div><div>=A0= =A0 }</div><div>=A0 =A0 open(FH, ">>/tmp/socket_get_addr.log&qu= ot;);</div><div>=A0 =A0 print FH localtime().": addr_str: $addr_str, a= ddr: ".inet_ntoa($addr[0])."\n";</div> <div>=A0 =A0 close(FH);</div><div>=A0 =A0 @addr;</div><div>}</div><div><br>= </div><div>From the log file:</div><div><div>Wed May 29 07:38:09 2013: addr= _str: <a href=3D"http://dk1.siteimprove.com" target=3D"_blank">dk1.siteimpr= ove.com</a>, addr: 93.160.60.57</div> <div>Wed May 29 07:38:09 2013: addr_str: <a href=3D"http://siteimprove.dk" = target=3D"_blank">siteimprove.dk</a>, addr: 93.160.60.57</div><div><br></di= v><div>the ip address of <a href=3D"http://siteimprove.dk" target=3D"_blank= ">siteimprove.dk</a> is wrong and at the same second just before a lookup o= f <a href=3D"http://dk1.siteimprove.com" target=3D"_blank">dk1.siteimprove.= com</a> gets the same ip address and that is correct.</div> <div><br></div><div>7 minutes later in the log file the <a href=3D"http://s= iteimprove.dk" target=3D"_blank">siteimprove.dk</a> domain is looked up cor= rectly:</div><div>Wed May 29 07:45:11 2013: addr_str: <a href=3D"http://sit= eimprove.dk" target=3D"_blank">siteimprove.dk</a>, addr: 93.160.60.87<br> </div><div><br></div><div>Manually looking up the ip on then server gets th= e right ip address.</div><div><br></div><div>this entry is in the hosts fil= e:=A0</div><div>93.160.60.87 =A0 =A0 =A0 =A0<a href=3D"http://siteimprove.d= k" target=3D"_blank">siteimprove.dk</a></div> <div><br></div><div>CentOS release 6.3 (Final)<br></div><div>Summary of my = perl5 (revision 5 version 10 subversion 1) configuration:<br></div><div>IO:= Socket:INET version:=A0$VERSION =3D "1.31";</div> <div><br></div><div>Any help would be greatly appreciated.</div><div><br></= div><div>Regards</div><div>Jesper Persson</div></div></div></div> </div></div></blockquote></div><br></div> --089e013c6a9051788f04ddd7b512--