Re: ParallelModuleQueue freezes...
Anna Petrášová <[email protected]> Fri, 28 May 2021 10:36:14 -0400
| Newsgroups | gmane.comp.gis.grass.devel |
|---|---|
| Message-ID | <CAE0EDEoqn4ws1HNYv1BwJCQhCai819pJLpO7v1q4QML1KGZXoA@mail.gmail.com> |
--===============0044739396419985095== Content-Type: multipart/alternative; boundary="000000000000f0156a05c364cc65" --000000000000f0156a05c364cc65 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, May 28, 2021 at 10:02 AM Stefan Blumentrath < [email protected]> wrote: > Some more context. > The t.rast.neighbors test seems to work with the current dev version > "2048e1016" from 2021-05-19T09:38:49+00:00 > It fails however, with 7.8.5 from 2020-12-22T15:55:09+00:00 and an older > development version "35edbf7dc" > from 2020-06-03T16:03:52+00:00 > > So, maybe one or more of the recent changes need backporting? > > https://github.com/OSGeo/grass/commits/master/python/grass/pygrass/module= s/interface/module.py This one: https://github.com/OSGeo/grass/pull/1407 The reason I didn't backport it was there has been a small change in API to make it work. But Python3 support is probably a good reason to backport it. > > > In the 7_8 release branch, the latest change is from May 2019: > > https://github.com/OSGeo/grass/commits/releasebranch_7_8/lib/python/pygra= ss/modules/interface/module.py > > Hope that helps identifying a solution? > > Cheers > Stefan > > -----Original Message----- > From: Stefan Blumentrath <[email protected]> > Sent: torsdag 27. mai 2021 23:09 > To: Stefan Blumentrath <[email protected]>; Luca Delucchi < > [email protected]>; Anna Petr=C3=A1=C5=A1ov=C3=A1 <[email protected]= om> > Cc: GRASS developers list <[email protected]> > Subject: RE: [GRASS-dev] ParallelModuleQueue freezes... > > See: > > https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithu= b.com%2FOSGeo%2Fgrass%2Fpull%2F1600&data=3D04%7C01%7C%7Cc5fe909715a6474= 31c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653= 048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6= Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3D5yfxG5gjlYlzgJn6yTP9zbUk7GE1JdcYP= xWIQFwUK%2Fo%3D&reserved=3D0 > Please test. > Unittests for t.rast.neighbors should work again with this small change..= . > > -----Original Message----- > From: grass-dev <[email protected]> On Behalf Of Stefan > Blumentrath > Sent: torsdag 27. mai 2021 08:19 > To: Luca Delucchi <[email protected]>; Anna Petr=C3=A1=C5=A1ov=C3=A1 < > [email protected]> > Cc: GRASS developers list <[email protected]> > Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... > > The combination ParallelModuleQueue and MultiModule is used in > t.rast.neighbors: > https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithu= b.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ft.ras= t.neighbors.py&data=3D04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6= cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTW= FpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3= D%7C1000&sdata=3D5QyN%2FR1cXgoWWGS2lgjKi8%2BeLX77w37bfMQzcP6O4bM%3D&= ;reserved=3D0 > > The testsuite of that module is deactivated for Python > 2. > > https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithu= b.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ftests= uite%2Ftest_neighbors.py&data=3D04%7C01%7C%7Cc5fe909715a647431c9308d921= 53b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUn= known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ= XVCI6Mn0%3D%7C1000&sdata=3DnQTdDIGmefI6F37zdjhyMDBYUrIxVaC%2FiA5FzeUK2K= U%3D&reserved=3D0 > > If you activate it by commenting out line 15. You will see the behavior..= . > > Thanks for looking into this! > > Cheers > Stefan > > > -----Original Message----- > From: Luca Delucchi <[email protected]> > Sent: torsdag 27. mai 2021 06:07 > To: Anna Petr=C3=A1=C5=A1ov=C3=A1 <[email protected]> > Cc: Stefan Blumentrath <[email protected]>; GRASS developers > list <[email protected]> > Subject: Re: [GRASS-dev] ParallelModuleQueue freezes... > > On Thu, 27 May 2021 at 04:07, Anna Petr=C3=A1=C5=A1ov=C3=A1 <kratochanna@= gmail.com> > wrote: > > > > Could you provide an example that is failing? The doctests are running > for me. I spent some time fixing it a couple months ago. I use Python 3.6= , > I wonder if newer Python would cause some issues. > > > > > https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithu= b.com%2FOSGeo%2Fgrass-addons%2Fpull%2F523%2Ffiles&data=3D04%7C01%7C%7Cc= 5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%= 7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2= luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DKT5h55J89LLXhK7uEuj= AmIwfksjPGyXEy2u7gYIc2Ns%3D&reserved=3D0 > line 544, the commented lines are not working for me.. > If you want to try t.vi just comment line 543 and uncomment the following > lines.. > > > Anna > > > > -- > ciao > Luca > > > https://eur01.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%2Fwww.lu= cadelu.org%2F&data=3D04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6c= ef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWF= pbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D= %7C1000&sdata=3DhM8CpSgjw7SFjGKxM9SFfWI3G8U3nr8eeCqedZORdso%3D&rese= rved=3D0 > _______________________________________________ > grass-dev mailing list > [email protected] > > https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists= .osgeo.org%2Fmailman%2Flistinfo%2Fgrass-dev&data=3D04%7C01%7C%7Cc5fe909= 715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C6375= 77465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIi= LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DIwP5QxiLzELTGrr393lqAWrlW= RhBVg%2FbOhrz9lBHMkw%3D&reserved=3D0 > --000000000000f0156a05c364cc65 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Fri, May 28, 2021 at 10:02 AM Stef= an Blumentrath <<a href=3D"mailto:[email protected]">Stefan.Blu= [email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_quote" = style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pa= dding-left:1ex">Some more context.<br> The t.rast.neighbors test seems to work with the current dev version<br> "2048e1016" from 2021-05-19T09:38:49+00:00<br> It fails however, with 7.8.5 from 2020-12-22T15:55:09+00:00 and an older de= velopment version "35edbf7dc"<br> from 2020-06-03T16:03:52+00:00<br> <br> So, maybe one or more of the recent changes need backporting?<br> <a href=3D"https://github.com/OSGeo/grass/commits/master/python/grass/pygra= ss/modules/interface/module.py" rel=3D"noreferrer" target=3D"_blank">https:= //github.com/OSGeo/grass/commits/master/python/grass/pygrass/modules/interf= ace/module.py</a></blockquote><div><br></div><div>This one:</div><div><a hr= ef=3D"https://github.com/OSGeo/grass/pull/1407">https://github.com/OSGeo/gr= ass/pull/1407</a></div><div><br></div><div>The reason I didn't=C2=A0bac= kport it was there has been a small change in API to=C2=A0make it work. But= Python3 support is probably a good reason to backport it.</div><div>=C2=A0= </div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;b= order-left:1px solid rgb(204,204,204);padding-left:1ex"><br> <br> In the 7_8 release branch, the latest change is from May 2019:<br> <a href=3D"https://github.com/OSGeo/grass/commits/releasebranch_7_8/lib/pyt= hon/pygrass/modules/interface/module.py" rel=3D"noreferrer" target=3D"_blan= k">https://github.com/OSGeo/grass/commits/releasebranch_7_8/lib/python/pygr= ass/modules/interface/module.py</a><br> <br> Hope that helps identifying a solution?<br> <br> Cheers<br> Stefan<br> <br> -----Original Message-----<br> From: Stefan Blumentrath <<a href=3D"mailto:[email protected]" = target=3D"_blank">[email protected]</a>> <br> Sent: torsdag 27. mai 2021 23:09<br> To: Stefan Blumentrath <<a href=3D"mailto:[email protected]" ta= rget=3D"_blank">[email protected]</a>>; Luca Delucchi <<a hr= ef=3D"mailto:[email protected]" target=3D"_blank">[email protected]</= a>>; Anna Petr=C3=A1=C5=A1ov=C3=A1 <<a href=3D"mailto:kratochanna@gma= il.com" target=3D"_blank">[email protected]</a>><br> Cc: GRASS developers list <<a href=3D"mailto:[email protected]" = target=3D"_blank">[email protected]</a>><br> Subject: RE: [GRASS-dev] ParallelModuleQueue freezes...<br> <br> See:<br> <a href=3D"https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2= F%2Fgithub.com%2FOSGeo%2Fgrass%2Fpull%2F1600&amp;data=3D04%7C01%7C%7Cc5= fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7= C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2l= uMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=3D5yfxG5gjlYlzgJn6= yTP9zbUk7GE1JdcYPxWIQFwUK%2Fo%3D&amp;reserved=3D0" rel=3D"noreferrer" t= arget=3D"_blank">https://eur01.safelinks.protection.outlook.com/?url=3Dhttp= s%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fpull%2F1600&amp;data=3D04%7C01%7= C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0= %7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQI= joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=3D5yfxG5gjlY= lzgJn6yTP9zbUk7GE1JdcYPxWIQFwUK%2Fo%3D&amp;reserved=3D0</a><br> Please test.<br> Unittests for t.rast.neighbors should work again with this small change...<= br> <br> -----Original Message-----<br> From: grass-dev <<a href=3D"mailto:[email protected]" ta= rget=3D"_blank">[email protected]</a>> On Behalf Of Stef= an Blumentrath<br> Sent: torsdag 27. mai 2021 08:19<br> To: Luca Delucchi <<a href=3D"mailto:[email protected]" target=3D"_bl= ank">[email protected]</a>>; Anna Petr=C3=A1=C5=A1ov=C3=A1 <<a hre= f=3D"mailto:[email protected]" target=3D"_blank">[email protected]<= /a>><br> Cc: GRASS developers list <<a href=3D"mailto:[email protected]" = target=3D"_blank">[email protected]</a>><br> Subject: Re: [GRASS-dev] ParallelModuleQueue freezes...<br> <br> The combination ParallelModuleQueue and MultiModule is used in t.rast.neigh= bors: <a href=3D"https://eur01.safelinks.protection.outlook.com/?url=3Dhttp= s%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.ne= ighbors%2Ft.rast.neighbors.py&amp;data=3D04%7C01%7C%7Cc5fe909715a647431= c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C63757746565304= 8701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik= 1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=3D5QyN%2FR1cXgoWWGS2lgjKi8%2BeLX7= 7w37bfMQzcP6O4bM%3D&amp;reserved=3D0" rel=3D"noreferrer" target=3D"_bla= nk">https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgit= hub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ft.r= ast.neighbors.py&amp;data=3D04%7C01%7C%7Cc5fe909715a647431c9308d92153b4= 45%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknow= n%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI= 6Mn0%3D%7C1000&amp;sdata=3D5QyN%2FR1cXgoWWGS2lgjKi8%2BeLX77w37bfMQzcP6O= 4bM%3D&amp;reserved=3D0</a><br> <br> The testsuite of that module is deactivated for Python > 2.<br> <a href=3D"https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2= F%2Fgithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbor= s%2Ftestsuite%2Ftest_neighbors.py&amp;data=3D04%7C01%7C%7Cc5fe909715a64= 7431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C6375774656= 53048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi= I6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=3DnQTdDIGmefI6F37zdjhyMDBYUrI= xVaC%2FiA5FzeUK2KU%3D&amp;reserved=3D0" rel=3D"noreferrer" target=3D"_b= lank">https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fg= ithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ft= estsuite%2Ftest_neighbors.py&amp;data=3D04%7C01%7C%7Cc5fe909715a647431c= 9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048= 701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1= haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=3DnQTdDIGmefI6F37zdjhyMDBYUrIxVaC%= 2FiA5FzeUK2KU%3D&amp;reserved=3D0</a><br> <br> If you activate it by commenting out line 15. You will see the behavior...<= br> <br> Thanks for looking into this!<br> <br> Cheers<br> Stefan<br> <br> <br> -----Original Message-----<br> From: Luca Delucchi <<a href=3D"mailto:[email protected]" target=3D"_= blank">[email protected]</a>> <br> Sent: torsdag 27. mai 2021 06:07<br> To: Anna Petr=C3=A1=C5=A1ov=C3=A1 <<a href=3D"mailto:[email protected]= om" target=3D"_blank">[email protected]</a>><br> Cc: Stefan Blumentrath <<a href=3D"mailto:[email protected]" ta= rget=3D"_blank">[email protected]</a>>; GRASS developers list &= lt;<a href=3D"mailto:[email protected]" target=3D"_blank">grass-dev= @lists.osgeo.org</a>><br> Subject: Re: [GRASS-dev] ParallelModuleQueue freezes...<br> <br> On Thu, 27 May 2021 at 04:07, Anna Petr=C3=A1=C5=A1ov=C3=A1 <<a href=3D"= mailto:[email protected]" target=3D"_blank">[email protected]</a>&g= t; wrote:<br> ><br> > Could you provide an example that is failing? The doctests are running= for me. I spent some time fixing it a couple months ago. I use Python 3.6,= I wonder if newer Python would cause some issues.<br> ><br> <br> <a href=3D"https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2= F%2Fgithub.com%2FOSGeo%2Fgrass-addons%2Fpull%2F523%2Ffiles&amp;data=3D0= 4%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf0= 2c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM= DAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=3DKT= 5h55J89LLXhK7uEujAmIwfksjPGyXEy2u7gYIc2Ns%3D&amp;reserved=3D0" rel=3D"n= oreferrer" target=3D"_blank">https://eur01.safelinks.protection.outlook.com= /?url=3Dhttps%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass-addons%2Fpull%2F523%2Ffile= s&amp;data=3D04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021= 314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d= 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&= amp;amp;sdata=3DKT5h55J89LLXhK7uEujAmIwfksjPGyXEy2u7gYIc2Ns%3D&amp;rese= rved=3D0</a> line 544, the commented lines are not working for me..<br> If you want to try <a href=3D"http://t.vi" rel=3D"noreferrer" target=3D"_bl= ank">t.vi</a> just comment line 543 and uncomment the following lines..<br> <br> > Anna<br> ><br> <br> --<br> ciao<br> Luca<br> <br> <a href=3D"https://eur01.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F= %2Fwww.lucadelu.org%2F&amp;data=3D04%7C01%7C%7Cc5fe909715a647431c9308d9= 2153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7C= Unknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL= CJXVCI6Mn0%3D%7C1000&amp;sdata=3DhM8CpSgjw7SFjGKxM9SFfWI3G8U3nr8eeCqedZ= ORdso%3D&amp;reserved=3D0" rel=3D"noreferrer" target=3D"_blank">https:/= /eur01.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%2Fwww.lucadelu.or= g%2F&amp;data=3D04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373= 021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZs= b3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C10= 00&amp;sdata=3DhM8CpSgjw7SFjGKxM9SFfWI3G8U3nr8eeCqedZORdso%3D&amp;r= eserved=3D0</a><br> _______________________________________________<br> grass-dev mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">grass-dev@li= sts.osgeo.org</a><br> <a href=3D"https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2= F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgrass-dev&amp;data=3D04%7C01= %7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7= C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJ= QIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=3DIwP5QxiL= zELTGrr393lqAWrlWRhBVg%2FbOhrz9lBHMkw%3D&amp;reserved=3D0" rel=3D"noref= errer" target=3D"_blank">https://eur01.safelinks.protection.outlook.com/?ur= l=3Dhttps%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgrass-dev&amp;= data=3D04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef3730213149018310= 55b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM= C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sd= ata=3DIwP5QxiLzELTGrr393lqAWrlWRhBVg%2FbOhrz9lBHMkw%3D&amp;reserved=3D0= </a><br> </blockquote></div></div> --000000000000f0156a05c364cc65-- --===============0044739396419985095== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ grass-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/grass-dev --===============0044739396419985095==--