Re: [jinterface] add erlang term parse/match/bind features
Dmitriy Kargapolov <[email protected]> Sun, 18 Jan 2015 12:18:47 -0500
| Newsgroups | gmane.comp.lang.erlang.patches |
|---|---|
| Message-ID | <CAAPzUBsEMMN5xLk-DcCjUEJshnFQ2zfTVDj4QZAXCvRJaR8C4g@mail.gmail.com> |
--===============8695663041786861643== Content-Type: multipart/alternative; boundary=089e0160b42012c124050cf066cc --089e0160b42012c124050cf066cc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi All, We couldn't find easy way to merge our implementations at this time. Instead I submitted PR #589: https://github.com/erlang/otp/pull/589, which comprises of very small addition to the JInterface library (match() and bind() generic methods). On top of this base change both approaches can be implemented (even with no more modifications to the core JInterface library). Thanks. On Thu, Jan 1, 2015 at 2:26 PM, Dmitriy Kargapolov < [email protected]> wrote: > Hi Vlad, > While continuing study your code I tried to answer your questions. > Reading this please keep in mind the difference in our approaches. In my > approach existing objects structure is re-used to keep pattern elements. > This was done for the sake of simplicity, to keep code less polluted > with new classes and interfaces. JInterface initially announced as > low-level interface to the Erlang, and I tried to follow this way. > > On 01/01/2015 09:12 AM, Vlad Dumitrescu wrote: > > Hi Dmitriy, > > > > I'm still going through your code, but I wonder if you could explain > > why equals should be implemented in terms of match? I think these are > > two different operations and can be relevant only if checking equals > > between a regular term and a pattern, but I can't see any reason to > > want to do that because these should never be equal. Or maybe I am > > missing a fine point?... > This was done for simplicity, to avoid duplicating the code, especially > when operating on complex data like lists, tuples, maps. Less code - > less errors. > Term considered to be pattern in opposite to regular object if it > contains variable placeholders. In such case it should not be used in > any way but for matching against other regular terms. (Regular term can > be considered as an edge case of pattern with no variables and therefore > can also be used as left side of the match operation). > In the light of above, calling equals method for the pattern(s) does not > make any sense and should not bother anyone. For regular term(s) match > and equals methods carry same semantics. > I know that equals method now has some runtime overheads. Imho this is > reasonable sacrificing. The equals() method is not much important. Java > code must preferably work with Java objects, not Erlang terms. What is > really important is to have effective way of importing Erlang terms > translating them to what is suitable for Java code. So much() > implementation is a priority over equals(). > > > > I am also wondering if OEObject itself could implement OEMatcher and > > just call match/bind recursively instead of checking "instanceof > > OEMatcher" in OEList, OEMap and OETuple. It feels easier to understand > > (less conditionals in the code). > In such case match/bin would be methods of all objects, including atoms, > integers, strings etc. This does not make much sense imho. I don=E2=80=99= t think > it is reasonable to replace one imperfection by another one. > > > > In my implementation, I did it in a slightly different way, mostly in > > order to modify existing classes as little as possible, by using an > > utility class to traverse terms and do matching and binding. > Our approaches are fundamentally different. I also tried to keep as much > code untouched as possible, but was reasonable lazy to re-create complex > Erlang terms structure in separate pattern object. My approach is a > compromise. It does require touching the core code. Your approach is > smart way to handle pattern matching without touching base code. It may > be (I guess) made as an extension to core JInterface in separate jar > having JInterface as a dependency. Frankly I didn=E2=80=99t have enough t= ime to > follow your way, my code contains less modifications and additions to > the JInterface and probably took less efforts to implement. > > > I also have more advanced conversion routines between Java and Erlang > > types (most useful are for Strings, List<->list and > > Array<->tuple/binary). Maybe you would like to check > > at > https://github.com/vladdu/otp/compare/erlang:master...jinterface_new_api? > I made for my project additional abstractions for type conversion > including lists of objects and maps. I=E2=80=99m not ready to share this,= since > it is not perfect at all. And I don=E2=80=99t see any comprehensive gener= ic way > of mapping Erlang data to/from Java data. Anyway, I think that such > stuff may be a separate library dedicated to type conversions and other > aspects of encode/decode of complex user=E2=80=99s data. JInterface as a > low-level library looks better when free of such additions. It is enough > to automate Erlang term structure matching moving type verification and > conversion to the next layer of abstraction or to user=E2=80=99s code (li= ke > custom binders in my case). > > > > > Maybe we should try to unify our code first and then submit it to OTP? > First of all, we (and community) should vote for the core approach, > answering the question: should we intervene into the base code or make > parallel patterns object structure? My opinion is that making parallel > structures is more error-prone and makes maintenance harder due to more > similar code to repeat in different places. > Second (less important though) question would be: how reasonable is to > add fancy data conversion helpers to the base JInterface library. > In the code I submitted there is may be space for improvements in the > OEParser class. If we'd like to bring basic data type > verification/conversions to the code. I would prefer to avoid this > though, at least for the initial version. > > > > Happy New Year and I guess that for you it's even Merry Christmas in a > > week, right? > Yes, thank you! > > > Vlad > > > > > > On Wed, Dec 31, 2014 at 5:31 PM, Dmitriy Kargapolov > > <[email protected] <mailto:[email protected]>> > > wrote: > > > > Hi Vlad, > > Thank you much for your notes. > > > > I agree some names were not good enough, I changed these per your > > suggestion. > > Old comment removed as well. > > > > As to OtpErangMap class - I was not going to re-implement it > > initially. But in order to add match/bind I had to do this. I > > tried to not add much new methods, keeping implementation rather > > "just enough" to work with maps. You are very welcome to > > add/change whatever you see reasonable in separate PR though. > > > > Best Regards and Happy New Year! > > - Dmitriy. > > > > git fetch https://github.com/x0id/otp.git > jinterface_pattern_matching > > > > > https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_ma= tching > > > https://github.com/x0id/otp/compare/erlang:master...jinterface_pattern_ma= tching.patch > > > > > > > > On 12/31/2014 10:27 AM, Vlad Dumitrescu wrote: > >> Hi Dmitriy, > >> > >> Nice implementation! I have something similar that I wanted to > >> submit, but I like some of your details better. > >> > >> I have a few comments, after browsing the code just briefly: > >> > >> - I would prefer to have the OEMap changes as a separate PR, as > >> it's a separate issue and there are more things to address there, > >> like for example, OEMap could also implement Map, like OEList > >> implements Iterable (and might implement List). > >> - I don't like the name OEVarrier, it doesn't mean anything. > >> Maybe OEMatcher would be better? > >> - OEBind might be clearer as OEBinding, as it contains a set of > >> bindings? > >> - I think there are still some merge issues, like for example > >> OEList:299 where the comment is from the old equals method > >> > >> best regards, > >> Vlad > >> > >> > >> On Tue, Dec 30, 2014 at 9:56 PM, Dmitriy Kargapolov > >> <[email protected] > >> <mailto:[email protected]>> wrote: > >> > >> This implements functionality similar to following C > >> functions, which are part of erl_interface application: > >> - ETERM *erl_format(FormatStr, ...); > >> - int erl_match(ETERM *Pattern, ETERM *Term); > >> > >> To acheve this new classes introduced: > >> * OtpErlangVar - variable placeholder; > >> * OtpErlangBind - variable values collection; > >> * OtpErlangParser - "erl_format" parser implementation; > >> * OtpErlangPattern - pattern abstraction with match/bind > >> functions; > >> > >> Classes representing composite objects OtpErlangList, > >> OtpErlangTuple, OtpErlangMap and new OtpErlangVar implement > >> interface OtpErlangVarrier defining match and bind functions > >> for these objects. > >> > >> Class OtpErlangMap reworked to be based on HashMap instead of > >> two separate lists keeping keys and values. This is close to > >> native semantics of maps and makes easier implementing basic > >> map manipulations. > >> > >> It addition to OtpErlangBind custom user's class may be used > >> as receiver of matched variables values. Java reflection is > >> used to prepare variable value setters during the parse > >> stage. Java doc has more details and examples. Test cases > >> implemented. > >> > >> git fetch https://github.com/x0id/otp.git > >> jinterface_pattern_matching > >> > >> Thanks. > >> > >> > >> > >> > >> > >> _______________________________________________ > >> erlang-patches mailing list > >> [email protected] <mailto:[email protected]> > >> http://erlang.org/mailman/listinfo/erlang-patches > >> > >> > > > > > > > --089e0160b42012c124050cf066cc Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Hi All,<div>We couldn't find easy way to merge our imp= lementations at this time. Instead I submitted PR #589: <a href=3D"https://= github.com/erlang/otp/pull/589">https://github.com/erlang/otp/pull/589</a>,= which comprises of very small addition to the JInterface library (match() = and bind() generic methods). On top of this base change both approaches can= be implemented (even with no more modifications to the core JInterface lib= rary).</div><div>Thanks.</div></div><div class=3D"gmail_extra"><br><div cla= ss=3D"gmail_quote">On Thu, Jan 1, 2015 at 2:26 PM, Dmitriy Kargapolov <span= dir=3D"ltr"><<a href=3D"mailto:[email protected]" target=3D"= _blank">[email protected]</a>></span> wrote:<br><blockquote c= lass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;= padding-left:1ex">Hi Vlad,<br> While continuing study your code I tried to answer your questions.<br> Reading this please keep in mind the difference in our approaches. In my<br= > approach existing objects structure is re-used to keep pattern elements.<br= > This was done for the sake of simplicity, to keep code less polluted<br> with new classes and interfaces. JInterface initially announced as<br> low-level interface to the Erlang, and I tried to follow this way.<br> <span class=3D""><br> On 01/01/2015 09:12 AM, Vlad Dumitrescu wrote:<br> > Hi Dmitriy,<br> ><br> </span><span class=3D"">> I'm still going through your code, but I w= onder if you could explain<br> > why equals should be implemented in terms of match? I think these are<= br> > two different operations and can be relevant only if checking equals<b= r> > between a regular term and a pattern, but I can't see any reason t= o<br> > want to do that because these should never be equal. Or maybe I am<br> > missing a fine point?...<br> </span>This was done for simplicity, to avoid duplicating the code, especia= lly<br> when operating on complex data like lists, tuples, maps. Less code -<br> less errors.<br> Term considered to be pattern in opposite to regular object if it<br> contains variable placeholders. In such case it should not be used in<br> any way but for matching against other regular terms. (Regular term can<br> be considered as an edge case of pattern with no variables and therefore<br= > can also be used as left side of the match operation).<br> In the light of above, calling equals method for the pattern(s) does not<br= > make any sense and should not bother anyone. For regular term(s) match<br> and equals methods carry same semantics.<br> I know that equals method now has some runtime overheads. Imho this is<br> reasonable sacrificing. The equals() method is not much important. Java<br> code must preferably work with Java objects, not Erlang terms. What is<br> really important is to have effective way of importing Erlang terms<br> translating them to what is suitable for Java code. So much()<br> implementation is a priority over equals().<br> <span class=3D"">><br> > I am also wondering if OEObject itself could implement OEMatcher and<b= r> > just call match/bind recursively instead of checking "instanceof<= br> > OEMatcher" in OEList, OEMap and OETuple. It feels easier to under= stand<br> > (less conditionals in the code).<br> </span>In such case match/bin would be methods of all objects, including at= oms,<br> integers, strings etc. This does not make much sense imho. I don=E2=80=99t = think<br> it is reasonable to replace one imperfection by another one.<br> <span class=3D"">><br> > In my implementation, I did it in a slightly different way, mostly in<= br> > order to modify existing classes as little as possible, by using an<br= > > utility class to traverse terms and do matching and binding.<br> </span>Our approaches are fundamentally different. I also tried to keep as = much<br> code untouched as possible, but was reasonable lazy to re-create complex<br= > Erlang terms structure in separate pattern object. My approach is a<br> compromise. It does require touching the core code. Your approach is<br> smart way to handle pattern matching without touching base code. It may<br> be (I guess) made as an extension to core JInterface in separate jar<br> having JInterface as a dependency. Frankly I didn=E2=80=99t have enough tim= e to<br> follow your way, my code contains less modifications and additions to<br> the JInterface and probably took less efforts to implement.<br> <span class=3D""><br> > I also have more advanced conversion routines between Java and Erlang<= br> > types (most useful are for Strings, List<->list and<br> > Array<->tuple/binary). Maybe you would like to check<br> > at <a href=3D"https://github.com/vladdu/otp/compare/erlang:master...ji= nterface_new_api" target=3D"_blank">https://github.com/vladdu/otp/compare/e= rlang:master...jinterface_new_api</a>?<br> </span>I made for my project additional abstractions for type conversion<br= > including lists of objects and maps. I=E2=80=99m not ready to share this, s= ince<br> it is not perfect at all. And I don=E2=80=99t see any comprehensive generic= way<br> of mapping Erlang data to/from Java data. Anyway, I think that such<br> stuff may be a separate library dedicated to type conversions and other<br> aspects of encode/decode of complex user=E2=80=99s data. JInterface as a<br= > low-level library looks better when free of such additions. It is enough<br= > to automate Erlang term structure matching moving type verification and<br> conversion to the next layer of abstraction or to user=E2=80=99s code (like= <br> custom binders in my case).<br> <span class=3D""><br> ><br> > Maybe we should try to unify our code first and then submit it to OTP?= <br> </span>First of all, we (and community) should vote for the core approach,<= br> answering the question: should we intervene into the base code or make<br> parallel patterns object structure? My opinion is that making parallel<br> structures is more error-prone and makes maintenance harder due to more<br> similar code to repeat in different places.<br> Second (less important though) question would be: how reasonable is to<br> add fancy data conversion helpers to the base JInterface library.<br> In the code I submitted there is may be space for improvements in the<br> OEParser class. If we'd like to bring basic data type<br> verification/conversions to the code. I would prefer to avoid this<br> though, at least for the initial version.<br> <span class=3D"">><br> > Happy New Year and I guess that for you it's even Merry Christmas = in a<br> > week, right?<br> </span>Yes, thank you!<br> <span class=3D""><br> > Vlad<br> ><br> ><br> > On Wed, Dec 31, 2014 at 5:31 PM, Dmitriy Kargapolov<br> </span>> <<a href=3D"mailto:[email protected]">dmitriy.kar= [email protected]</a> <mailto:<a href=3D"mailto:dmitriy.kargapolov@gmail= .com">[email protected]</a>>><br> <div><div class=3D"h5">> wrote:<br> ><br> >=C2=A0 =C2=A0 =C2=A0Hi Vlad,<br> >=C2=A0 =C2=A0 =C2=A0Thank you much for your notes.<br> ><br> >=C2=A0 =C2=A0 =C2=A0I agree some names were not good enough, I changed = these per your<br> >=C2=A0 =C2=A0 =C2=A0suggestion.<br> >=C2=A0 =C2=A0 =C2=A0Old comment removed as well.<br> ><br> >=C2=A0 =C2=A0 =C2=A0As to OtpErangMap class - I was not going to re-imp= lement it<br> >=C2=A0 =C2=A0 =C2=A0initially. But in order to add match/bind I had to = do this. I<br> >=C2=A0 =C2=A0 =C2=A0tried to not add much new methods, keeping implemen= tation rather<br> >=C2=A0 =C2=A0 =C2=A0"just enough" to work with maps. You are = very welcome to<br> >=C2=A0 =C2=A0 =C2=A0add/change whatever you see reasonable in separate = PR though.<br> ><br> >=C2=A0 =C2=A0 =C2=A0Best Regards and Happy New Year!<br> >=C2=A0 =C2=A0 =C2=A0- Dmitriy.<br> ><br> >=C2=A0 =C2=A0 =C2=A0git fetch <a href=3D"https://github.com/x0id/otp.gi= t" target=3D"_blank">https://github.com/x0id/otp.git</a> jinterface_pattern= _matching<br> ><br> >=C2=A0 =C2=A0 =C2=A0<a href=3D"https://github.com/x0id/otp/compare/erla= ng:master...jinterface_pattern_matching" target=3D"_blank">https://github.c= om/x0id/otp/compare/erlang:master...jinterface_pattern_matching</a><br> >=C2=A0 =C2=A0 =C2=A0<a href=3D"https://github.com/x0id/otp/compare/erla= ng:master...jinterface_pattern_matching.patch" target=3D"_blank">https://gi= thub.com/x0id/otp/compare/erlang:master...jinterface_pattern_matching.patch= </a><br> ><br> ><br> ><br> >=C2=A0 =C2=A0 =C2=A0On 12/31/2014 10:27 AM, Vlad Dumitrescu wrote:<br> >>=C2=A0 =C2=A0 =C2=A0Hi Dmitriy,<br> >><br> >>=C2=A0 =C2=A0 =C2=A0Nice implementation! I have something similar t= hat I wanted to<br> >>=C2=A0 =C2=A0 =C2=A0submit, but I like some of your details better.= <br> >><br> >>=C2=A0 =C2=A0 =C2=A0I have a few comments, after browsing the code = just briefly:<br> >><br> >>=C2=A0 =C2=A0 =C2=A0- I would prefer to have the OEMap changes as a= separate PR, as<br> >>=C2=A0 =C2=A0 =C2=A0it's a separate issue and there are more th= ings to address there,<br> >>=C2=A0 =C2=A0 =C2=A0like for example, OEMap could also implement Ma= p, like OEList<br> >>=C2=A0 =C2=A0 =C2=A0implements Iterable (and might implement List).= <br> >>=C2=A0 =C2=A0 =C2=A0- I don't like the name OEVarrier, it doesn= 't mean anything.<br> >>=C2=A0 =C2=A0 =C2=A0Maybe OEMatcher would be better?<br> >>=C2=A0 =C2=A0 =C2=A0- OEBind might be clearer as OEBinding, as it c= ontains a set of<br> >>=C2=A0 =C2=A0 =C2=A0bindings?<br> >>=C2=A0 =C2=A0 =C2=A0- I think there are still some merge issues, li= ke for example<br> >>=C2=A0 =C2=A0 =C2=A0OEList:299 where the comment is from the old eq= uals method<br> >><br> >>=C2=A0 =C2=A0 =C2=A0best regards,<br> >>=C2=A0 =C2=A0 =C2=A0Vlad<br> >><br> >><br> >>=C2=A0 =C2=A0 =C2=A0On Tue, Dec 30, 2014 at 9:56 PM, Dmitriy Kargap= olov<br> >>=C2=A0 =C2=A0 =C2=A0<<a href=3D"mailto:dmitriy.kargapolov@gmail.= com">[email protected]</a><br> </div></div><div><div class=3D"h5">>>=C2=A0 =C2=A0 =C2=A0<mailto:<= a href=3D"mailto:[email protected]">[email protected]= </a>>> wrote:<br> >><br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0This implements functionality sim= ilar to following C<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0functions, which are part of erl_= interface application:<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0- ETERM *erl_format(FormatStr, ..= .);<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0- int erl_match(ETERM *Pattern, E= TERM *Term);<br> >><br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0To acheve this new classes introd= uced:<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* OtpErlangVar - variable placeho= lder;<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* OtpErlangBind - variable values= collection;<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* OtpErlangParser - "erl_for= mat" parser implementation;<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* OtpErlangPattern - pattern abst= raction with match/bind<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0functions;<br> >><br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Classes representing composite ob= jects OtpErlangList,<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0OtpErlangTuple, OtpErlangMap and = new OtpErlangVar implement<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0interface OtpErlangVarrier defini= ng match and bind functions<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for these objects.<br> >><br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Class OtpErlangMap reworked to be= based on HashMap instead of<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0two separate lists keeping keys a= nd values. This is close to<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0native semantics of maps and make= s easier implementing basic<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0map manipulations.<br> >><br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0It addition to OtpErlangBind cust= om user's class may be used<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0as receiver of matched variables = values. Java reflection is<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0used to prepare variable value se= tters during the parse<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0stage. Java doc has more details = and examples. Test cases<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0implemented.<br> >><br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0git fetch <a href=3D"https://gith= ub.com/x0id/otp.git" target=3D"_blank">https://github.com/x0id/otp.git</a><= br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0jinterface_pattern_matching<br> >><br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Thanks.<br> >><br> >><br> >><br> >><br> >><br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0_________________________________= ______________<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0erlang-patches mailing list<br> </div></div>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"mailto:erl= [email protected]">[email protected]</a> <mailto:<a href=3D= "mailto:[email protected]">[email protected]</a>><br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"http://erlang.org/mail= man/listinfo/erlang-patches" target=3D"_blank">http://erlang.org/mailman/li= stinfo/erlang-patches</a><br> >><br> >><br> ><br> ><br> <br> <br> </blockquote></div><br></div> --089e0160b42012c124050cf066cc-- --===============8695663041786861643== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ erlang-patches mailing list [email protected] http://erlang.org/mailman/listinfo/erlang-patches --===============8695663041786861643==--