Re: Gelex with Unicode
"Eric Bezault ericb-D6Qt/9opevxWk0Htik3J/[email protected] [gobo-eiffel]" <[email protected]> Tue, 7 Jul 2020 10:27:36 +0200
| Newsgroups | gmane.comp.lang.eiffel.gobo.general |
|---|---|
| Message-ID | <[email protected]> |
--wQxlCi9O7V1TCpTfFOpUEAv3HsNJvaep0ZYBHib Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit On 06/07/2020 21:22, Alexander Kogtenkov kwaxer-JGs/[email protected] [gobo-eiffel] wrote: > > > Yes, reading only a small portion of a file is one use case. Another one > is to see in which cases the scanner goes backwards (you are completely > right: I forgot about the scenario when some data is unread), how > frequently this happens and whether the scanner unreads only what it > read before or something else. I understand that in general all the > functionality is used. But for specific scanner rules and coding style, > only a subset might be required. > Clearly, this is an area for experiments and benchmarks, but if there is > some guidance about conditions that trigger use of particular buffer > features, it would be great to have a look. I cannot tell on top of my head all cases, but in these two cases (with / in the pattern, and using `less`) the scanner goes backwards: https://github.com/EiffelSoftware/EiffelStudio/blob/master/Src/framework/parser/parser/eiffel.l#L71 https://github.com/EiffelSoftware/EiffelStudio/blob/master/Src/framework/parser/parser/eiffel.l#L96 There is a mode where the buffer is filled in one character at a time. It's in interactive mode. See third paragraph of: http://www.gobosoft.com/eiffel/gobo/gelex/scanner.html But it requires another scanner skeleton class, and apparently the performance is not that great. I don't say that this interactive mode is exactly what you suggested. But what I want to point out is that unless we really have a case where we have a performance issue when using the new Unicode implementation compared to what we currently have when writing UTF-8 sequences in the patterns, and we cannot find a workaround (like using the smaller buffer when just parsing the head of the class text to discover the name of the class), then it's probably not worth the effort to try to improve things since there is a good chance that in the end it will not improve anything. For what the Gobo compiler (compiled with itself) and other tools like `gelint` are concerned, I did not notice any performance issue when switching to the new Unicode implementation (even when parsing 17,000 classes). > (Side note. I do not remember if I received a confirmation that the > private message with the tables has reached you. If not, I can resend > the message from some other mail account in case the previous one was > considered spam by automatic filters.) My answer was not well formatted, that's probably why you did not see it. Here it is again: On 30/06/2020 10:46, Eric Bezault ericb-D6Qt/9opevxWk0Htik3J/[email protected] [gobo-eiffel] wrote: > On 30/06/2020 10:07, Alexander Kogtenkovkwaxer-JGs/[email protected] [gobo-eiffel] > wrote: > > Yes, I understand most of the points you mention, and thank > > you for the additional clarification in the two recent messages. The > > difficult part is to keep everything in mind to build the complete > > picture. That’s why I mentioned it would be great to have some > > tables that summarize all the details. I’ve prepared such tables > > myself and I’m attaching them to the email. (In case they do not > > pass through because of the mailing list filters, I’ll send the > > tables in private.) I’d like to keep them for the future, so let > > me know, please, if you would like to include them in the > > documentation, or if there is some other place to put them. > > I will have to study your table in details to validate whether it > is accurate or not, but yes it's a good idea to add it to the > documentation. -- Eric Bezault mailto:ericb-D6Qt/9opevxWk0Htik3J/[email protected] http://www.gobosoft.com --wQxlCi9O7V1TCpTfFOpUEAv3HsNJvaep0ZYBHib Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/htm= l4/strict.dtd"> <html> <head> </head> =20 <body style=3D"background-color: #fff;"> <span style=3D"display:none"> </span> <!--~-|**|PrettyHtmlStartT|**|-~--> <div id=3D"ygrp-mlmsg" style=3D"position:relative;"> <div id=3D"ygrp-msg" style=3D"z-index: 1;"> <!--~-|**|PrettyHtmlEndT|**|-~--> <div id=3D"ygrp-text" > =20=20=20=20=20=20 =20=20=20=20=20=20 <p>On 06/07/2020 21:22, Alexander Kogtenkov kwaxer-JGs/[email protected] [gobo-eiff= el] wrote:<br> > <br> > <br> > Yes, reading only a small portion of a file is one use case. Another o= ne <br> > is to see in which cases the scanner goes backwards (you are completel= y <br> > right: I forgot about the scenario when some data is unread), how <br> > frequently this happens and whether the scanner unreads only what it <= br> > read before or something else. I understand that in general all the <b= r> > functionality is used. But for specific scanner rules and coding style= , <br> > only a subset might be required.<br> > Clearly, this is an area for experiments and benchmarks, but if there = is <br> > some guidance about conditions that trigger use of particular buffer <= br> > features, it would be great to have a look.<br> <br> I cannot tell on top of my head all cases, but in these two cases<br> (with / in the pattern, and using `less`) the scanner goes backwards:<br> <br> https://github.com/EiffelSoftware/EiffelStudio/blob/master/Src/framework/pa= rser/parser/eiffel.l#L71<br> https://github.com/EiffelSoftware/EiffelStudio/blob/master/Src/framework/pa= rser/parser/eiffel.l#L96<br> <br> There is a mode where the buffer is filled in one character at a time.<br> It's in interactive mode. See third paragraph of:<br> <br> http://www.gobosoft.com/eiffel/gobo/gelex/scanner.html<br> <br> But it requires another scanner skeleton class, and apparently the<br> performance is not that great. I don't say that this interactive mode<b= r> is exactly what you suggested. But what I want to point out is that<br> unless we really have a case where we have a performance issue when<br> using the new Unicode implementation compared to what we currently<br> have when writing UTF-8 sequences in the patterns, and we cannot<br> find a workaround (like using the smaller buffer when just parsing<br> the head of the class text to discover the name of the class), then<br> it's probably not worth the effort to try to improve things since<br> there is a good chance that in the end it will not improve anything.<br> For what the Gobo compiler (compiled with itself) and other tools like<br> `gelint` are concerned, I did not notice any performance issue when<br> switching to the new Unicode implementation (even when parsing 17,000<br> classes).<br> <br> > (Side note. I do not remember if I received a confirmation that the <b= r> > private message with the tables has reached you. If not, I can resend = <br> > the message from some other mail account in case the previous one was = <br> > considered spam by automatic filters.)<br> <br> My answer was not well formatted, that's probably why you did not see<b= r> it. Here it is again:<br> <br> On 30/06/2020 10:46, Eric Bezault ericb-D6Qt/9opevxWk0Htik3J/[email protected] [gobo-eiffel] wrote:<b= r> > On 30/06/2020 10:07, Alexander Kogtenkovkwaxer-JGs/[email protected] [gobo-eiffel]= <br> > wrote:<br> > > Yes, I understand most of the points you mention, and thank<br> > > you for the additional clarification in the two recent messages.= The<br> > > difficult part is to keep everything in mind to build the comple= te<br> > > picture. That=E2=80=99s why I mentioned it would be great to hav= e some<br> > > tables that summarize all the details. I=E2=80=99ve prepared suc= h tables<br> > > myself and I=E2=80=99m attaching them to the email. (In case the= y do not<br> > > pass through because of the mailing list filters, I=E2=80=99ll s= end the<br> > > tables in private.) I=E2=80=99d like to keep them for the future= , so let<br> > > me know, please, if you would like to include them in the<br> > > documentation, or if there is some other place to put them.<br> ><br> > I will have to study your table in details to validate whether it<br> > is accurate or not, but yes it's a good idea to add it to the<br> > documentation.<br> <br> -- <br> Eric Bezault<br> mailto:ericb-D6Qt/9opevxWk0Htik3J/[email protected]<br> http://www.gobosoft.com<br> </p> </div> =20=20=20=20=20 <!--~-|**|PrettyHtmlStart|**|-~--> <div style=3D"color: #fff; height: 0;">__._,_.___</div> =20=20=20=20=20=20=20=20=20=20 =20=20 =20 =20=20=20=20 <div style=3D"clear:both"> </div> <div id=3D"fromDMARC" style=3D"margin-top: 10px;"> <hr style=3D"height:2px ; border-width:0; color:#E3E3E3; background= -color:#E3E3E3;"> Posted by: Eric Bezault <ericb-D6Qt/9opevxWk0Htik3J/[email protected]> <hr style= =3D"height:2px ; border-width:0; color:#E3E3E3; background-color:#E3E3E3;"> </div> <div style=3D"clear:both"> </div> <table cellspacing=3D4px style=3D"margin-top: 10px; margin-bottom: 10px= ; color: #2D50FD;"> <tbody> <tr> <td style=3D"font-size: 12px; font-family: arial; font-weight: bo= ld; padding: 7px 5px 5px;" > <a style=3D"text-decoration: none; color: #2D50FD= " href=3D"https://groups.yahoo.com/neo/groups/gobo-eiffel/conversations/mes= sages/1931;_ylc=3DX3oDMTJwZ3JyM2FzBF9TAzk3MzU5NzE0BGdycElkAzE1MjIyNzAEZ3Jwc= 3BJZAMxNzA1MDA2NzY0BG1zZ0lkAzE5MzEEc2VjA2Z0cgRzbGsDcnBseQRzdGltZQMxNTk0MTEw= NDYz?act=3Dreply&messageNum=3D1931">Reply via web post</a> </td> <td>•</td> <td style=3D"font-size: 12px; font-family: arial; padding: 7px 5p= x 5px;" > <a href=3D"mailto:ericb-D6Qt/9opevxWk0Htik3J/[email protected]?subject=3DRe%3A%20%5Bgobo-= eiffel%5D%20Gelex%20with%20Unicode" style=3D"text-decoration: none; color: = #2D50FD;"> Reply to sender </a> </td> <td>•</td> <td style=3D"font-size: 12px; font-family: arial; padding: 7px 5p= x 5px;"> <a href=3D"mailto:[email protected]?subject=3DRe%3A%2= 0%5Bgobo-eiffel%5D%20Gelex%20with%20Unicode" style=3D"text-decoration: none= ; color: #2D50FD"> Reply to group </a> </td> <td>•</td> <td style=3D"font-size: 12px; font-family: arial; padding: 7px 5p= x 5px;" > <a href=3D"https://groups.yahoo.com/neo/groups/gobo-eiffel/conv= ersations/newtopic;_ylc=3DX3oDMTJlNzA2MjQzBF9TAzk3MzU5NzE0BGdycElkAzE1MjIyN= zAEZ3Jwc3BJZAMxNzA1MDA2NzY0BHNlYwNmdHIEc2xrA250cGMEc3RpbWUDMTU5NDExMDQ2Mw--= " style=3D"text-decoration: none; color: #2D50FD">Start a New Topic</a> </td> <td>•</td> <td style=3D"font-size: 12px; font-family: arial; padding: 7px 5p= x 5px;color: #2D50FD;" > <a href=3D"https://groups.yahoo.com/neo/groups/= gobo-eiffel/conversations/topics/1923;_ylc=3DX3oDMTM0ZDBoNXNpBF9TAzk3MzU5Nz= E0BGdycElkAzE1MjIyNzAEZ3Jwc3BJZAMxNzA1MDA2NzY0BG1zZ0lkAzE5MzEEc2VjA2Z0cgRzb= GsDdnRwYwRzdGltZQMxNTk0MTEwNDYzBHRwY0lkAzE5MjM-" style=3D"text-decoration: = none; color: #2D50FD;">Messages in this topic</a> (9) </td> </tr> </tbody> </table> =20=20=20=20=20=20=20=20 <!------- Start Nav Bar ------> <!-- |**|begin egp html banner|**| --> <!-- |**|end egp html banner|**| --> <div id=3D"ygrp-grfd" style=3D"font-family: Verdana; font-size: 12px; p= adding: 15px 0;"> =20=20=20=20=20=20 <!-- |**|begin egp html banner|**| --> To Post a message, send it to: [email protected]<BR= > To Unsubscribe, send a blank message to: gobo-eiffel-unsubscribe-B11MqFFcr06IwRZHo2/[email protected]= m=20=20=20=20=20=20 <!-- |**|end egp html banner|**| --> </div> =20=20 =20 <!-- |**|begin egp html banner|**| --> <div id=3D"ygrp-vital" style=3D"background-color: #f2f2f2; font-family: Ver= dana; font-size: 10px; margin-bottom: 10px; padding: 10px;"> <span id=3D"vithd" style=3D"font-weight: bold; color: #333; text-transf= orm: uppercase; "><a href=3D"https://groups.yahoo.com/neo/groups/gobo-eiffe= l/info;_ylc=3DX3oDMTJlZDkxNDdvBF9TAzk3MzU5NzE0BGdycElkAzE1MjIyNzAEZ3Jwc3BJZ= AMxNzA1MDA2NzY0BHNlYwN2dGwEc2xrA3ZnaHAEc3RpbWUDMTU5NDExMDQ2Mw--" style=3D"t= ext-decoration: none;">Visit Your Group</a></span> <ul style=3D"list-style-type: none; margin: 0; padding: 0; display: in= line;"> </ul> </div> <div id=3D"ft" style=3D"font-family: Arial; font-size: 11px; margin-top: 5p= x; padding: 0 2px 0 0; clear: both;"> <a href=3D"https://groups.yahoo.com/neo;_ylc=3DX3oDMTJkczhjYW5wBF9TAzk3Mz= U5NzE0BGdycElkAzE1MjIyNzAEZ3Jwc3BJZAMxNzA1MDA2NzY0BHNlYwNmdHIEc2xrA2dmcARzd= GltZQMxNTk0MTEwNDYz" style=3D"float: left;"><img src=3D"https://s.yimg.com/= ru/0.9.12/min/css/yahoo_en-US_f_pw_101x21.png" height=3D"21" width=3D"101" = alt=3D"Yahoo! Groups" style=3D"border: 0;"/></a> <div style=3D"color: #747575; float: right;"> • <a href=3D"https://i= nfo.yahoo.com/privacy/us/yahoo/groups/details.html" style=3D"text-decoratio= n: none;">Privacy</a> • <a href=3D"mailto:gobo-eiffel-unsubscribe@yaho= ogroups.com?subject=3DUnsubscribe" style=3D"text-decoration: none;">Unsubsc= ribe</a> • <a href=3D"https://info.yahoo.com/legal/us/yahoo/utos/terms= /" style=3D"text-decoration: none;">Terms of Use</a> </div> </div> <br> <!-- |**|end egp html banner|**| --> </div> <!-- ygrp-msg --> =20 <!-- Sponsor --> <!-- |**|begin egp html banner|**| --> <div id=3D"ygrp-sponsor" style=3D"width:160px; float:right; clear:none; m= argin:0 0 25px 0; background: #fff;"> <!-- Start Recommendations --> <div id=3D"ygrp-reco"> </div> <!-- End Recommendations --> </div> <!-- |**|end egp html banner|**| --> <div style=3D"clear:both; color: #FFF; font-size:1px;">.</div> </div> <img src=3D"http://geo.yahoo.com/serv?s=3D97359714/grpId=3D1522270/grpspI= d=3D1705006764/msgId=3D1931/stime=3D1594110463" width=3D"1" height=3D"1"> <= br> <img src=3D"http://y.analytics.yahoo.com/fpc.pl?ywarid=3D515FB27823A7407E&a= =3D10001310322279&js=3Dno&resp=3Dimg" width=3D"1" height=3D"1">=20 <div style=3D"color: #fff; height: 0;">__,_._,___</div> <!--~-|**|PrettyHtmlEnd|**|-~--> </body> <!--~-|**|PrettyHtmlStart|**|-~--> <head> <style type=3D"text/css"> <!-- #ygrp-mkp { border: 1px solid #d8d8d8; font-family: Arial; margin: 10px 0; padding: 0 10px; } #ygrp-mkp hr { border: 1px solid #d8d8d8; } #ygrp-mkp #hd { color: #628c2a; font-size: 85%; font-weight: 700; line-height: 122%; margin: 10px 0; } #ygrp-mkp #ads { margin-bottom: 10px; } #ygrp-mkp .ad { padding: 0 0; } #ygrp-mkp .ad p { margin: 0; } #ygrp-mkp .ad a { color: #0000ff; text-decoration: none; } #ygrp-sponsor #ygrp-lc { font-family: Arial; } #ygrp-sponsor #ygrp-lc #hd { margin: 10px 0px; font-weight: 700; font-size: 78%; line-height: 122%; } #ygrp-sponsor #ygrp-lc .ad { margin-bottom: 10px; padding: 0 0; } #actions { font-family: Verdana; font-size: 11px; padding: 10px 0; } #activity { background-color: #e0ecee; float: left; font-family: Verdana; font-size: 10px; padding: 10px; } #activity span { font-weight: 700; } #activity span:first-child { text-transform: uppercase; } #activity span a { color: #5085b6; text-decoration: none; } #activity span span { color: #ff7900; } #activity span .underline { text-decoration: underline; } .attach { clear: both; display: table; font-family: Arial; font-size: 12px; padding: 10px 0; width: 400px; } .attach div a { text-decoration: none; } .attach img { border: none; padding-right: 5px; } .attach label { display: block; margin-bottom: 5px; } .attach label a { text-decoration: none; } =20=20 blockquote { margin: 0 0 0 4px; } .bold { font-family: Arial; font-size: 13px; font-weight: 700; } .bold a { text-decoration: none; } dd.last p a { font-family: Verdana; font-weight: 700; } dd.last p span { margin-right: 10px; font-family: Verdana; font-weight: 700; } dd.last p span.yshortcuts { margin-right: 0; } div.attach-table div div a { text-decoration: none; } div.attach-table { width: 400px; } div.file-title a, div.file-title a:active, div.file-title a:hover, div.fi= le-title a:visited { text-decoration: none; } div.photo-title a, div.photo-title a:active, div.photo-title a:hover, div= ..photo-title a:visited { text-decoration: none; } div#ygrp-mlmsg #ygrp-msg p a span.yshortcuts { font-family: Verdana; font-size: 10px; font-weight: normal; } .green { color: #628c2a; } .MsoNormal { margin: 0 0 0 0; } o { font-size: 0; } #photos div { float: left; width: 72px; } #photos div div { border: 1px solid #666666; height: 62px; overflow: hidden; width: 62px; } #photos div label { color: #666666; font-size: 10px; overflow: hidden; text-align: center; white-space: nowrap; width: 64px; } #reco-category { font-size: 77%; } #reco-desc { font-size: 77%; } .replbq { margin: 4px; } #ygrp-actbar div a:first-child { /* border-right: 0px solid #000;*/ margin-right: 2px; padding-right: 5px; } #ygrp-mlmsg { font-size: 13px; font-family: Arial, helvetica,clean, sans-serif; *font-size: small; *font: x-small; } #ygrp-mlmsg table { font-size: inherit; font: 100%; } #ygrp-mlmsg select, input, textarea { font: 99% Arial, Helvetica, clean, sans-serif; } #ygrp-mlmsg pre, code { font:115% monospace; *font-size:100%; } #ygrp-mlmsg * { line-height: 1.22em; } #ygrp-mlmsg #logo { padding-bottom: 10px; } #ygrp-msg p a { font-family: Verdana; } #ygrp-msg p#attach-count span { color: #1E66AE; font-weight: 700; } #ygrp-reco #reco-head { color: #ff7900; font-weight: 700; } #ygrp-reco { margin-bottom: 20px; padding: 0px; } #ygrp-sponsor #ov li a { font-size: 130%; text-decoration: none; } #ygrp-sponsor #ov li { font-size: 77%; list-style-type: square; padding: 6px 0; }=20 #ygrp-sponsor #ov ul { margin: 0; padding: 0 0 0 8px; } #ygrp-text { font-family: Georgia; } #ygrp-text p { margin: 0 0 1em 0; } #ygrp-text tt { font-size: 120%; } #ygrp-vital ul li:last-child { border-right: none !important;=20 }=20 --> </style> </head> <!--~-|**|PrettyHtmlEnd|**|-~--> </html> <!-- end group email --> --wQxlCi9O7V1TCpTfFOpUEAv3HsNJvaep0ZYBHib--