Re: Use ClearSilver in Multi-threading environment?
caox <[email protected]> Tue, 27 Dec 2011 16:07:11 +0800
| Newsgroups | gmane.text.clearsilver.general |
|---|---|
| Message-ID | <[email protected]> |
--Apple-Mail-29-921164309 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, Brandon We use ClearSilver as our template engine in our product, and we faced the = problem your referred a few months ago. In our situation, we have lots of developers put different cs marco into on= e global util.cs file so that we can include it in different cs pages for r= e-using. As this global file grows, the parsing time of every cs page that = includes it becomes very long. And it makes our system slow, as a bottlenec= k, and most of the time is consumed by parsing of util.cs. Then we decided to re-use the CSPARSE structure of each cs pages. Since it = is not thread safe, we need to make one CSPARSE instance for each threads. That means we need 8 instances for 8 threads. Besides, we can't use cs synt= ax such as <?cs include :Var ?>. Typically, we have same number threads as = the CPU cores, and it won't be a problem since the system memory is quite e= nough to hold lots of CSPARSE structures. But we still want to optimize it = in a more decent way. There maybe several choices. As you mentioned, we could split the parse and= the render process to make the parse read-only. And I was also wondering i= f we could change the implementation of cs:include or give a callback to im= plement it, like we could include a pre-prased structure directly. Do you have any suggestions? B.R. caox On 2011-9-4, at =E4=B8=8B=E5=8D=884:50, Brandon Long wrote: > On Sat, Sep 3, 2011 at 10:51 AM, Jacky Wang <[email protected]> w= rote: > > Hi, > > > > Thanks for the helps and ClearSilver runs flawlessly on my machine. > > > > So I'm curiosity on the tips/best practices of its usage under multi-th= reading environment. > > > > The sample code is listed as following: > > - callback is the function. > > - both cs and hdf are shared pointers. > > - output is a local ctx pointer. > > > > <code> > > cs->hdf =3D hdf; > > cs_render(cs, &output, &callback); // Do I need to put a lock here? > > </code> > > > > My understanding is that if the cs_render actually doesn't change the s= tructure of either cs or hdf, the lock is not needed (thus won't hurt the p= erformance). Does it sound? >=20 > No, this isn't true. I'm sure that CSPARSE contains data structures > which are updated on render, and cs commands like set definitely > modify hdf. For example, we don't pass the callback around, its > actually set in the CSPARSE struct. I guess in theory we could split > CSPARSE into both a PARSE and a RENDER struct, so that the PARSE would > be read-only for render, but it hasn't been the typical way we've done > this. Typically, the parse is somewhat specific to the instance, > though I'm sure that's not necessary (jsilver definitely tries to > re-use parses, since parsing is expensive. It also means that the > equivalent of cs_render in jsilver is thread-safe with a shared cs, > but again, HDF isn't thread-safe if the cs render has side affects). >=20 > From a normal standpoint, cs & hdf are thread safe in the sense that > you can use separate instances from separate threads safely, but they > are not safe to use from multiple threads at once. There is a concept > of a "global HDF", though, which can be loaded and shared across > threads, as cs_parse/cs_render treat it as read-only. That isn't to > say you can update it from another thread while its being used, but > you'd typically load a bunch of data into it at startup, and use it as > a read-only structure from that point. >=20 > The cgi libraries are mostly not thread-safe with the exception of the > escape functions. This is because the underlying CGI interface is > based on global data (environment variables, stdin & stdout). Since > all access to that is through the cgiwrap interface, I've debated > making the wrapper thread-safe so that you could "mimic" the CGI > interface in a thread-safe fashion, but I'm unsure how useful that > would end up being in practice. >=20 > Brandon >=20 --Apple-Mail-29-921164309 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/htm= l4/strict.dtd"> <html> <head> </head> <body style=3D"background-color: #fff;"> <span style=3D"display:none"> </span> <!--~-|**|PrettyHtmlStartT|**|-~--> <div id=3D"ygrp-mlmsg" style=3D"position:relative;"> <div id=3D"ygrp-msg" style=3D"z-index: 1;"> <!--~-|**|PrettyHtmlEndT|**|-~--> <div id=3D"ygrp-text" > =20=20=20=20=20=20 =20=20=20=20=20=20 <p><div>Hi, Brandon</div><div><br></div>We use ClearSilver as our tem= plate engine in our product, and we faced the problem your referred a= few months ago.<div><br></div><div>In our situation, we have lots of devel= opers put different cs marco into one global util.cs file so that we can in= clude it in different cs pages for re-using. As this global file grows, the= parsing time of every cs page that includes it becomes very long. And it m= akes our system slow, as a bottleneck, and most of the time is consumed by = parsing of util.cs.</div><div><br></div><div>Then we decided to re-use the = CSPARSE structure of each cs pages. Since it is not thread safe, we need to= make one CSPARSE instance for each threads.</div><div><br></div><div>That = means we need 8 instances for 8 threads. Besides, we can't use cs syntax su= ch as <?cs include :Var ?>. Typically, we have same number threads as= the CPU cores, and it won't be a problem since the system memory is quite = enough to hold lots of CSPARSE structures. But we still want to optimize it= in a more decent way.</div><div><br></div><div>There maybe several choices= . As you mentioned, we could split the parse and the render process to make= the parse read-only. And I was also wondering if we could change the imple= mentation of cs:include or give a callback to implement it, like we could i= nclude a pre-prased structure directly.</div><div><br></div><div>Do you hav= e any suggestions?</div><div><br></div><div>B.R.</div><div><br></div><div>c= aox</div><div><br><div><div>On 2011-9-4, at =E4=B8=8B=E5=8D=884:50, Brandon= Long wrote:</div><br class=3D"Apple-interchange-newline"><blockquote type= =3D"cite"> <div style=3D"background-color: #fff;"> <span> </span> <div id=3D"ygrp-text"><p>On Sat, Sep 3, 2011 at 10:51 AM, Jacky Wang &l= t;<a href=3D"mailto:jacky.chao.wang%40gmail.com">[email protected]<= /a>> wrote:<br> > Hi,<br> ><br> > Thanks for the helps and ClearSilver runs flawlessly on my machine.<br= > ><br> > So I'm curiosity on the tips/best practices of its usage under multi-t= hreading environment.<br> ><br> > The sample code is listed as following:<br> > - callback is the function.<br> > - both cs and hdf are shared pointers.<br> > - output is a local ctx pointer.<br> ><br> > <code><br> > cs->hdf =3D hdf;<br> > cs_render(cs, &output, &callback); // Do I need to put a= lock here?<br> > </code><br> ><br> > My understanding is that if the cs_render actually doesn't change the = structure of either cs or hdf, the lock is not needed (thus won't hurt the = performance). Does it sound?<br> <br> No, this isn't true. I'm sure that CSPARSE contains data structures<br> which are updated on render, and cs commands like set definitely<br> modify hdf. For example, we don't pass the callback around, its<br> actually set in the CSPARSE struct. I guess in theory we could split<br> CSPARSE into both a PARSE and a RENDER struct, so that the PARSE would<br> be read-only for render, but it hasn't been the typical way we've done<br> this. Typically, the parse is somewhat specific to the instance,<br> though I'm sure that's not necessary (jsilver definitely tries to<br> re-use parses, since parsing is expensive. It also means that the<br> equivalent of cs_render in jsilver is thread-safe with a shared cs,<br> but again, HDF isn't thread-safe if the cs render has side affects).<br> <br> From a normal standpoint, cs & hdf are thread safe in the sense that<br= > you can use separate instances from separate threads safely, but they<br> are not safe to use from multiple threads at once. There is a concept<br> of a "global HDF", though, which can be loaded and shared across<br> threads, as cs_parse/cs_render treat it as read-only. That isn't to<br> say you can update it from another thread while its being used, but<br> you'd typically load a bunch of data into it at startup, and use it as<br> a read-only structure from that point.<br> <br> The cgi libraries are mostly not thread-safe with the exception of the<br> escape functions. This is because the underlying CGI interface is<br> based on global data (environment variables, stdin & stdout). Since<br= > all access to that is through the cgiwrap interface, I've debated<br> making the wrapper thread-safe so that you could "mimic" the CGI<br> interface in a thread-safe fashion, but I'm unsure how useful that<br> would end up being in practice.<br> <br> Brandon<br> </p> </div> =20=20=20=20=20 =20=20=20=20 </div> <!-- end group email --> </blockquote></div><br></div></p> </div> =20=20=20=20=20 <!--~-|**|PrettyHtmlStart|**|-~--> <div style=3D"color: #fff; height: 0;">__._,_.___</div> =20=20=20=20=20=20=20=20 =20=20 =20=20=20 <div id=3D"ygrp-actbar" style=3D"clear: both; margin-bottom: 10px; whit= e-space: nowrap; color: #666; padding-top: 15px;"> <div> <a href=3D"mailto:[email protected]?subject=3DRe%3A%20Use%20ClearS= ilver%20in%20Multi-threading%20environment%3F" style=3D"margin-right: 0; pa= dding-right: 0;"> Reply to <span style=3D"font-weight: 700;">sender</span></a> | <a href=3D"mailto:[email protected]?subject=3DRe%3A%20Use= %20ClearSilver%20in%20Multi-threading%20environment%3F"> Reply to <span style=3D"font-weight: 700;">group</span></a> | <a href=3D"http://groups.yahoo.com/group/ClearSilver/post;_ylc= =3DX3oDMTJwNWs0bm9pBF9TAzk3MzU5NzE0BGdycElkAzM1NjQxMzcEZ3Jwc3BJZAMxNzA3Mjgx= OTQyBG1zZ0lkAzE0MjkEc2VjA2Z0cgRzbGsDcnBseQRzdGltZQMxMzI0OTczMjU4?act=3Drepl= y&messageNum=3D1429">Reply <span style=3D"font-weight: 700;">via web post</= span></a> | <a href=3D"http://groups.yahoo.com/group/ClearSilver/post;_ylc= =3DX3oDMTJldmdmNzZzBF9TAzk3MzU5NzE0BGdycElkAzM1NjQxMzcEZ3Jwc3BJZAMxNzA3Mjgx= OTQyBHNlYwNmdHIEc2xrA250cGMEc3RpbWUDMTMyNDk3MzI1OA--" style=3D"font-weight:= 700;">Start a New Topic</a> </div> <a href=3D"http://groups.yahoo.com/group/ClearSilver/messag= e/1412;_ylc=3DX3oDMTM0MDVvdGxzBF9TAzk3MzU5NzE0BGdycElkAzM1NjQxMzcEZ3Jwc3BJZ= AMxNzA3MjgxOTQyBG1zZ0lkAzE0MjkEc2VjA2Z0cgRzbGsDdnRwYwRzdGltZQMxMzI0OTczMjU4= BHRwY0lkAzE0MTI-">Messages in this topic</a> (<span style=3D"font-weight: 700;">3</span>) </div>=20 <!------- Start Nav Bar ------> <!-- |**|begin egp html banner|**| --> <div id=3D"ygrp-vital" style=3D"background-color: #e0ecee; font-family: Ver= dana; font-size: 10px; margin-bottom: 10px; padding: 10px;"> <span id=3D"vithd" style=3D"font-weight: bold; color: #333; text-tran= sform: uppercase; ">Recent Activity:</span> <ul style=3D"list-style-type: none; margin: 0; padding: 0; display: inl= ine;"> <li style=3D"border-right: 1px solid #000; font-weight: 700; di= splay: inline; padding: 0 5px; margin-left: 0;"> <span class=3D"cat"><a href=3D"http://groups.yahoo.com/group/ClearSil= ver/members;_ylc=3DX3oDMTJmZDl1cmE1BF9TAzk3MzU5NzE0BGdycElkAzM1NjQxMzcEZ3Jw= c3BJZAMxNzA3MjgxOTQyBHNlYwN2dGwEc2xrA3ZtYnJzBHN0aW1lAzEzMjQ5NzMyNTg-?o=3D6"= style=3D"text-decoration: none;">New Members</a></span> <span class=3D"ct" style=3D"color: #ff7900;">1</span> </li> </ul> =20=20=20=20 <div style=3D"clear: both; padding-top: 2px; color: #1e66ae;"> <a href=3D"http://groups.yahoo.com/group/ClearSilver;_ylc=3DX3oDMTJldGw= 3Y2NyBF9TAzk3MzU5NzE0BGdycElkAzM1NjQxMzcEZ3Jwc3BJZAMxNzA3MjgxOTQyBHNlYwN2dG= wEc2xrA3ZnaHAEc3RpbWUDMTMyNDk3MzI1OA--" style=3D"text-decoration: none;">Vi= sit Your Group</a> </div> </div> <div id=3D"ygrp-mkp" style=3D"border: 1px solid #d8d8d8; clea= r: both; float: left; font-family: Arial; margin: 5px 0 10px 0; padding: 0 = 10px;"> <div id=3D"hd" style=3D"color: #628c2a; font-size: 85%; font-weight: = 700; line-height: 122%; margin: 10px 0;">MARKETPLACE</div> <div id=3D"ads" style=3D"margin-bottom: 10px;"> <div class=3D"ad" style=3D"color: #628C2A; font-f= amily: Arial; font-weight: 700; padding: 0 0;"> <p><a href=3D"http://global.ard.yahoo.com/SIG=3D15o= rfqvto/M=3D493064.14543979.14562481.13298430/D=3Dgroups/S=3D1707281942:MKP1= /Y=3DYAHOO/EXP=3D1324980458/L=3Dd7ae901a-3061-11e1-b1ef-73ba27b66dd9/B=3DMi= UwSEoGYnY-/J=3D1324973258664868/K=3D4pEPQ70BJkMHrs6cb7D5TA/A=3D6060255/R=3D= 0/SIG=3D1194m4keh/*http://us.toolbar.yahoo.com/?.cpdl=3Dgrpj">Stay on top o= f your group activity without leaving the page you're on - Get the Yahoo! T= oolbar now.</a></p> <script language=3Djavascript> if(window.yzq_d=3D=3Dnull)window.yzq_d=3Dnew Object(); window.yzq_d['MiUwSEoGYnY-']=3D'&U=3D13c7lejuv%2fN%3dMiUwSEoGYnY-%2fC%3d493= 064.14543979.14562481.13298430%2fD%3dMKP1%2fB%3d6060255%2fV%3d1'; </script><noscript><img width=3D1 height=3D1 alt=3D"" src=3D"http://us.bc.y= ahoo.com/b?P=3Dd7ae901a-3061-11e1-b1ef-73ba27b66dd9&T=3D1d5578v10%2fX%3d132= 4973258%2fE%3d1707281942%2fR%3dgroups%2fK%3d5%2fV%3d2.1%2fW%3dH%2fY%3dYAHOO= %2fF%3d3043583802%2fH%3dY29udGVudD0iR287UG9kY2FzdHM7V2lkZ2V0cztGbGlja3I7WWF= ob29fU2VhcmNoX01hcmtldGluZztGaW5hbmNlO0Jvb2ttYXJrO01lc3NhZ2VfQm9hcmRzO0NhbG= VuZGFyO0V2ZW50czsiIGRpc2FibGVzaHVmZmxpbmc9IjEiIHNlcnZlSWQ9ImQ3YWU5MDFhLTMwN= jEtMTFlMS1iMWVmLTczYmEyN2I2NmRkOSIgc2l0ZUlkPSI0NDUyNTUxIiB0U3RtcD0iMTMyNDk3= MzI1ODYzMzIzMyIg%2fQ%3d-1%2fS%3d1%2fJ%3d28228962&U=3D13c7lejuv%2fN%3dMiUwSE= oGYnY-%2fC%3d493064.14543979.14562481.13298430%2fD%3dMKP1%2fB%3d6060255%2fV= %3d1"></noscript><!--QYZ ,;;MKP1;1707281942;2;--> </div> </div> </div> =20=20 <div id=3D"ft" style=3D"font-family: Arial; font-size: 11px; margin-top: 5p= x; padding: 0 2px 0 0; clear: both;"> <a href=3D"http://groups.yahoo.com/;_ylc=3DX3oDMTJkbjF1czM4BF9TAzk3NDc2NT= kwBGdycElkAzM1NjQxMzcEZ3Jwc3BJZAMxNzA3MjgxOTQyBHNlYwNmdHIEc2xrA2dmcARzdGltZ= QMxMzI0OTczMjU4" style=3D"float: left;"><img src=3D"http://l.yimg.com/a/i/u= s/yg/logo/us.gif" height=3D"15" width=3D"137" alt=3D"Yahoo! Groups" style= =3D"border: 0;"/></a> <div style=3D"color: #747575; float: right;">Switch to: <a href=3D"mailto= :[email protected]?subject=3DChange Delivery Format: = Traditional" style=3D"text-decoration: none;">Text-Only</a>, <a href=3D"mai= lto:[email protected]?subject=3DEmail Delivery: Digest" cl= ass=3D"margin-rt" style=3D"text-decoration: none;">Daily Digest</a> • = <a href=3D"mailto:[email protected]?subject=3DUnsubsc= ribe" style=3D"text-decoration: none;">Unsubscribe</a> • <a href=3D"ht= tp://docs.yahoo.com/info/terms/" style=3D"text-decoration: none;">Terms of = Use</a></div> </div> <!-- |**|end egp html banner|**| --> </div> <!-- ygrp-msg --> <!-- Sponsor --> <!-- |**|begin egp html banner|**| --> <div id=3D"ygrp-sponsor" style=3D"width:160px; float:right; clear:none; m= argin:0 0 25px 0; background: #fff;"> <!-- Start Recommendations --> <div id=3D"ygrp-reco"> </div> <!-- End Recommendations --> </div> <!-- |**|end egp html banner|**| --> <div style=3D"clear:both; color: #FFF; font-size:1px;">.</div> </div> <img src=3D"http://geo.yahoo.com/serv?s=3D97359714/grpId=3D3564137/grpspI= d=3D1707281942/msgId=3D1429/stime=3D1324973258/nc1=3D4507179/nc2=3D5191954/= nc3=3D3848641" width=3D"1" height=3D"1"> <br> <div style=3D"color: #fff; height: 0;">__,_._,___</div> <!--~-|**|PrettyHtmlEnd|**|-~--> </body> <!--~-|**|PrettyHtmlStart|**|-~--> <head> <style type=3D"text/css"> <!-- #ygrp-mkp { border: 1px solid #d8d8d8; font-family: Arial; margin: 10px 0; padding: 0 10px; } #ygrp-mkp hr { border: 1px solid #d8d8d8; } #ygrp-mkp #hd { color: #628c2a; font-size: 85%; font-weight: 700; line-height: 122%; margin: 10px 0; } #ygrp-mkp #ads { margin-bottom: 10px; } #ygrp-mkp .ad { padding: 0 0; } #ygrp-mkp .ad p { margin: 0; } #ygrp-mkp .ad a { color: #0000ff; text-decoration: none; } #ygrp-sponsor #ygrp-lc { font-family: Arial; } #ygrp-sponsor #ygrp-lc #hd { margin: 10px 0px; font-weight: 700; font-size: 78%; line-height: 122%; } #ygrp-sponsor #ygrp-lc .ad { margin-bottom: 10px; padding: 0 0; } a { color: #1e66ae; } #actions { font-family: Verdana; font-size: 11px; padding: 10px 0; } #activity { background-color: #e0ecee; float: left; font-family: Verdana; font-size: 10px; padding: 10px; } #activity span { font-weight: 700; } #activity span:first-child { text-transform: uppercase; } #activity span a { color: #5085b6; text-decoration: none; } #activity span span { color: #ff7900; } #activity span .underline { text-decoration: underline; } .attach { clear: both; display: table; font-family: Arial; font-size: 12px; padding: 10px 0; width: 400px; } .attach div a { text-decoration: none; } .attach img { border: none; padding-right: 5px; } .attach label { display: block; margin-bottom: 5px; } .attach label a { text-decoration: none; } =20=20 blockquote { margin: 0 0 0 4px; } .bold { font-family: Arial; font-size: 13px; font-weight: 700; } .bold a { text-decoration: none; } dd.last p a { font-family: Verdana; font-weight: 700; } dd.last p span { margin-right: 10px; font-family: Verdana; font-weight: 700; } dd.last p span.yshortcuts { margin-right: 0; } div.attach-table div div a { text-decoration: none; } div.attach-table { width: 400px; } div.file-title a, div.file-title a:active, div.file-title a:hover, div.fi= le-title a:visited { text-decoration: none; } div.photo-title a, div.photo-title a:active, div.photo-title a:hover, div= .photo-title a:visited { text-decoration: none; } div#ygrp-mlmsg #ygrp-msg p a span.yshortcuts { font-family: Verdana; font-size: 10px; font-weight: normal; } .green { color: #628c2a; } .MsoNormal { margin: 0 0 0 0; } o { font-size: 0; } #photos div { float: left; width: 72px; } #photos div div { border: 1px solid #666666; height: 62px; overflow: hidden; width: 62px; } #photos div label { color: #666666; font-size: 10px; overflow: hidden; text-align: center; white-space: nowrap; width: 64px; } #reco-category { font-size: 77%; } #reco-desc { font-size: 77%; } .replbq { margin: 4px; } #ygrp-actbar div a:first-child { /* border-right: 0px solid #000;*/ margin-right: 2px; padding-right: 5px; } #ygrp-mlmsg { font-size: 13px; font-family: Arial, helvetica,clean, sans-serif; *font-size: small; *font: x-small; } #ygrp-mlmsg table { font-size: inherit; font: 100%; } #ygrp-mlmsg select, input, textarea { font: 99% Arial, Helvetica, clean, sans-serif; } #ygrp-mlmsg pre, code { font:115% monospace; *font-size:100%; } #ygrp-mlmsg * { line-height: 1.22em; } #ygrp-mlmsg #logo { padding-bottom: 10px; } #ygrp-mlmsg a { color: #1E66AE; } #ygrp-msg p a { font-family: Verdana; } #ygrp-msg p#attach-count span { color: #1E66AE; font-weight: 700; } #ygrp-reco #reco-head { color: #ff7900; font-weight: 700; } #ygrp-reco { margin-bottom: 20px; padding: 0px; } #ygrp-sponsor #ov li a { font-size: 130%; text-decoration: none; } #ygrp-sponsor #ov li { font-size: 77%; list-style-type: square; padding: 6px 0; }=20 #ygrp-sponsor #ov ul { margin: 0; padding: 0 0 0 8px; } #ygrp-text { font-family: Georgia; } #ygrp-text p { margin: 0 0 1em 0; } #ygrp-text tt { font-size: 120%; } #ygrp-vital ul li:last-child { border-right: none !important;=20 }=20 --> </style> </head> <!--~-|**|PrettyHtmlEnd|**|-~--> </html> <!-- end group email --> --Apple-Mail-29-921164309--