Re: In 2.3.19, sharing cache storage

Wednesday, July 23, 2014, 10:56:31 PM, Jaime Garza wrote:

>> -----Original Message-----
>> From: Daniel Dekany [mailto:[email protected]]
>> Sent: Wednesday, July 23, 2014 1:06 PM
>> To: Jaime Garza
>> Cc: Daniel Dekany
>> Subject: Re: [Freemarker-devel] In 2.3.19, sharing cache storage
>> 
>> Wednesday, July 23, 2014, 8:48:28 PM, Jaime Garza wrote:
>> 
>> > The modifications that I have done have to do with adaptations to FM
>> > to make it more of a DSL. We use FM for formatting emails and SMS
>> > messages. I have modified mostly FMParser.jj keeping a series of svn
>> > patch files for each one of my modifications, hopefully helping me
>> > move to a more recent FM. We have extensions for sql queries with a
>> > <#data> directive, data phrasing as a builtin for sequences, ?unique
>> > for sequences, etc.
>> 
> Yeah I know that. We have a myriad of functions we implement. Our
> additions look, as you mention, terse. It appears as part of the
> language, which it now is. It would have been a dream to be able to
> add additional built-ins, and I have been following the list for a
> while now, and I know that this is a common ask and that you are
> thinking on ways to do it. I look forward to the moment we can have
> "legal" built-ins in the environment. Adding directives in a
> standard way with <# would have been ideal as well.

In the far future I plan to add namespace prefixes via colon. This is
very similar to #import at first glance, but has some important
benefits. One of them is that if you have a function (either defined
in FTL or in Java) then you can call it both like my:unique(seq) or
with postfix suntax, seq?my:unique. (Also then the "built-in" term is
gone - they are just core functions.) A side benefit is that
<#my:data/> perhaps looks less off than <@my.date/>. And of course,
just like #imports can be done on Configuration-level, prefixes could
be defined there too, so it doesn't have to be done in each template
again and again.

Now, if adding those "my:"-s (or whatever prefix you chose, obviously)
and maybe the explicit "/>"-s still seen as too awkward (is it?), this
idea could be further extended so that you can define your own
template language be directives/functions to the core namespace in the
Configuration level (read: pre-parse-time). I'm afraid of that though,
as then it's too easy to create custom languages, so people will do it
even when it's doesn't really worth the chaos that it causes later
heads (manual doesn't mach, copy-pasting from forums doesn't work, IDE
plugings complain, etc.). OTOH it would be useful when otherwise
someone really had to make a patched FreeMarker version.

>> You can have custom directives and functions without modifying FM, as you
>> certainly know. Surely they are not as flexible/terse as what can be achieved
>> with core directives and built-ins... Making them closer is a future goal.
>> 
>> > We maintain our own FM document based on the
>> > 2.3.19 docs. We needed some validation support,
>> 
>> Validation of what?
>
> Users declare their own namespaces in the UI. These are attached to
> database rows and appear in the root namespace. We need to know if
> they are really used in the templates at all, for backend
> optimizations.
> We warn users if they declare them and then they do not use them.

Sounds like that part can be solved by traversing the AST and collect
the variables referenced. (There's no publISHED API for that yet
though.)

> For this purpose we follow templates depth-first in a parsing phase
> as much as possible. If the include uses string expressions, we warn
> too, as we cannot proceed. We use the parse flag to follow includes.
> Etc. It is kind of elaborate, but it ended being very powerful.

This is maybe also solvable with AST traversal.

> In fact, I have been following the discussion of editor plugins and
> how we can get the parsing tree, and how people have done this in
> their own way, and then communicate with you. This is a way in which
> we discover the namespace for us, in-depth. I instrumented
> FMParser's PrimaryExpression and Include for this. I did not have to
> open DollarVariable.java or Include.java for it.
>
>> 
>> > multiple error messages per template (one error was not acceptable,)
>> 
>> Is that multiple parsing errors or?
>
> We do multiple parse errors, and lexical errors as well. We
> implemented a recover mechanism to stabilize the parsing process as
> much as possible, by skipping tokens, and backtracking. We then
> return a list of errors as opposed to one error. This was a big
> change.

This is also in the plans because of the IDE plugins.

So yeah... looks like all well known issues again. As always, the
problem is the lack of time.

-- 
Thanks,
 Daniel Dekany

>> 
>> > and others.
>> 
>> --
>> Thanks,
>>  Daniel Dekany
>> 
>> 
>> > Thanks
>> > Jaime
>> >
>> > -----Original Message-----
>> > From: Daniel Dekany [mailto:[email protected]]
>> > Sent: Wednesday, July 23, 2014 11:22 AM
>> > To: Jaime Garza
>> > Cc: [email protected]
>> > Subject: Re: [Freemarker-devel] In 2.3.19, sharing cache storage
>> >
>> > The problem with the approach is that Template-s are bound to a
>> > Configuration instance (they store a reference to it), and sometimes
>> > may reach back to it during template execution. However, can't you
>> > just use a common Configuration instance on all those places that share
>> templates?
>> >
>> > (Out of curiosity... what's the modification in 2.3.19 is about?)
>> >
>> > --
>> > Thanks,
>> >  Daniel Dekany
>> >
>> > Wednesday, July 23, 2014, 5:51:52 PM, Jaime Garza wrote:
>> >
>> >> Hi Daniel,
>> >>
>> >> I am trying to share an instance of MruCacheStorage across threads.
>> >> I keep it around across multiple threads and when creating my
>> >> Configuration, I simply do a Configuration#setCacheStorage(). This
>> >> internally calls Configuration#createTemplateCache, which calls
>> >> cache.setConfiguration(this), which calls TemplateCache#clear(), which
>> >> eventually clears the storage.
>> >>
>> >> My use case is that I know that a set of around 200 threads will be
>> >> accessing the same templates over and over (executing the same
>> >> template(s) on a different root namespace) concurrently. I want the
>> >> loading of templates to happen as infrequently as possible. Also,
>> >> since I am on a slightly customized version of 2.3.19, I cannot come to the
>> greatest new version.
>> >>
>> >> The question (or questions) is: is there a way to implement my
>> >> strategy? My current workaround is nasty. I have created a subclass of
>> >> MruCacheStorage, which reads a flag "canClear" from thread local, and
>> >> then decides whether to let the clear proceed or not. I sandwitched
>> >> the call to setCacheStorage() with canClear = false and canClear =
>> >> true
>> >>
>> >>         SharedCacheSettings.disallowClear();
>> >>         config.setCacheStorage(storage);
>> >>         SharedCacheSettings.allowClear();
>> >>
>> >> AND
>> >>
>> >> public class SharedCacheStorage extends MruCacheStorage {
>> >>     public SharedCacheStorage(int maxStrongSize, int maxSoftSize) {
>> >>         super(maxStrongSize, maxSoftSize);
>> >>     }
>> >>
>> >>     @Override
>> >>     public void clear() {
>> >>         if (!SharedCacheSettings.canClear())
>> >>             return;
>> >>
>> >>         super.clear();
>> >>     }
>> >> }
>> >>
>> >> What is your opinion about my hack? What kind of trouble am I inviting?
>> >>
>> >> Thanks in advance.
>> >> Jaime Garza


------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds