Re: Macro caching and other caching

What do you use for logTag (template name) when you are using evaluate()?

On Tue, Jul 31, 2012 at 8:01 AM, Bradley Wagner
<[email protected]> wrote:
> Doing both. In the other case we're using a classpath resource loader to
> evaluate templates like this:
>
> VelocityContext = ... a context that we're building each time ...
> VelocityEngine engine = ... our single engine ...
> Template template = engine.getTemplate(templatePath);
> StringWriter writer = new StringWriter();
> template.merge(context, writer);
>
> However, we only have 7 of those static templates in our whole system.
>
> On Tue, Jul 31, 2012 at 10:52 AM, Nathan Bubna <[email protected]> wrote:
>>
>> And you're sure you're only using VelocityEngine.evaluate?  Not
>> loading templates through the resource loader?  Or are you doing both?
>>
>> On Mon, Jul 30, 2012 at 2:51 PM, Bradley Wagner
>> <[email protected]> wrote:
>> > Nathan,
>> >
>> > Tokens are referenced by
>> > org.apache.velocity.runtime.parser.node.ASTReference which seem to be
>> > referenced by arrays of org.apache.velocity.runtime.parser.node.Nodes.
>> > Most
>> > of the classes referencing these things are AST classes in the
>> > org.apache.velocity.runtime.parser.node package.
>> >
>> > Here's our properties file:
>> >
>> > runtime.log.logsystem.class =
>> >
>> > org.apache.velocity.runtime.log.Log4JLogChute,org.apache.velocity.runtime.log.AvalonLogSystem
>> >
>> > runtime.log.logsystem.log4j.logger=com.hannonhill.cascade.velocity.VelocityEngine
>> >
>> > runtime.log.error.stacktrace = false
>> > runtime.log.warn.stacktrace = false
>> > runtime.log.info.stacktrace = false
>> > runtime.log.invalid.reference = true
>> >
>> > input.encoding=UTF-8
>> > output.encoding=UTF-8
>> >
>> > directive.foreach.counter.name = velocityCount
>> > directive.foreach.counter.initial.value = 1
>> >
>> > resource.loader = class
>> >
>> > class.resource.loader.description = Velocity Classpath Resource Loader
>> > class.resource.loader.class =
>> > org.apache.velocity.runtime.resource.loader.ClasspathResourceLoader
>> >
>> > velocimacro.permissions.allow.inline.local.scope = true
>> >
>> > Thanks!
>> > Bradley
>> >
>> > On Mon, Jul 30, 2012 at 4:47 PM, Nathan Bubna <[email protected]> wrote:
>> >>
>> >> On Mon, Jul 30, 2012 at 1:30 PM, Bradley Wagner
>> >> <[email protected]> wrote:
>> >> > Thanks for the input.
>> >> >
>> >> > What we're seeing is that Velocity seems to be holding on to a lot
>> >> > of org.apache.velocity.runtime.parser.Token objects (around 5
>> >> > million).
>> >> > We
>> >> > allow people to write arbitrary Velocity templates in our system and
>> >> > are
>> >> > evaluating them with:
>> >> >
>> >> > VelocityEngine.evaluate(Context context, Writer writer, String
>> >> > logTag,
>> >> > Reader reader)
>> >> >
>> >> > I was under the impression that Templates evaluated this way are
>> >> > inherently
>> >> > not cacheable. Is that the case? If that's not true, is there a way
>> >> > to
>> >> > control the cache Velocity is using for these?
>> >>
>> >> me too.  just out of curiosity, what properties are you using for
>> >> configuration?  and can you tell any more about what class is holding
>> >> onto those Tokens?
>> >>
>> >> > On Thu, Jul 19, 2012 at 10:26 AM, Alex Fedotov <[email protected]>
>> >> > wrote:
>> >> >
>> >> >> I think that Velocity has one global hash table for macros from the
>> >> >> *.vm
>> >> >> libraries and that is more or less static for the life time of the
>> >> >> Velocity
>> >> >> engine.
>> >> >>
>> >> >> I wish there there was a mechanism to control the list of the *.vm
>> >> >> files
>> >> >> and their order of lookup for each individual merge (thread). This
>> >> >> would
>> >> >> facilitate macro overloads based on the context.
>> >> >> Unfortunately this feature is not available.
>> >> >>
>> >> >> I think the 1.7 behavior is (more or less):
>> >> >>
>> >> >> When template reference is found (i.e. #parse("x")) it is looked-up
>> >> >> in
>> >> >> the
>> >> >> resource cache and if found there (with all the expiration checks,
>> >> >> etc.)
>> >> >> the parsed AST tree is used.
>> >> >> If not found the template is loaded from the file, actually parsed
>> >> >> and
>> >> >> put
>> >> >> into the cache. During the actual parsing process the macros that
>> >> >> are
>> >> >> defined in the template are put into the macro manager cache which
>> >> >> is
>> >> >> organized as:
>> >> >> "defining template name (name space)" => "macro name" => AST macro
>> >> >> code
>> >> >> The AST is then rendered in the current context running #parse.
>> >> >>
>> >> >> When the time comes to call a macro there is a lookup process which
>> >> >> can
>> >> >> be
>> >> >> influenced by some props, but the most general case is:
>> >> >>
>> >> >> 1. Lookup in the global *.vm files, if found use that.
>> >> >> 2. Lookup in the same "name space" that calls the macro, if found
>> >> >> use
>> >> >> that.
>> >> >> 3. Going back through the "list" of the #parse-d templates lookup in
>> >> >> each
>> >> >> name space on the stack.
>> >> >>
>> >> >> The stack can be actually very long too, for example
>> >> >>
>> >> >> #foreach($templ in [1..5])
>> >> >>   #parse("${templ}.vtl")
>> >> >> #end
>> >> >>
>> >> >> #mymacro()
>> >> >>
>> >> >> The lookup list here would contain:
>> >> >>
>> >> >> 1.vtl, 2.vtl, 3.vtl, 4.vtl, 5.vtl
>> >> >>
>> >> >> This is true even for cases where the name is the same:
>> >> >>
>> >> >> #foreach($item in [1..5])
>> >> >>   #parse('item.vtl')
>> >> >> #end
>> >> >>
>> >> >> The lookup list here would contain:
>> >> >>
>> >> >> item.vtl, item.vtl, item.vtl, item.vtl, item.vtl
>> >> >>
>> >> >> There is no attempt to optimize the lookup list and collapse the
>> >> >> duplicates.
>> >> >>
>> >> >> Unfortunately 1.7 also had some nasty concurrency bugs there that
>> >> >> had
>> >> >> to do
>> >> >> with clearing the name space of all the macros and repopulating it
>> >> >> again on
>> >> >> each parse which did not work at all with multiple threads.
>> >> >> One thread could clear the name space while another was doing a
>> >> >> lookup,
>> >> >> etc.
>> >> >>
>> >> >> I think there was an effort to redesign that part in 2.0, but I have
>> >> >> not
>> >> >> looked at that yet.
>> >> >>
>> >> >> Alex
>> >> >>
>> >> >> On Wed, Jul 18, 2012 at 5:42 PM, Bradley Wagner <
>> >> >> [email protected]> wrote:
>> >> >>
>> >> >> > Hi,
>> >> >> >
>> >> >> > We recently made some changes to our software to use just a single
>> >> >> > VelocityEngine as per recommendations on this group.
>> >> >> >
>> >> >> > We ran into an issue where macros were all of the sudden being
>> >> >> > shared
>> >> >> > across template renders because we had not
>> >> >> > specified: velocimacro.permissions.allow.inline.local.scope =
>> >> >> > true.
>> >> >> > However, we also had not ever turned on caching in our props file
>> >> >> > with: class.resource.loader.cache = true.
>> >> >> >
>> >> >> > Does this mean that macros are cached separately from whatever is
>> >> >> > being
>> >> >> > cached in the class.resource.loader.cache cache? Is there any way
>> >> >> > to
>> >> >> > control that caching or is just using this property the
>> >> >> > way: velocimacro.permissions.allow.inline.local.scope = true
>> >> >> >
>> >> >> > One side effect of our recent changes is that the app seems to
>> >> >> > have
>> >> >> > an
>> >> >> > increased mem footprint. We're not *sure* it can be attributed to
>> >> >> velocity
>> >> >> > but I was trying to see what kinds of things Velocity could be
>> >> >> > hanging on
>> >> >> > to and how much memory they might be taking up.
>> >> >> >
>> >> >> > Thanks!
>> >> >> >
>> >> >>
>> >
>> >
>
>