Re: Continue Eclipse Freemarker DLTK?

Tuesday, July 30, 2013, 12:20:31 AM, Angelo zerr wrote:

>
>
>
> 2013/7/29 Daniel Dekany <[email protected]>
> Monday, July 29, 2013, 3:57:38 PM, Angelo zerr wrote:
>
>> Hi Daniel,
>>
> [snip]
>>> OK, so I think branch should be named with *-debug or* -tools becaus it's not linked to dltk.
>>
>> (Sure, it could be "-tools". It's linked to the DLTK *development*, as
>> it will be only merged back when the DLTK plugin is released.)
>
> Ok, is it possible for you to create this branch if you start something aboute the debugger?
> I'm not a big expert with Git.

Sure.

>>> fix bug with debug unsert ftl
>>
>> Which bug is that?
>
> It was a long time, but if I remember when you write a.ftl which
> contains <#include b.ftl > directive,and if you set a a breakpoint
> in the b.ftl and you debug the a.ftl, debugger should open b.ftl and
> I needed filename from the breakpoint to retrieve it.

Sounds strange... will see when we run into it.

>> OK, if I remember I have modified the DebugBreak to set a breakpint
>> in the end of the #if (as you have explained).
>> You can remove that if you wish.

Since end-tag breakpoints won't work correctly like this, and I don't
think we should spend time implementing it now, I will.

>>> (Breakpoint should stores too filename (the full path of the template)).
>>
>> You mean, the template name (TemplateLoader-relative path) is not
>> enough, you need an absolute local filename associated with the
>> breakpoint?
>
> Yes it that, but I could retry again to give you more info, if you wish.

Yes, I think we need to discuss this a bit. I will tell what do I
think about this topic, and then you could tell me what do I miss,
etc.

When you add a breakpoint in Eclipse, the plugin must be able to deduce
the template name of the template File (by File I mean real local
file, like an absolute java.io.File), since Template-s are registered
with the template name, not with a File path (especially as templates
aren't always stored on the file system). To be able to deduce the
template name from the File, there need to be a project-level plugin
setting where you can specify the *set* of template root directories.
If you try to set a breakpoint in a template that's not inside one of
those directories, that's an error, because then the plugin can't
deduce the template name by removing the containing template root
directory from the beginning of the path of the File. When the
breakpoint is triggered, the pluign will only get back the template
name, not the template File, so now the plugin has to deduce the File
from the template name. But since you can have multiple template root
directories, the conversion in that direction can lead to an ambiguity
(like, is "index.ftl" /WEB-INF/template/index.ftl or
/WEB-INF/classes/com/example/templates/index.ftl, if both exists). So
you need the File that points to the actual source file. Is that the
goal maybe? Because then attaching the File to the breakpoint won't
work reliably either, as when you tell the DebuggerService to add a
breakpoint, it will add it to *all* registered templates with that
name. So, we have a problem of the fundamental kind here, AFAIS.

To solve this, I believe the TemplateLoader need to provide some more
meta-information that then will be attached to the Template. That
facility will be useful in many cases, but here, the usage would be
attaching the File to the Template, when the TemplateLoader loads
template from File-s (and only the TemplateLoader knows that it does).
Then that information is available for template registration etc. So
now the Breakpoint could carry a template name (required) and a File
(optional). This will be still tricky for the Eclipse plugin, because
that File will be the one from the build directory (bin directory) of
Eclipse, not the source File where you put the breakpoint in the
editor. I wonder if you can query in Eclipse what the output File for
a source File will be, because a far as the DebuggerService is
concerned, you have to add the breakpoint with the output File path,
not the source File path. So I'm not sure if it's workable, or just
will be the source of odd bugs where breakpoints don't trigger.
Actually, we can keep using the template name as the sole identifier
for registering and adding preakpoints, but when the breakpoint is
triggered, you have more hope to open the right source file if you
have the path of the built File...

>> Will that file name by set by the plugin, and will only be
>> used by the plugin? I see no problem with that then, although I wonder
>> if it would be useful to simply allow associating an arbitrary
>> serializable object with a Breakpoint. Then that object could be a
>> java.io.File, or a JavaBean that stores more info that's useful for
>> the IDE to come back when the suspension event occurs.
>
> Perhaps it's a good idea to have a "data" object attached to the breakpoint?

That's what I meant above.

>>  * syntax errors (add start/end offset info of the token)
>
>> Tokens do have start/end line and column. Is it buggy or something?
>> AST nodes (TemplateObject-s) also have start/end location, except when
>> a constant expressions is substituted with a plain Java objects (like
>> String).
>
> For the moment I have line and column information. But it seems
> that I need start and end offset to highlight the error. 

Then maybe you mean you only have that in ParseException? If so, I
believe I can fix that real quick.

>> Well, it can work, but I would put some questions:
>>
>> - What are the advantatges of the SPI approach over the approach that
>>   only uses properties (the one I have proposed)?
>
> It's a "standard" mean to retrieve some service with Java5/Java6. 

Only, you use standard solution instead of your in-house one when it
reasonably well fits the problem to solve.

>> - If I provide "freemarker.debug.password", that will make most
>>   DebuggerService "accessible", how does the user contol which
>>   implementation will be choosen (or why is it not a problem)?
> OK, we could add a key in the property like -DdebuggerService=rmi to select the rmi debugger.

So then, first you have to figure out what the implementation name is
(like "rmi"), and then the SPI API will scan through the META-INF-s to
give us an enumeration of all DebuggerServiceProviders it has found
(and how reliably that works, I don't know... everything related to
class-loaders used to be a mess), but we really just ask them "is it
your name?", and only one of them will say "yes" (hopefully, or else
we have a problem). All this time you knew which class you wanted, but
run a circle to get there. I mean, isn't it simpler just tell in a
property that "I want This class", then try to load that class (no
META-INF scanning etc)?

I really don't want to be difficult here, but I only see added
complexity. I think that the point of SPI is automatic discovery (or
what else?). But here that's exactly what doesn't work. You can't just
drop your DebuggerService implementation into the class path here, you
will have to select(!) and configure it with system properties...

>> - Isn't it problematic that we are using the class-loader of FreeMarker
>>   to find the SPI implementations? If FreeMarker comes from a more
>>   generic directory than the SPI implementation, it won't be found, I
>>   think. Then we won't even give an error, but fall back to
>>   RMIDebuggerService (unlike in the properties-only solution).
> The SPI implementations should be stores in the same classpath than
> Freemarker.

FM has a class loading mechanism in FreeMarker in ClassUtil that first
tries to load from the context class-loader of the current Thread. I'm
not sure if it's terrible or a good thing... it's used pretty much
everywhere in FM to load classes though.

> For OSGi context, you must create a fragment linked to the
> Freemarker which contains the SPI implementation.

With OSGi you can do that (tell what classes see which classes), but
not much elsewhere. Also, since FreeMarker implements a dynamic
language, it has "DynamicImport-Package: *", and I suppose that takes
away the explicit control.

> We use SPI with XDocReport which works in OSGi context to register
> document kind service (docx, odt,) converter type (docx->pdf,
> docx->html, etc) and it works great.

SPI is surely good fit for stuff like converters, because you
*naturally* have the criteria to chose the one you need from the list
of SPI implementations. I don't question that.

>> - Is it setting different DebuggerService-s per Configuration useful?
>>   Like, does it have a known real-world use-case?
>
> For IDE, its useful. In my case I can debug 2 ftl. Each ftl are
> linked to a Configuration. So each Configuration must have a dbgp debugger.

Wouldn't setting DBGp debugger globally work in that case?

(The reason I have reservations regarding the usefulness of
per-Configuration debugger service is that normally you can't
reasonably easily fiddle with the Configuration(s!) used inside the
application you want to debug. You had to find the related
configuration file (if there's one at all) and pray that it uses
FreeMarker's string-based "settings" API so you can specify
"debugger_service". The even bigger problem is that then you have
modified a file that's part of the product, rather than of the
development environment. That's rather messy.)

>> If you like it, I will clean my code and create a pull request.
>>
>> As of the clean-up, DebuggerServiceRegistry strikes me as needless,
>> unless it will actually become a registry in the future (in which
>> case, what will ot do?). Now all it does is getting a "default"
>> ServiceRegistry found via the SPI API. That could be done in
>> DebuggerService.getInstance().
>
> In my case DebuggerService is an interface and not a class, that's
> why I have created  DebuggerServiceRegistry

OK...

> My big need is to use my dbgp debugger service.

Why's DBGp important BTW? I'm just asking as I don't know much about
it... (I suppose Eclipse doesn't require it, since the Liferay IDE
doesn't use it either.)

> So it should be cool if you provide this API (my pull request and
> commits was just to give you some ideas, ut don't hesitate to do
> other thing). I will adapt my DLTK code with your API.

I think I will just create 2.3-gae-tools and pull both your changes
into it, and then you can continue work there. Most of the work will
be DBGp and DLTK/plugin related I suppose, so we can just settle the
other issues in parallel, without you feeling blocked. I will do this
today as time allows.

-- 
Thanks,
 Daniel Dekany


------------------------------------------------------------------------------
Get your SQL database under version control now!
Version control is standard for application code, but databases havent 
caught up. So what steps can you take to put your SQL databases under 
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk