Re: Language Infrastructure

JMB,

I think you're right that there isn't much in the way of tutorials. Most of
what I've figured out was by just digging around in the source code. Some
of that, if you look, is easy, but others systems are hard to sort through.
This source is pretty helpful:

https://github.com/GeertjanWielenga/Python4NetBeans

It's source, not a tutorial, but it uses most of the new systems.

In terms of the systems I found the most confusing getting up and running,
here they are:

   1. *Find Usages*: Computing these is easy in a compiler, but hooking
   everything up for the user interface didn't appear to be. NetBeans has a
   system for it, but how to adapt it I didn't have time to figure out.
   2. *Profiler*: So far as I can tell, there just isn't an easy way to
   integrate into the existing profiling system. I'm probably wrong, but I
   couldn't figure out a way to do it that didn't involve some nasty hacks I
   didn't want to do.
   3. *Refactoring*: The auto-rename system is easy to plug into (and
   awesome), but the bigger user interface system for refactoring in
   Java/other languages is harder. When I look at the implementation
   internally in NB source, there's a lot of copy pasting of big classes going
   on. In contrast, InstantRename is built into DefaultLanguageConfig and is
   very easy to use because of it.
   4. *Debugging*: While we have debugging working and have for a while,
   the way you hook it into NetBeans is incredibly complicated and hard to
   understand. There is also, so far as I can tell, no documentation.
   5. *Indexing*: I played around with this system for hours, but I could
   never get it to do anything interesting and am not even really sure how it
   is intended to work. As such, I end up doing all my indexing elsewhere in
   our projects.
   6. *Hints*: The hints system is probably one of the easier ones to get
   working in this list, but there is almost no documentation and the names of
   the methods in the API aren't very descriptive, so I was basically guessing
   at first how you plugged in. If I recall, the only tutorial I found was one
   on adding a hint to Java, but that doesn't help you much for custom
   languages.


There are lots of others things to discuss. Doing this kind of stuff in
modern NetBeans is an order of magnitude easier than it used to be.
Speaking personally though, I think the best change the team could make
would be to add Find Usages, the Profiler, Refactoring beyond instant
rename, and debugging (e.g., setting watches, variables, trees) into
DefaultLanguageConfig by default, with appropriate interfaces for hooking
up the UI. That change alone would make integrating these systems much
easier for developers --- or at least me, I don't know.

That's my $0.02,

Stefik


On Wed, May 4, 2016 at 2:25 AM, Jean-Marc Borer <[email protected]> wrote:

> I was a little bit confuse about this "new" language structure, because it
> is not documented at all. Nothing in the java docs. I was finally able to
> find this http://wiki.netbeans.org/GsfToParsingAndIndexingApiMigration
>
> Hope it helps others.
>
> Any news about the updated language support tutorials?
>
> Cheers,
>
> JMB
>
> On Tue, Oct 27, 2015 at 8:00 PM, Eirik Bakke <[email protected]> wrote:
>
>> I've added an MIT license header to it. That should let you redistribute
>> under any of the other licenses.
>>
>> https://gist.github.com/eirikbakke/fdda0e6b43ed1b2d54eb
>>
>> -- Eirik
>>
>> From: Andreas Stefik <[email protected]>
>> Reply-To: "[email protected]" <[email protected]>
>> Date: Tuesday, October 27, 2015 at 2:16 PM
>>
>> To: "[email protected]" <[email protected]>
>> Subject: [platform-dev] Re: Language Infrastructure
>>
>> Eirik,
>>
>> I have plugged it in to the Quorum project. So far as I can tell,
>> everything looks good. Over the next few days, I'll deploy a beta version
>> of Quorum in NetBeans to my team here and see if anyone else notices any
>> problems, as I also like the idea of having a standard version that folks
>> can use in the tutorials. Since it looks visually pretty solid, I'm fine
>> with being guinea pigs.
>>
>> Just because I know people will ask, and as such it should be included in
>> the file, what license do you "officially" want your code under? NetBeans
>> is CDDL, Quorum is BSD, but MIT is common. All else being equal, I would
>> recommend BSD because it's pretty open that's also what Antlr is under.
>>
>> Let me know,
>>
>> Stefik
>>
>>
>>
>> On Tue, Oct 27, 2015 at 10:23 AM, Eirik Bakke <[email protected]> wrote:
>>
>>> Yes, feel free (you or anyone else) to use LexerInputCharStream for any
>>> purpose. No warranties, express or implied etc. etc. :-)
>>>
>>> If you try it, let me know if you observe any bugs. LexerInputCharStream
>>> has worked fine in my platform app for the last 11 months, though I only
>>> use it in the context of a single-line Excel-style formula bar (with an
>>> ANTLR4-defined formula language). Having it work fine in two different
>>> applications would be a nice sign that it works properly.
>>>
>>> > Also, just because I don't remember if it's used anywhere, why does
>>> size() throw an unsupported op? Looking at ours, we are clearly returning
>>> garbage as well, so I'm guessing that's not called, but it's been a while
>>> since I've put much thought into the antlr 4 wrapper in quorum.
>>>
>>> LexerInputCharStream being an ANTLR CharStream implementation over
>>> NetBeans' LexerInput, LexerInputCharStream.size()
>>> throws UnsupportedOperationException because the underlying LexerInput
>>> exposes no concept of an input size. It could in theory be found by reading
>>> the entire stream and counting the characters until EOF, but that would be
>>> ugly. Luckily, the size() Javadoc
>>> <http://www.antlr.org/api/Java/org/antlr/v4/runtime/IntStream.html#size--> says
>>> UnsupportedOperationException can be thrown "if the size of the stream is
>>> unknown", which seems like the right thing to do here.
>>>
>>> -- Eirik
>>>
>>> From: Andreas Stefik <[email protected]>
>>> Reply-To: "[email protected]" <[email protected]>
>>> Date: Tuesday, October 27, 2015 at 12:32 PM
>>>
>>> To: "[email protected]" <[email protected]>
>>> Subject: [platform-dev] Re: Language Infrastructure
>>>
>>> I'm tempted to switch to this myself. We're under BSD. Would you allow
>>> us to use it, barring credit is given where credit is due (e.g., same
>>> implementation with the name of the class changed)? Our works, but yours is
>>> great and really clean.
>>>
>>> Also, just because I don't remember if it's used anywhere, why does
>>> size() throw an unsupported op? Looking at ours, we are clearly returning
>>> garbage as well, so I'm guessing that's not called, but it's been a while
>>> since I've put much thought into the antlr 4 wrapper in quorum.
>>>
>>> Stefik
>>>
>>> On Tue, Oct 27, 2015 at 9:07 AM, Eirik Bakke <[email protected]> wrote:
>>>
>>>> Here's a version of the CharStream implementations with the
>>>> non-standard dependencies removed (replaced by Parameters.notNull as Tim
>>>> proposed):
>>>>
>>>> https://gist.github.com/eirikbakke/fdda0e6b43ed1b2d54eb
>>>>
>>>> -- Eirik
>>>>
>>>> From: Andreas Stefik <[email protected]>
>>>> Reply-To: "[email protected]" <[email protected]>
>>>> Date: Tuesday, October 27, 2015 at 2:00 AM
>>>>
>>>> To: "[email protected]" <[email protected]>
>>>> Subject: [platform-dev] Re: Language Infrastructure
>>>>
>>>> That's what I was thinking too, although I've never used that
>>>> particular API.
>>>>
>>>> Stefik
>>>>
>>>> On Mon, Oct 26, 2015 at 10:50 PM, Tim Boudreau <[email protected]>
>>>> wrote:
>>>>
>>>>> AFAIK, Preconditions is just a set of checks - equivalent to
>>>>> NetBeans'
>>>>> Parameters.notNull("foo", foo);
>>>>>
>>>>> i.e. nothing terribly exciting that couldn't be handled other ways.
>>>>>
>>>>> -Tim
>>>>>
>>>>> On Tue, Oct 27, 2015 at 1:47 AM, Andreas Stefik <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> This is awesome, Eirik. I have to admit, like we say in a comment
>>>>>> somewhere, we just basically grabbed it off stack overflow for that piece.
>>>>>> We've tested it and it seems to work, but your implementation looks very
>>>>>> thoughtful. Nice work!
>>>>>>
>>>>>> The only potential downside I see is that it does have this one
>>>>>> external dependency:
>>>>>>
>>>>>> com.google.common.base.Preconditions
>>>>>>
>>>>>> So far as I can tell, that's not a standard "out of the box"
>>>>>> dependency you get in theNetBeans platform, which would mean folks would
>>>>>> need to load in an external module. That's not a big deal, but it's an
>>>>>> extra step and many folks may not need it in the general case. Could we do
>>>>>> some kind of hybrid for the tutorial, with that dependency removed, but
>>>>>> largely Eirik's implementation, as opposed to the stack overflow one (which
>>>>>> we use)?
>>>>>>
>>>>>> Just a thought.
>>>>>>
>>>>>> Stefik
>>>>>>
>>>>>>
>>>>>> On Mon, Oct 26, 2015 at 7:02 PM, geertjan wielenga <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>>
>>>>>>> On 27-10-2015 2:48, Eirik Bakke wrote:
>>>>>>>
>>>>>>> Great to hear that ANTLR-related platform documentation is being
>>>>>>> updated!
>>>>>>>
>>>>>>> As Andreas mentioned, the tutorial at
>>>>>>> http://wiki.netbeans.org/New_Language_Support_Tutorial_Antlr is a
>>>>>>> bit outdated. In particular, the supplied AntlrCharStream file works only
>>>>>>> on ANTLR version 3. I spent a fair amount of time a while back writing a
>>>>>>> version updated for ANTLR 4, which you are welcome to include in the
>>>>>>> updated documentation:
>>>>>>>
>>>>>>> https://gist.github.com/eirikbakke/51cf4c9375880acd4741
>>>>>>>
>>>>>>> This corresponds to Andreas' AntlrCharStream
>>>>>>> <https://bitbucket.org/stefika/quorum-language/src/99ab52b508298cebb5810e7ebcfc2883ada7251f/IDE/Sodbeans/Quorum/src/org/quorum/language/AntlrCharStream.java?at=hints&fileviewer=file-view-default> in
>>>>>>> the Quorum repo. Either could be used--in any case the old ANTLR tutorial
>>>>>>> would benefit from being updated to include an ANTLR 4-compatible
>>>>>>> CharStream implementation.
>>>>>>>
>>>>>>>
>>>>>>> Briliiant.
>>>>>>>
>>>>>>> Gj
>>>>>>>
>>>>>>> -- Eirik
>>>>>>>
>>>>>>> From: Andreas Stefik <[email protected]>
>>>>>>> Reply-To: "[email protected]" <[email protected]>
>>>>>>> Date: Monday, October 26, 2015 at 5:52 PM
>>>>>>> To: "[email protected]" <[email protected]>
>>>>>>> Subject: [platform-dev] Re: Language Infrastructure
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Oct 26, 2015 at 2:42 PM, geertjan wielenga <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Yes, Andreas, please do this -- I will pick up your outline and
>>>>>>>> expand it to a fullblown tutorial.
>>>>>>>>
>>>>>>>> And, in terms of ANTLR, have you found this tutorial?
>>>>>>>> http://wiki.netbeans.org/New_Language_Support_Tutorial_Antlr
>>>>>>>>
>>>>>>>> Gj
>>>>>>>>
>>>>>>>
>>>>>>> A lot of that tutorial was actually stolen from Quorum source and I
>>>>>>> recall writing pieces of it (uncredited, but that's fine because I don't
>>>>>>> really care). :)
>>>>>>>
>>>>>>> But, that's for Antlr 3 and uses the old language infrastructure,
>>>>>>> which is much more difficult to understand. It won't work with the latest
>>>>>>> versions (All star was a major breakthrough in computer science, from my
>>>>>>> perspective, and changed many things, making the upgrade to 4 massively
>>>>>>> worth it). In other words, I wouldn't take that code and use it. It's
>>>>>>> possible you can hack it into working, but that's not generally how things
>>>>>>> are done today, to my knowledge.
>>>>>>>
>>>>>>> Stefik
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> http://timboudreau.com
>>>>>
>>>>
>>>>
>>>
>>
>