Re: Message 3 (Re: About Park - points to consider when designing a new programming language)

On Friday 23 June 2006 23:08, Omer Zak wrote:
> On Fri, 2006-06-23 at 21:06 +0300, Shlomi Fish wrote:
> > Hi Omer!
> >
> > I hope you don't mind me CCing this message to shlomif-beta-reviewers.
> > They may be interested in it. I don't see anything particularly personal
> > in it.
>
> Actually, Hackers-IL is a more suitable mailing list for this
> discussion.  It is really about how a programming language could support
> programming in the large.
>

Yes. Replying to Hackers-IL.

> > > ref: http://www.shlomifish.org/park-lisp-fooware/
> > >
> > > I believe that a new non-DSL can justify its existence only if it
> > > addresses the problems of developing a giant application (O(10^7 lines
> > > of source code)).  I raised the subject in Hackers-IL few years ago.
> >
> > Yes, I know. Well, most codebases never reach this kind of volume. I know
> > that Amazon.com has a ~100 Million codebase (which is constantly
> > evolving):
> >
> > http://xrl.us/nowb
> >
> > (and a lot of it is in Perl and in other high-level, succint languages)
> >
> > But this is relatively rare. As MJD points out here:
> >
> > http://perl.plover.com/yak/12views/samples/notes.html
> >
> > (Search for "LISP") he writes about a dozen one-liners a day. So one
> > thing I'd like to make Park suitable for is writing short scripts and
> > programs.
>
> To do quick one-liners, I think that a properly designed IDE would be
> better than a new language.  I have in mind an IDE which integrates with
> a powerful search engine:
> You type in concepts as keywords.
> The search engine searches through library documentation and cookbooks.
> You get, as results, links to relevant libraries (from all programming
> languages) and relevant coding cookbook entries.
>

Well, I don't write my one-liners using an IDE. I write them in a Konsole 
(KDE's Terminal app) window with Bash. Sometimes I fire up gvim if the code 
becomes too hairy and doesn't do what I want, at which point it often becomes 
more than one line.

I sometimes also consult various man pages or perldoc's (perldoc is perl's 
online help system, similar to man). Usually I don't need Google or a 
different search engine.

I believe most one liners assume the programmer has a ready, instant, 
knowledge of most of the core functions and concepts he needs to write stuff. 
Like I told people a million times already, when I started seriously working 
on UNIX, I wrote most of my shell scripts in Perl. (And I know of several 
other programmers who got introduced to UNIX shell scripting thru Perl or 
Python or whatever.). It simply that I didn't had enough ready knowledge of 
the shells that were available on my host UNIXes to write it using it.

My intention in Park is to have an actually useful Lisp dialect that people 
could use in place of Perl, Python, etc.

> > My expectation from languages is that unless it's a truly estoric
> > language (like Intercal) it can scale well. Hell, even PHP which has some
> > serious maintainability problems (at least IMO) can scale well to large
> > or very large codebases.
>
> A language needs to be both scalable and maintainable.  If PHP has
> serious maintainability problems, then it is not suitable as main
> language for O(10^7) projects.

Maybe maintainability is not the right word. PHP code if written properly is 
maintainable. (and if it isn't written properly, but still works, it can 
usually be refactored to be maintainable) What I meant is that in PHP it is 
easier to introduce various bugs if you're not careful. It's not impossible 
to write perfectly good code in PHP, just requires more discipline. 

( I don't mean to spread FUD on PHP here - this is just my opinion and my 
impression. I can prove it by various means.)

I believe PHP would be suitable for writing > n*10M LOCs codebases. But you'll 
probably need to have very good and discplined programmers for achieving 
this. (But this goes without saying for any other language you'd like to 
maintain such codebases in).

>
> > > - Support for mixed-language software development.
> >
> > Do you mean mixed-programming language or mixed human-language (I.e:
> > Unicode, etc.)
>
> I mean mixed-programming languages, as in Inline::Guile, which mixes
> Scheme into Perl (when it works).
>

OK, I see.

Well, if Park would be C-hosted (which I'd like to - I started writing a 
section about it - it's on: http://xrl.us/no6o ), then it can somehow have 
Inline::Perl5, Inline::Python, and Inline::Perl5::Inline::*. :-)

If Park would be implemented above Parrot (or a different portable VM 
like .NET), then it should be able to interface with other languages written 
for this virtual machine. 

There is also the concept of a Common Request Broker, Web Services, YAML, 
POE/Twisted/etc., SWIG, and plain ol' TCP/IP.

These are mostly an implementation detail.

Note that this is a very hard problem, and I do not claim to have a perfect 
(or even too good) solution for it.

> > > - Multi-versioning of support code (it would be cool to be able to link
> > > one part of a giant application with version 1.5 of a library, and
> > > another part of the same application with version 1.6.3 of the same
> > > library).
> >
> > That's interesting. I don't think Perl supports it, but Python might if I
> > understood its module system correctly. Of course, we also have to take
> > into consideration a Perl "Safe;" like protected environment module which
> > is often used for IRC eval bots, embedded Perl etc..
> >
> > The Pythoneers tried implementing something like that in Python, but they
> > kept discovering security problems, and ended up abandoning by concluding
> > it was a difficult problem.
> >
> > One possible solution would be to set a different root namespace or do a
> > namespace assignment. If each namespace (and sub-namespace) is a
> > first-order object (which was what I planned for Rindolf) then we can mix
> > and match them.
>
> This is more of distribution (in the sense of Gentoo, Ubuntu or
> Mandrake) issue.  Different programs can link to different libraries,
> and be glued together using a script file to form a single gigantic
> application.
>
> For example, consider what is needed to be able to keep both versions
> 1.4 and 1.9 of automake in a Debian system.
>

I see.

I'd like to think about it a bit. But thanks for raising this problem.

> > > - Testability (probably supported by aspect orientation).
> >
> > What do you mean by "Testability"? I'm waiting for an answer.
>
> Let's keep it open-ended question here.  Additional brains are invited
> to discuss implications of testability upon language and IDE design.
>
> An hypothetical language, which does not allow you to run separately a
> module from a big application, with reasonable amount of scaffolding -
> does not support testability.

I see. Well, the way I see it, without the Law of Demeter, and general 
modularity concerns, you can always design such a problematic program in any 
language no matter how suitable. I'm not aware of any such problems in Perl 5 
or Python, because they provide the means for easily mocking (or scaffolding  
as you call it) objects and modules. (Albeit from my impression some of the 
generic MockObject APIs are not worth the extra trouble of using).

I heard that .NET had a problem in creating a generic mock object, but I'd 
rather leave it to other people here. 

>
> > > - Debugability (a language which can help detect heisenbugs can be a
> > > winner).
> >
> > Funny that you mention it. I recently encountered a heisenbug in "perl
> > -d" and one in the entire debugger interface. Now the default perl
> > debugger is very hairy and error-prone, and I could only reproduce this
> > heisenbug occasionally, and not too many others could reproduce it.
> >
> > One suggestion for handling heisenbugs that has been raised is a
> > reversible debugger. See for example:
> >
> > http://use.perl.org/article.pl?sid=06/06/22/1210201
> >
> > Do you have any other suggestions?
>
> Not really.  But can try to use theory (http://lambda-the-ultimate.org/)
> to specify a language, in which it is possible to prove that certain
> types of heisenbugs cannot occur.  For example, require explicit
> declaration of variables, which are shared among threads, so that the
> compiler (or interpreter) can check, at compile time, that accesses to
> those variables are properly locked.
>

Do you include static run-time checking there? (Like the Stanford checker, 
etc.)

I'd rather not go too much into the direction of a language in which bugs can 
be proven not to exist for fear of the following quote:

<<<
"UNIX was not designed to stop its users from doing stupid things, as that 
would also stop them from doing clever things." – Doug Gwyn
>>>

So I'm not sure whether such a feature would prevent doing smart things like 
doing self-writing code, or even generating functions on the fly, or 
dynamically changing the class inheritance graph.

> > > - Several views of the source code (LabVIEW is suited to certain
> > > personality types, among whom I am not).  Syntax coloring.  Finding all
> > > references to a concept [variable, procedure, object] without being
> > > confused by having its name reused elsewhere; or having several names
> > > aliased to it.
> >
> > That's pretty much impossible to do all the time in Lisp (if I understand
> > you correctly). While someone can implement a syntax coloring editor
> > which will work if your code does not use macros or fancy stuff like that
> > too much.
>
> Can you design Park to support this, even when macros are used?
> It may be a good idea to design an IDE, which would allow a developer to
> have a view of code, which could be generated by a LISP macro, as the
> macro is being developed.
>
> I think that it is more of IDE issue than language issue, however.
>

I also think Alan Turing has something to say about it too:

http://en.wikipedia.org/wiki/Halting_problem

If we generalise the Halting problem than we can say a program to conclusively 
analyse a different computer program would be impossible to write. I don't 
see how I can get an IDE to analyse a macro and determine how it does what 
it's supposed to do. The best way would probably be to document the behaviour 
of the macros in the program a la doxygen or javadoc. But this will require 
an extra work from the programmer.

Note that my intention is that the core Park language and support libraries 
will be rich and powerful so that people will often not have to resort to use 
macros to redefine the language. This means there would be some kind 
of "Perly Park" subset of Park where you write programs like you would in 
Perl.

<<<<<<
<adeht> rindolf: a nice way of writing CL code is to imagine your dream 
language for expressing that particular problem, and then realizing it :)
<rindolf> You probably wouldn't use a regex in Python either.
<rindolf> adeht: I see.
<rindolf> adeht: I'm not going to implement Perl in Common Lisp. :-D
>>>>>>
	--- #lisp-il today.

BTW, tomorrow I intend to write my classic Graham's Function program ( 
http://www.shlomifish.org/Vipe/lecture/Perl/Graham-Function/ ) in Smalltalk 
and/or Common Lisp. I already have versions in Perl 5 (which is the 
original), in O'Caml and in Perl 6. Does anyone volunteer to write it in 
COBOL or Intercal? ;-)

Regards,

	Shlomi Fish

---------------------------------------------------------------------
Shlomi Fish      shlomif-ik1l9ssToec+JF/[email protected]
Homepage:        http://www.shlomifish.org/

95% of the programmers consider 95% of the code they did not write, in the
bottom 5%.


------------------------ Yahoo! Groups Sponsor --------------------~--> 
Yahoo! Groups gets a make over. See the new email design.
http://us.click.yahoo.com/XISQkA/lOaOAA/yQLSAA/saFolB/TM
--------------------------------------------------------------------~-> 

 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/hackers-il/

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/