Re: Pike in the News

On Mon, 23 May 2022 at 05:28, <[email protected]> wrote:
> b) Where's the source code?
>
> Here, I actually sort of agree that there's a discoverability problem.

Yeah, more than sort-of for me; I can't really just link someone to
the landing page and assume that they'll be able to find everything.
Not sure that it counts as low-hanging fruit, but it's certainly fruit
if someone wants to put in some effort.

> Some things I found really interesting:
>
> a) Concurrency, ie the GIL
>
> What are the ways for improving concurrency, and how big of a problem is
> it? I'm sure it can be a real problem, just wondering if anyone has any
> experience probing what they are? Aside from unlocking in c-language
> modules, seems like the solutions for single-process improvements are
> pretty invasive. Is there an opportunity to provide some tooling to
> allow tasks to be spread across a number of pike processes in order to
> get around the GIL? I did some work with zeromq, which positions itself
> as a way to improve concurrency without the risks of multi-threading.
> Perhaps some sort of pike clustering module/etc could provide
> performance improvements?

Removing the GIL? I think that that's likely to be a major issue,
given how hard it is to do it in CPython. Or rather, given that every
attempt to remove the GIL from CPython has resulted in enough of a
performance hit for single-core work that it's simply not worth the
cost.

There was some work being done a while ago on memory arenas, where
different threads could have their own address spaces, and thus their
own independent locks; the GIL would then only be needed for guarding
access to global/shared objects. Did that go anywhere?

Another thing that I'd love to see, in terms of concurrency, is some
enhancements to asynchronous I/O. Pike currently *far* outstrips
Python in the convenience of going to a back-end event loop (just
return -1 from main(), where in Python you have to play around with
the asyncio module); but on the flip side, Python has actual native
support for async functions, and Pike only recently gained the
underlying structure of continue functions. If I could just dream for
a bit, here's what I'd love to see in Pike:

1) Asynchronous functions are declared to return their "ultimate"
return value, but can always yield a Concurrent.Future without needing
to declare this. Currently, I use declarations like this:

continue mapping|Concurrent.Future some_function(...) { ... }

which is just noise - in reality, it will only ever "return ([...])"
and only ever "yield some_future".

2) A built-in function that tells you whether something is a continue
function, a continuation function, or a regular function. I currently
use this hack to recognize when it's the special function returned
from a continue function, but there's no way to know about the other
two:

int(0..1) is_genstate(mixed x) {return functionp(x) &&
has_value(sprintf("%O", x), "\\u0000");}

Even better if it can indicate that a continuation function has
finished. (Or maybe that's a separate function.) That would, with no
ambiguity, tell you whether the last thing it did was "yield" or
"return". Currently, the only way to tell is to call the function
again and see if you get back a 0, which is not guaranteed anyway,
since it could yield 0.

3) Minor, but an easier way to raise an exception into a continue
function. Currently, the way to do that is:

resp = gen(0) {throw(err);};

and I don't think that's very obvious. A direct way to throw something
into the continue function like "raise_in(gen, err)" would make that
clearer, and hopefully more efficient (the convenient shorthand that I
used there, since it's implicit-lambda, is probably quite
inefficient).

4) A function (probably in the Concurrent module) that takes all of
the above and wraps up the logic of repeatedly calling a generator
function with the results of a promise. The way I've implemented it is
the spawn_task class here:

https://github.com/Rosuav/StilleBot/blob/master/globals.pike#L150

(Line number might be wrong in the future, as that's a live project
and does see changes on a regular basis.)

With all of those features, the language itself would have great
concurrency support, and you could simply call
Concurrent.spawn(some_func()) to set an asynchronous operation going.

> b) Embedded/embedding
>
> 2 possible meanings here;
>
> a) embedding a pike interpreter in another program, which is possible
> and I've shipped several projects that use this. Don't think this would
> be of interest to most people, but could be made a lot nicer with just a
> little effort.
>
> b) running some variant of a pike interpreter on microcontroller scale
> devices (Ala micropython, etc). Potentially huge audience, but seems
> like the pike interpreter isn't really suited to being scaled down to
> fit/run: a pretty hefty controller would be necessary and without a real
> os and limited ram, you'd probably give up a lot of the things that pike
> excels at: networking, file I/0, etc. Could be really interesting to see
> what the smallest device you could run a real OS and pike on and tailor
> that to embedded development. Ie, can you do it with a device you can
> get for 10 currency units reliably? I'd certainly be willing to put some
> cycles toward that if there's interest.

That would be interesting, but I'm not the person to talk to. The
nearest I've ever come to working with a microcontroller is a
Raspberry Pi, and that runs a full Linux with full support, and that's
not a ten-currency-unit board.

> c) Runtimes: JVM, WASM, etc
>
> Probably not ever going to happen, but wouldn't it be nice if it did?

If I had more hours in the day, I would love to look into WASM as a
target, as it'd let me use Pike front end and back end. But so long as
we're limited to the usual twenty-four, it's not likely to happen from
me.

> I feel like there is some low hanging fruit that could at the very least
> make life easier for the random person who ends up on the Pike website:
>
> a) fixing broken links
> b) making the repository easier to find

Agreed and agreed.

> c) how about a faq? all of these things come up from time to time, and
> it seems like they ought to be written down and easy to find. I know
> there are several documents like this out there, but perhaps a little
> effort to collect and present them would be useful?

Perhaps. There's always a FAQ somewhere, but the problem is that it
seldom has the question someone actually wants answered right at this
moment. I'd rather the mailing list were easier for people to find -
it's always so quiet here, I certainly wouldn't object to seeing some
new folks with questions!

> d) Mailing list archive. I have the data, and a search engine with the
> content loaded, but a browsable archive would be really nice. I could
> possibly help with that if there's interest.

Yes please. That would be great. Permalinks would also allow them to
be shared around.

> d) Either hosting or prominently linking to tools built with/for pike
> that could allow someone to use pike for something useful (thinking
> editor support, fcgi, etc). I've got a lot of this information already,

That'd be cool too.

I'm happy to contribute to some of this. Obviously hours in the day
are, as mentioned, finite, but I'd be happy to plunge some of them
into these sorts of improvements.

ChrisA