Re: Announcing C::Blocks, a different way to interface Perl and C code
[email protected] Sat, 24 May 2014 13:34:07 +1000
| Newsgroups | perl.xs,perl.inline |
|---|---|
| Message-ID | <E99D8E8F78FC4741AEB930A7DDBD5077@OwnerPC311012> |
------=_NextPart_000_0076_01CF7754.D687B590 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sounds pretty cool, David. A plodder like me is probably just going to stick with Inline, but it = would be great to see C::Blocks takes off. And it has the attractiveness = to do so. (I like the way you can so easily just plonk the C code right in there = amongst the perl code .... makes Inline look not-so-inline :-) Cheers, Rob From: David Mertens=20 Sent: Friday, May 23, 2014 10:35 PM To: [email protected] ; Perl Inline Mail List=20 Subject: Announcing C::Blocks, a different way to interface Perl and C = code Hey everyone, tl;dr: C::Blocks is a new TinyCC-based module, presently only available = on github. (1) It jit-compiles blocks of C code, building and inserting = OPs into the Perl OP tree, making invocation of C code essentially free. = (2) It will allow different blocks of C code to share function and = struct declarations, thus removing the need to always recompile perl.h, = an otherwise major cost of jit-compiling C code that can interface with = Perl and Perl data structures. I am currently seeking help and encouragement to squash the segfaults = that currently prevent the completion of the second feature. :-) ---- I like Perl, but I like C, too. I would like to be able to write and = call C code from Perl in about as painless a way as possible. Inline::C = is nice, as are XS::TCC and C::TinyCompiler, but we can do better. = C::Blocks is my attempt to do better. Pain point 1. C code should be a first class citizen. With XS::TCC and = C::TinyCompiler, you pass your code to the compiler via a string. With = Inline::C, you either place your code at the bottom of your script in a = __DATA__ section, or you enclose it in a string. Steffen's module is = probably the most transparent in this sense. Still, working with an = interface that requires me to compile a string to get my product feels = the same as compiling a regex from a string. This is Perl! We can do = better! C::Blocks does better by using a keyword parser hook. Blocks of code = that you want executed are called like so: print "Before cblock\n"; cblock { printf("In cblock\n"); } print "After cblock\n"; If stdio.h is included, you get the output Before cblock In cblock After cblock Because it uses a parser hook, the C code really is inline with your = Perl code. Pain point 2. Calling C code should be obvious and cheap. All three = modules discussed so far provide a mechanism for calling C functions. = This means that for a simple, small operation, I must wrap my idea into = a function one place and invoke it in another. Furthermore, if I want to = repeatedly call a block of C code in a loop, I must define that block of = code somewhere outside of the loop, potentially very far from the call = site. C::TinyCompiler suffers further because it uses a complicated and = rather slow calling mechanism. C::Blocks solves this by extracting and jit-compiling the C code at Perl = parse time, generating an OP and inserting it into the Perl OP tree. = This means that you can insert your C code exactly where you want it and = not worry about repeated re-compiles. If you were to wrap the example = given above in a for loop, you would see how this works. Pain point 3. Sharing C code should be as easy as sharing Perl code. = C::TinyCompiler provides a fairly complex mechanism to allow modules to = add declarations and symbols to a compiler context. Any string that uses = that will need to recompile those declarations, however, tempting me to = prematurely optimize by placing all of my C code in one giant string = instead of interspersed among my Perl code. Neither XS::TCC nor = Inline::C provide much (if any) automated machinery to share code. C::Blocks provides a mechanism to share function declarations, struct = definitions, and other identifiers with other cblocks in the current = lexical scope, as well as to share them on a per-package basis. It is = even more versatile than normal Perl function scoping, allowing you to = correctly correlate functionality with lexical scope. (It is also = somewhat buggy, as discussed next.) Pain point 4. Changing C code should not cost anything. Inline::C can = take seconds to recompile a changed set of C code. In contrast, there is = no cost associated with changing code when using XS::TCC and = C::TinyCompiler because they jit-compile their code. That comes at the = cost, however, of always compiling everything each time you invoke your = Perl script. If your C code needs the Perl C API, you will have to = re-parse perl.h every time you compile a code block, which can happen = many times with each execution of your script. Inline::C's caching = mechanism provides a big win in that respect, unless you change your = code. It would be nice if we could somehow cache the result of parsing = ``#include "perl.h"''. C::Blocks uses a fork of tcc that I've been working on for many months = aimed at allowing one compiler context to share its symbol table with = other compiler contexts. This is related to the previous point. The = sharing mechanism discussed in the previous point applies to = preprocessor includes, so once I have compiled a block that uses the = Perl headers, I can share all of those declarations with later = compilation units, without recompiling. In future work, I plan to store = these symbol tables to disk so that they don't even need to be re-parsed = each time you run your script. ---- C::Blocks currently addresses, completely, pain points 1 and 2 above. It = has taken many months to hack on tcc to reach this point. I have now = encountered some segfault-causing issues when trying to share code, yet = I have a hard time reproducing those segfaults with direct tests on tcc. = If you think this sounds like a cool project, I would appreciate some = camaraderie as I try to dig into the internals of tcc. You can find me = on perl's IRC network hanging out on #pdl, #xs, and #tinycc, among other = channels. You can find my work at https://github.com/run4flat/C-Blocks If you would like to help out, let me know and I will give you a tour = through the codebase. :-) Any help or encouragement would be much appreciated! Thanks! David --=20 "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." -- Brian Kernighan ------=_NextPart_000_0076_01CF7754.D687B590 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <HTML><HEAD></HEAD> <BODY dir=3Dltr> <DIV dir=3Dltr> <DIV style=3D"FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000"> <DIV>Sounds pretty cool, David.</DIV> <DIV>A plodder like me is probably just going to stick with Inline, but = it would=20 be great to see C::Blocks takes off. And it has the attractiveness to do = so.</DIV> <DIV>(I like the way you can so easily just plonk the C code right in = there=20 amongst the perl code .... makes Inline look not-so-inline :-)</DIV> <DIV> </DIV> <DIV>Cheers,</DIV> <DIV>Rob</DIV> <DIV> </DIV> <DIV=20 style=3D"BORDER-TOP-COLOR: #000000; BORDER-BOTTOM-COLOR: #000000; = PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #000000 4px solid; = BORDER-RIGHT-COLOR: #000000"> <DIV=20 style=3D'FONT-SIZE: small; TEXT-DECORATION: none; FONT-FAMILY: = "Calibri"; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; = DISPLAY: inline'> <DIV style=3D"FONT: 10pt tahoma"> <DIV style=3D"font-color: black"><B>From:</B> <A = [email protected]=20 href=3D"mailto:[email protected]">David Mertens</A> </DIV> <DIV><B>Sent:</B> Friday, May 23, 2014 10:35 PM</DIV> <DIV><B>To:</B> <A [email protected]=20 href=3D"mailto:[email protected]">[email protected]</A> ; <A = [email protected]=20 href=3D"mailto:[email protected]">Perl Inline Mail List</A> </DIV> <DIV><B>Subject:</B> Announcing C::Blocks, a different way to interface = Perl and=20 C code</DIV></DIV></DIV></DIV> <DIV=20 style=3D"BORDER-TOP-COLOR: #000000; BORDER-BOTTOM-COLOR: #000000; = PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #000000 4px solid; = BORDER-RIGHT-COLOR: #000000"> <DIV=20 style=3D'FONT-SIZE: small; TEXT-DECORATION: none; FONT-FAMILY: = "Calibri"; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; = DISPLAY: inline'> <DIV dir=3Dltr> <DIV> <DIV> <DIV> <DIV>Hey everyone,<BR><BR></DIV> <DIV>tl;dr: C::Blocks is a new TinyCC-based module, presently only = available on=20 github. (1) It jit-compiles blocks of C code, building and inserting OPs = into=20 the Perl OP tree, making invocation of C code essentially free. (2) It = will=20 allow different blocks of C code to share function and struct = declarations, thus=20 removing the need to always recompile perl.h, an otherwise major cost of = jit-compiling C code that can interface with Perl and Perl data=20 structures.<BR></DIV> <DIV> </DIV> <DIV>I am currently seeking help and encouragement to squash the = segfaults that=20 currently prevent the completion of the second feature. :-)<BR></DIV> <DIV><BR>----<BR><BR></DIV> <DIV>I like Perl, but I like C, too. I would like to be able to write = and call C=20 code from Perl in about as painless a way as possible. Inline::C is = nice, as are=20 XS::TCC and C::TinyCompiler, but we can do better. C::Blocks is my = attempt to do=20 better.<BR><BR><B>Pain point 1</B>. C code should be a first class = citizen. With=20 XS::TCC and C::TinyCompiler, you pass your code to the compiler via a = string.=20 With Inline::C, you either place your code at the bottom of your script = in a=20 __DATA__ section, or you enclose it in a string. Steffen's module is = probably=20 the most transparent in this sense. Still, working with an interface = that=20 requires me to compile a string to get my product feels the same as = compiling a=20 regex from a string. This is Perl! We can do better!<BR><BR></DIV> <DIV>C::Blocks does better by using a keyword parser hook. Blocks of = code that=20 you want executed are called like so:<BR><BR></DIV> <DIV> print "Before cblock\n";<BR></DIV> <DIV> cblock=20 {<BR></DIV> printf("In = cblock\n");<BR> <DIV> }<BR></DIV> <DIV> print "After cblock\n";<BR></DIV> <DIV> </DIV> <DIV>If stdio.h is included, you get the output<BR><BR></DIV> <DIV> Before cblock<BR></DIV> <DIV> In cblock<BR></DIV> <DIV> After cblock<BR></DIV> <DIV> </DIV> <DIV>Because it uses a parser hook, the C code really is inline with = your Perl=20 code.<BR><BR></DIV> <DIV><B>Pain point 2</B>. Calling C code should be obvious and cheap. = All three=20 modules discussed so far provide a mechanism for calling C functions. = This means=20 that for a simple, small operation, I must wrap my idea into a function = one=20 place and invoke it in another. Furthermore, if I want to repeatedly = call a=20 block of C code in a loop, I must define that block of code somewhere = outside of=20 the loop, potentially very far from the call site. C::TinyCompiler = suffers=20 further because it uses a complicated and rather slow calling=20 mechanism.<BR><BR></DIV> <DIV>C::Blocks solves this by extracting and jit-compiling the C code at = Perl=20 parse time, generating an OP and inserting it into the Perl OP tree. = This means=20 that you can insert your C code exactly where you want it and not worry = about=20 repeated re-compiles. If you were to wrap the example given above in a = for loop,=20 you would see how this works.<BR><BR></DIV> <DIV><B>Pain point 3</B>. Sharing C code should be as easy as sharing = Perl code.=20 C::TinyCompiler provides a fairly complex mechanism to allow modules to = add=20 declarations and symbols to a compiler context. Any string that uses = that will=20 need to recompile those declarations, however, tempting me to = prematurely=20 optimize by placing all of my C code in one giant string instead of = interspersed=20 among my Perl code. Neither XS::TCC nor Inline::C provide much (if any)=20 automated machinery to share code.<BR><BR></DIV> <DIV>C::Blocks provides a mechanism to share function declarations, = struct=20 definitions, and other identifiers with other cblocks in the current = lexical=20 scope, as well as to share them on a per-package basis. It is even more=20 versatile than normal Perl function scoping, allowing you to correctly = correlate=20 functionality with lexical scope. (It is also somewhat buggy, as = discussed=20 next.)<BR></DIV> <DIV> </DIV> <DIV><B>Pain point 4</B>. Changing C code should not cost anything. = Inline::C=20 can take seconds to recompile a changed set of C code. In contrast, = there is no=20 cost associated with changing code when using XS::TCC and = C::TinyCompiler=20 because they jit-compile their code. That comes at the cost, however, of = always=20 compiling everything each time you invoke your Perl script. If your C = code needs=20 the Perl C API, you will have to re-parse perl.h every time you = <I>compile</I> a=20 code block, which can happen many times with each execution of your = script.=20 Inline::C's caching mechanism provides a big win in that respect, unless = you=20 change your code. It would be nice if we could somehow cache the result = of=20 parsing ``#include "perl.h"''.<BR><BR></DIV> <DIV>C::Blocks uses a fork of tcc that I've been working on for many = months=20 aimed at allowing one compiler context to share its symbol table with = other=20 compiler contexts. This is related to the previous point. The sharing = mechanism=20 discussed in the previous point applies to preprocessor includes, so = once I have=20 compiled a block that uses the Perl headers, I can share all of those=20 declarations with later compilation units, without recompiling. In = future work,=20 I plan to store these symbol tables to disk so that they don't even need = to be=20 re-parsed each time you run your script.<BR></DIV> <DIV> </DIV>----<BR><BR></DIV> <DIV>C::Blocks currently addresses, completely, pain points 1 and 2 = above. It=20 has taken many months to hack on tcc to reach this point. I have now = encountered=20 some segfault-causing issues when trying to share code, yet I have a = hard time=20 reproducing those segfaults with direct tests on tcc. If you think this = sounds=20 like a cool project, I would appreciate some camaraderie as I try to dig = into=20 the internals of tcc. You can find me on perl's IRC network hanging out = on #pdl,=20 #xs, and #tinycc, among other channels. You can find my work at <A=20 href=3D"https://github.com/run4flat/C-Blocks">https://github.com/run4flat= /C-Blocks</A><BR><BR></DIV> <DIV>If you would like to help out, let me know and I will give you a = tour=20 through the codebase. :-)<BR></DIV> <DIV> </DIV> <DIV>Any help or encouragement would be much appreciated! = Thanks!<BR>David<BR=20 clear=3Dall></DIV></DIV></DIV> <DIV> <DIV> <DIV> <DIV> <DIV><BR>-- <BR>"Debugging is twice as hard as writing the code in the = first=20 place.<BR> Therefore, if you write the code as cleverly as = possible, you=20 are,<BR> by definition, not smart enough to debug it." -- Brian=20 Kernighan<BR></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV>= </BODY></HTML> ------=_NextPart_000_0076_01CF7754.D687B590--