Re: Bridging GProlog with other systems [Was: Re: FFI experimentation, some advice please...]

On Fri, Jul 5, 2013 at 4:23 PM, emacstheviking <[email protected]> wrote:
> Good mail. My first "real target" once I get FFI working is "cURL"... apart
> from MySQL, most if the new kids on the block these days have an HTTP(S)
> REST API making them easy to talk to, as usual the devil will be in the
> details. I mentally want to be able to do something like this:
>
>    http_req(get, 'url', [ param(K,V), param(K,V), ... ], Response).

    Indeed having a way to make HTTP requests would be an even simpler
solution.  However I doubt it would get you too far in talking
directly with the resources (in terms of implementation effort I
mean).

    For example Riak, CouchDB, and many of the NoSQL solutions out
there which implement a REST-ful approach, will require more than just
an URL and some URL-encoded parameters.  For example the two
previously cited solutions require you to set appropriate headers, and
carefully interpret both the returned status and headers besides the
body.  As such you'll need to provide many other HTTP-related tools to
tackle things like headers, encoding, chunks, etc.

    However if you take this approach I would suggest looking over
webmachine, written in Erlang which could provide an insight in both
how "intricate" the REST-ful world is and how to tackle it.

      https://github.com/basho/webmachine

    Therefore I would predict that even in such a case you'll revert
to a solution where you use HTTP from GProlog just to invoke a
"driver" that really interacts with the resource.


>>     Moreover, in response to the original poster idea of using MySQL,
>> Redis, etc. directly from GProlog, I think it's highly impractical
>> decision, especially since all development must be done in C (or
>> wrapping C++ in C code).  (Is there any "production-ready" RabbitMQ
>> library for C?  How about for Riak, CouchDB or many other "NoSQL"
>> solutions?)
>
> I wouldn't say highly impractical, just the need to be methodical and
> meticoulous to get a core FFI system in place like Lisp or Haskell.
>
> I have already used "librabbitmq" and "libmysqlcppcon" from C/C++ code and I
> know that they work well... as I said, if gprolog can speak FFI and/or cURL
> then we are away!

    My "impractical" statement referred to the availability (and
quality) of C libraries for "modern" resources, services, databases,
etc.  Many of them are first written for more "popular" programming
languages like Java, Python, Ruby, etc.


> I have evaluated MQZero too but we didn't use it and went
> for a clustered RabbitMQ solution.

    Please note that I've proposed ZeroMQ as a transport between
GProlog and the "driver" that happens to speak with the actual
resource.  For this purpose (of bridging just GProlog with a
"resource" driver) RabbitMQ is an overkill.


> I will check out your stuff over the weekend to see what I may have missed
> as well in terms of ideas for solutions.
>
> [...]
>
> I never discount anything though so I will def. read those github resources,
> being a hacker junky! I had a quick look for ten minutes and that is some
> pretty sexy code that is.

    Please do so and ask me if you encounter problems.  Please note
that my code is just a "toy" prototype.


> For me though, I kind of like the idea of minimal moving parts and being
> able to  produce the smallest piece of kit that just works is what drives
> me. The more technologies you mix the more failure points you have and then
> it snowballs as you wrap measures around each failure point etc etc I am
> sure we have all been there and seen it happen; what was a small simple
> working system now looks hideous.

    Although I agree that small and simple is better, I tend to
disagree on artificially reducing the number of moving parts.
Currently we live in a distributed world, where even the simplest
application has a few moving parts (web client, web server, database).
 Thus I think the focus should move into the direction of finding
reliable solutions to the distributed problem instead of avoiding it.

    That's another reason why I wouldn't just use a C library from
GProlog:  it would provide me with a very low-level interface in
return of reducing moving parts.  However the code that bridges the
abstraction level is still there with all the complexity.

    Ciprian.