Re: Request for Comment: Callable CRM114 Classifiers (libcrm114) --> OOBC

On Mon, Sep 22, 2008 at 5:17 PM, Bill Yerazunis <[email protected]> wrote:
>
> [[[  Nota Bene:  This is just a design, not working code.  But something
>     like this (code cleanup) is something we probably need to do anyway
>     to make it possible to move forward in future work without tons of
>     almost-but-not-quite-identical cargo-culted C code in the classifiers.
>
>     Anyway- any comments?  Broken-nesses?
>
>        - Bill Yerazunis ]]]

As part of the commenting on this RFC, I am going to work on a bit of
PoC software this weekend as I like to see stuff 'animated' and the
discussions up to now have given me a much better idea of what I like
and don't like in the proposal, but I think it benefits our progress
if I can show a working API - for myself at the very least.




Working up to that, there is my pet peeve, which is, regrettably
though understandably, almost completely lacking from this design as
well (as it is from so many software designs, sigh):



OUT-OF-BAND COMMUNICATIONS

which consist of three equally important parts:

1- error handling and [subsequent] signaling (e.g. return codes -
these /are/ in the proposal to a limited degree)

2- error reporting (which is quite different from 'signaling':
signaling = 'it happened!', reporting = '*what* happened *where* [and
*why* would be nice too]'

3- *monitoring* / diagnostics  (which is /not/ only useful in error
conditions, but is required to analyse all 'suspect situations')



ad 1: this can be handled by the return codes as described in Bill's
proposal. I like absence of exceptions in rules (KISSed rules), so the
simple rule I prefer here is this: return value 0 means success,
positive return values signal informationals[*] (suspect situations
which nevertheless were executed to completion), negative values are
errors and fatals (fatal errors are just 'more worrying' than other
errors).

What is an error? Anything that will prevent the [libcrm114] process
from executing according to all assumptions (input + output
assumptions, execution assumptions). Example errors: missing inputs,
failure to calculate an [intermediate] value, failure to access
required resources.

What is an informational? Anything that is out of the ordinary but
does not cause the process to fail the assumptions. These are a bit
like people who'd be called 'hinky' by police officers: they show
'suspect' [nonverbal] behaviour, but just this feeling alone is not
enough to pick them off the street. Example informationals: inputs
which produce a single 'feature' for classification: theoretically
these are valid messages (so they fit the input assumptions) but the
produced classification is highly suspect as the classification is
based on that single feature individual. It's somewhat like applying
group dynamics results on a single particular individual to check his
behaviour: not what you should do (but it happens all the time in
meatspace).


[*] I call these 'informationals' akin to the informational errorcodes
one has in VMS; some would call these 'warnings' (and often I would
too), but right here, 'warning' would be a completely wrong word to
use as it has a high risk of triggering the wrong ideas in your
mind(s), *especially* when you've never developed software on VMS
using system calls (sys$ anyone? Or am I that much a dinosaur already?
;-) ).

http://h71000.www7.hp.com/doc/73final/6023/6023pro_001.html

http://h71000.www7.hp.com/doc/82final/5973/5973pro_022.html
(... and all that goodness in a single 32-bit integer. <swoon! /> )

http://mail.python.org/pipermail/doc-sig/2001-August/002028.html

http://www.mindspring.com/~blackhart/I_Miss_VMS.html
;-) (One correction there: it's not designed, it's _engineered_. Rare quality.)





ad 2: error reporting is important to help engineers and users
diagnose and remedy issues. I like my error reports to be very
verbose: I'd rather get too much to work with (heck, I can filter! The
tool is called a 'brain'!) that too little. Folks old enough will
remember the compilers who would simply report 'syntax error' for
every failure to comply. Good luck finding the offending bit then.

Verbosity cannot be done through 'printf()' et al: some (embedded)
systems don't have these, several possible target applications will be
GUI-based and thus need platform-dependent messages to get the error
report to show up on screen; server/daemon-ized systems should
preferably use syslog (or maybe the EventLog (Windows)) or other
[networked] means to help remote administration and problem solving.
--> error reporting hooks are required elements of the interface.

(simple example: take the [UNIX] getopt(3) interface: in this sense it
is crappy, because any errors occurring within getopt() et al will be
pushed out using hardcoded fprintf() to stderr; you are bound to that
error channel (and error format besides), no way to redirect it.
(Hence I carry around my own getopt_ex() which does provide an error
reporting callback:

getopt():

int getopt(int argc, char *const *argv, const char *optstring);

getopt_ex():

int getopt_ex(int argc, char *const *argv, const char *optstring, void
(*print_error)(void *, const char *, ...), void *propagate);




ad 3: diagnostics is NOT about adding extra verbosity a la stack trace
on top of error reporting. Electronic engineers will know the JTAG
interface, which can be used to monitor / diagnose 'live' hardware:
while the system runs, you collect/receive bits of information from
all [important] parts so you can see what is going on.

(People who have worked with me before, will recall several such
software components dubbed 'JTAG-something' in server/core systems,
where those can be used to spit out any [configurable] amount of
diagnostic information on [selected] subsystems, even in live
production environments (where they have proven most useful!). ;-) )

http://en.wikipedia.org/wiki/JTAG

>From very far off, it's a bit similar to the current 'crm114 -t' and
'crm114 -T' trace modes. These are a good starting point. What is
lacking in the current '-t' and '-T' is a programmable 'target': a la
error reporting, one should be able to provide a (separate!!!)
callback/hook to pick up and store/display such diagnostics: in a UNIX
server environment, I may wish to have all my errors directed at a
[remote] syslog channel, while any diagnostic output for particular
'traced/JTAGged' runs should end up at a *different* location (remote
or local [sys]logfile: you won't make your admins happy by feeding a
multi-megabyte trace into their server logs, and unhappy admins tend
to act like BOFH after a while:

http://bofh.ntk.net/
;-)

(Besides, blasting that much data into a syslog pipeline will get you
[lots of] lost UDP packets and thus no or b0rked log output to look
at. Mission failure.)

Selective section tracing is useful, but when you want a dry KISS
approach (Keep It Scruffy for Senile coders)[*] you can leave that
part out and just provide -t/-T with an API-developer programmable
target hook (function + propagator); good enough.

[*]I am more a 'wet kiss' person. Wet kisses are... ah, dang!
Deranging again! What I got to say is this:

Interns should sometimes look but not touch -- akin to small kids
running amok in a store -- if you know whut ahm sayin'.








Oh yeah: *proper* callback hooks consist of both (1) a 'callback' AND
(2) a 'propagator' a la the getopt_ex() above. The 'propagator' (=
'propagating' arbitrary data element: a 'void*' in C) is not to be
used by the core system but only to be passed along to the callback at
each invocation.
When you also pass the libcrm114 control structure to each callback
(not shown in getopt_ex()) you have the humble beginnings of a Visitor
Pattern as one might implement it in 'C', BTW. In case you thought
this was going RBG (rock bottom geek) again and the migraine starts:
this is design, mate. ;-)

C++/C#/Java/etc.etc. extension developers will love you for that
propagator thingy as they can pass mandatory object/thread/etc.
*context* to the callback, so that they can 'export' the hook to the
programming environment the extension is meant for. (Without the
propagator, you'd have a VERY HARD TIME to hook a functor to a
libcrm114 callback hook, for instance. No matter what the language the
functor is written in.)

http://en.wikipedia.org/wiki/Function_object

http://www.newty.de/fpt/functor.html

... and when anyone thinks they've got what it takes to do without the
propagate element, check this solution and see if you like it:

http://fscked.org/projects/minihax/c-functor-c-function-pointer-conversion
(attentive readers will note the 2 words 'relatively portable' in
there. ;-)  A black soul indeed, but R.E.S.P.E.C.T. for the man who
did concoct that one up. I am not worthy! And _so_ envious it actually
hurts!   <genuflect with happiness />  )



-- 
Met vriendelijke groeten / Best regards,

Ger Hobbelt

--------------------------------------------------
web: http://www.hobbelt.com/
 http://www.hebbut.net/
mail: [email protected]
mobile: +31-6-11 120 978
--------------------------------------------------

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/