Re: U+FFFF

On 31/07/2013, at 7:02 PM, Jan Wielemaker wrote:

> On 07/31/2013 03:28 AM, Richard A. O'Keefe wrote:
> 
>> That's not actually an answer.  The question is how did *THIS*
>> U+FFFF get there in *THIS* case and what *SHOULD* it have been?
> 
> Nope. As the OP says, it comes from an external tool.

Yes, I know that.  I spent some time trying to find out
what tool it is.

> That is not in our control. It is illegal Unicode, so if the input was supposed to be
> Unicode, we can raise the flag or just pass it over to the next
> processing stage.

There are several different standpoints we might take.

One is "what, if anything, should be done to SWI Prolog about this?"
Do we agree that the answer is "right now, nothing."

But the standpoint I was trying to take there was
"what should the original poster do about the U+FFFF in his data?"
And for _that_, the answer "reject it" is probably wrong,
and the answer "just treat it like any other character" is _also_
probably wrong.  The question "How did THIS U+FFFF get there in
THIS case" is evaded, not answered, by saying "from an external
tool".  The question is whether the external tool was passing on
bad data _it_ was given -- in which case the original poster might
be able to correct that source -- or whether the external tool
originated the U+FFFF with the "intent" of signifying something
by it.  And that means "what SHOULD it have been" is relevant,
because in that case the original poster needs to know what
significance the external tool "intends" to convey by emitting
that code so that the original poster can figure out what to do
about it.

When the original poster _does_ figure out what effect to produce
when the external tool sends a U+FFFF (perhaps it means different
things in different contexts) the question will then be how to
detect that situation and produce that effect in Prolog.


>> The question remains for someone trying to *interpret* this
>> stuff, "what on earth was THIS occurrence of U+FFFF supposed
>> to mean? how did the originator INTEND it to be processed?"
> 
> I guess there is no meaningful answer to that.

More than likely there _is_ a meaningful answer,
and whoever wrote JTransformer (whichever JTransformer it is)
is the person or are the group to ask for it.

The paper "A Declarative Approach for Software Modeling",
Mayer Goldberg and Guy Wiener, PADL 2012, LNCS 7149,
mentions _a_ JTransformer:

	JTransformer is a logic-based query and transformation
	engine for Java code based on the Eclipse IDE.

If this _is_ the JTransformer that the original poster was
talking about, then the paper "A Comparison of Logic-Based
Infrastructures for Concern Detection and Extraction",
Günter Kniesel, Jan Hannemann, & Tobias Rho, Proc of AOSD 2007
Workshop on Linking Aspect Technology and Evolution (LATE'07)
tells us that
	JTransformer uses the platform-independent,
	free, SWI-Prolog implementation for storing,
	analyzing, and transforming factbases.

So if _that_ is the external tool that produce the U+FFFF that
bothered the original poster, it may well be an SWI Prolog issue
after all.

If the original poster is still listening, how about a URL
for the JTransformer you're using so that we can know instead
of speculating?