parsing EXPRESS (ISO 10303-11)

[email protected] (Ingo Wichmann) Sun, 15 Sep 2002 17:37:53 +0200
Newsgroups perl.recdescent
Message-ID <[email protected]>
Hi!

I'm writing a skript to parse the language EXPRESS (ISO 10303-11).
Until now it looked like an easy job to do with Parse::RecDescent.
But now i'm looking at the strange outputs of my skript and i wonder
whats happening.

Maybe your more motivated to help me if i tell you that you can
get my skript from http://savannah.gnu.org and its gpl.

Here is an extract of the exp-file i want to parse:

--- begin

REFERENCE FROM Lichtsignalanlage (oertliche_Zuordnung_LS);

ENTITY Strassenklasse;
    Kennung                       : STRING(1);
    Langtext                      : STRING;
UNIQUE
    Kennung_eindeutig             : Kennung;
END_ENTITY;

--- end

You can get the hole file under
http://savannah.gnu.org/cgi-bin/viewcvs/okstraperl/okstraperl/express2perl/expdateien/e001-1003.exp

to be exact, i've parsed the exp-file file before to get the comments
out, but the changes should concern only whitespace.

Here are the rules involved with this extract in the order they appear:
(The strange things happen after the comment marked with ##, so if 
you're in a hurry continue reading there)
The complete skript you find under
http://savannah.gnu.org/cgi-bin/viewcvs/okstraperl/okstraperl/express2perl/parsetest.pl

--- begin

schema_body:	interface_specification(s?) constant_decl(?) ( declaration 
| rule_decl )(s?)
# The extract shown is part of a schema_body.
interface_specification:
		reference_clause | use_clause
reference_clause:
		'REFERENCE' 'FROM' schema_ref ( '(' resource_or_rename
		( ',' resource_or_rename )(s?) ')' )(?) ';'
schema_ref:	schema_id
		{ $return = $item[1] }
schema_id:	simple_id
		{ $return = $item[1] }
simple_id:	m/[a-zA-Z]\w*/
		{ $return = $item[1] }
resource_or_rename:
		resource_ref ( 'AS' rename_id )(?)
resource_ref:	constant_ref | entity_ref | function_ref | procedure_ref | 
type_ref
constant_ref:	constant_id
		{ $return = $item[1] }
constant_id:	simple_id
		{ $return = $item[1] }
# Seems like parsing this reference_clause worked correctly [1]
declaration:	entity_decl | function_decl | procedure_decl | type_decl
entity_decl:	entity_head entity_body 'END_ENTITY' ';'
entity_head:	'ENTITY' entity_id subsuper(?) ';'
entity_id:	simple_id
		{ $return = $item[1] }
subsuper:	supertype_constraint(?) subtype_declaration(?)
entity_body:	explicit_attr(s?) derive_clause(?) inverse_clause(?) 
unique_clause(?) where_clause(?)
explicit_attr:	attribute_decl ( ',' attribute_decl )(s?)  ':' ( 
'OPTIONAL' )(?) base_type ';'
attribute_decl:	attribute_id | qualified_attribute
attribute_id:	simple_id
		{ $return = $item[1] }
base_type:	aggregation_types | simple_types | named_types
simple_types:	binary_type | boolean_type | integer_type | 
logical_type 		| number_type | real_type | string_type
string_type:	'STRING' width_spec(?)
width_spec:	'(' width ')' ( 'FIXED' )(?)
width:		numeric_expression
numeric_expression:
		simple_expression
simple_expression:
		term ( add_like_op term )(s?)
term:		factor ( multiplication_like_op factor )(s?)
factor:		simple_factor ( '**' simple_factor )(?)
simple_factor:	aggregate_initializer | entity_constructor |
		enumeration_reference | interval | query_expression |
		( unary_op(?) ( '(' expression ')' | primary ) )
primary:	literal | ( qualifiable_factor qualifier(s?) )
literal:	binary_literal | integer_literal | logical_literal | 
real_literal | string_literal
integer_literal:
		digits
		{ $return = $item[1] }
digits:		/\d+/o
		{ $return = $item[1] }
# With these rules, the part just before the keyword 'UNIQUE' should be 
matched
## The next rule does not match correclty:
unique_clause:	'UNIQUE' ( unique_rule ';' )(s)
unique_rule:	( label ':' )(?) referenced_attribute ( ',' 
referenced_attribute )(s?)
referenced_attribute:
		attribute_ref | qualified_attribute

--- end

With
$::RD_AUTOACTION = q { print join("--",@item),"\n" };
set, the skript produces the following output
(Again, the weird things happen after the ## comment)

--- begin

# here the reference_clause starts:
resource_ref--oertliche_Zuordnung_LS
resource_or_rename--1--ARRAY(0x905b764)
_alternation_1_of_production_1_of_rule_reference_clause--(--1--ARRAY(0x905a5f8)--)
reference_clause--REFERENCE--FROM--Lichtsignalanlage--ARRAY(0x9058914)--;
interface_specification--1

# referenc clauses are over, no use clauses and no constant_decl here, 
so lets try with a declaration:
subsuper--ARRAY(0x9058a34)--ARRAY(0x905b728)
entity_head--ENTITY--Strassenklasse--ARRAY(0x90601a0)--;
# Oh, fine, looks like an entity_declaration.

attribute_decl--Kennung
literal--1
primary--1
_alternation_1_of_production_1_of_rule__alternation_1_of_production_6_of_rule_simple_factor--1
_alternation_1_of_production_6_of_rule_simple_factor--ARRAY(0x90b8bac)--1
simple_factor--1
factor--1--ARRAY(0x90b89e4)
term--1--ARRAY(0x90b8a20)
simple_expression--1--ARRAY(0x90f3464)
numeric_expression--1
width--1
width_spec--(--1--)--ARRAY(0x90f317c)
string_type--STRING--ARRAY(0x905b8f0)
simple_types--1
base_type--1
explicit_attr--1--ARRAY(0x90588fc)--:--ARRAY(0x905a748)--1--;
attribute_decl--Langtext
string_type--STRING--ARRAY(0x90f3194)
simple_types--1
base_type--1
explicit_attr--1--ARRAY(0x905b6c8)--:--ARRAY(0x905a580)--1--;

## until here everything is fine, now it gets strange:
attribute_decl--UNIQUE
# What? attribute_decl == 'UNIQUE' ?
# What does this attribute_decl belong to? To unique_rule?
# How can there be a unique_rule with an attr 'UNIQUE' if
# there is no unique_clause starting with and "eating" the keyword 
'UNIQUE' ?
_alternation_1_of_production_1_of_rule_unique_rule--Kennung_eindeutig--:
referenced_attribute--Kennung
unique_rule--ARRAY(0x905b9c8)--1--ARRAY(0x90f30e0)
_alternation_1_of_production_1_of_rule_unique_clause--1--;
referenced_attribute--END_ENTITY
unique_rule--ARRAY(0x905b9bc)--1--ARRAY(0x90f3230)
_alternation_1_of_production_1_of_rule_unique_clause--1--;
referenced_attribute--TYPE
unique_rule--ARRAY(0x905b9ec)--1--ARRAY(0x905b9e0)
unique_clause--UNIQUE--ARRAY(0x905b92c)
entity_body--ARRAY(0x905a5ec)--ARRAY(0x90588d8)--ARRAY(0x90601c4)--ARRAY(0x905a6

--- end

Any hints?

To reproduce this output download
http://savannah.gnu.org/cgi-bin/viewcvs/okstraperl/okstraperl/express2perl/okstra-parse.pl
and
http://savannah.gnu.org/cgi-bin/viewcvs/okstraperl/okstraperl/express2perl/expdateien/e001-1003.exp
Save the latter in a subdirectory named "expdateien".
Execute okstra-parse.pl. A file named STDIN will be produced.
Download
http://savannah.gnu.org/cgi-bin/viewcvs/okstraperl/okstraperl/express2perl/parsetest.pl
Execute on a shell
cat STDIN | parsetest.pl
and see the trouble.

Thanks,

Ingo

[1] In fact this reference_clause references a entity_ref, not a 
constant_ref, but they are grammaticaly the same. I have to find out how 
to deal with this later.