[committed] cobol: Speed up parsing
Robert Dubner <[email protected]>
| Newsgroups | gmane.comp.gcc.patches |
|---|---|
| Message-ID | <[email protected]> |
From: Robert Dubner <[email protected]> Date: Tue, 18 Aug 2026 20:42:17 -0400 Subject: [PATCH] cobol: Speed up parsing. Recently added parameter validation introduced symbol-table scans for function and program names. Remove the scans and instead maintain maps of functions and programs. There was also an incongruous per-statement cost to removing alphanumeric temporaries. To avoid it, when such temporaries are added, iterators to them are kept in a separate list. When comes time to send alphanumeric temporaries to codegen, the iterators are dereferenced to produce a small ephemeral list and then to clean up the main temporaries list. Reduce lexer scans of symbol table. The lexer uses the symbol table to distinguish Level 88 types and to unwind named literals (so the parser sees only the literal). It maintains a cache but had a fallback to scan the symbol table in case of a cache miss. Unnecessary scanning for paragraph and section names cost about 1/3 of the time in the parser. Remove symbol-table scans from the lexer in Procedure Division. If the CDF adds a named literal (with CDF DEFINE) update the lexer's cache. Retain cdf_state SC while conditional compilation is false. The initial change was to recognize the COBOL nonce word FILLER, to prevent looking it up in the token-name list, for efficiency. That change made other problems apparent. Use the cdf_state start condition for all scanning while conditional compilation prevents parsing. Do not push more than one cdf_state onto the SC stack, and do not leave cdf_state until parsing resumes. This reduces the amount of start-condition state preservation required when the CDF parser is invoked. The change unveiled some latent problems, mostly in debug messages. Co-authored-by: James K. Lowden <[email protected]> gcc/cobol/ChangeLog: * cdf.y: Revise debug message and token numbers. Remove USED_TOKEN macro, introduce SAVE_LOOKAHEAD. * gengen.cc (class DECL_COUNTER): Add VAR_DECL creation counter. (gg_define_from_declaration): Likewise. * parse.y: Move named XML tokens to first token list and recognize FILLER. * scan.l: Eliminate scan used for debug message. * scan_ante.h (level_found): Remove function. (PUSH_CDF_STATE): Push cdf_state SC at most once. (trim_location): Less verbose debug message. (symbol_lower_name): New function. (symbol_exists): Use symbol_lower_name and accept override parameter. (scanner_cache_update): New function. (unquote): New function for string literals. * scan_post.h (start_condition_str): Update with new SC names. (run_cdf): Do not pop from cdf_state unconditionally. (RECENT): Remove macro. (same_end): Remove function. (same_token): Same. (next_token): Same. (prelex): Use cdf::last_lexed and cdf::had_lookahead(). (yylex): Remove one early return. * symbols.cc (symbol_program): Use new map of programs. (symbol_function_impl): Use new map of functions. (symbol_temporaries): Use new list of alphanumerics. (symbol_temporary_alphanumerics): New class to house alphanumerics list. (symbol_program_add): Populate new maps. * token_names.h: Regenerate. * util.cc (cdf_field_add): Use scanner_cache_update. libgcobol/ChangeLog: * posix/README.md: Updated documentation. * posix/shim/open.cc (posix_open): Remove O_LARGEFILE from POSIX mode flags. --- gcc/cobol/cdf.y | 205 +++--- gcc/cobol/gengen.cc | 59 ++ gcc/cobol/parse.y | 32 +- gcc/cobol/scan.l | 431 +++++++------ gcc/cobol/scan_ante.h | 108 +++- gcc/cobol/scan_post.h | 238 +++---- gcc/cobol/symbols.cc | 122 +++- gcc/cobol/token_names.h | 1166 +++++++++++++++++----------------- gcc/cobol/util.cc | 6 +- libgcobol/posix/README.md | 41 ++ libgcobol/posix/shim/open.cc | 3 +- 11 files changed, 1312 insertions(+), 1099 deletions(-) diff --git a/gcc/cobol/cdf.y b/gcc/cobol/cdf.y index 81198daff37..7145d97b01a 100644 --- a/gcc/cobol/cdf.y +++ b/gcc/cobol/cdf.y @@ -79,22 +79,31 @@ operator<<( std::ostream& os, cbl_loc_t const& loc ) { const char * keyword_str( int token ); namespace cdf { -static int current_token; -static int used_token( int token ) { - dbgmsg("%s: %s", __func__, keyword_str(token)); - return current_token = token; -} - -int used_token() { return current_token; } + static int lookahead_token_kind; + static int lookahead( int token ) { + if( token < 1 ) { + dbgmsg("%s: look ahead to %s", __func__, keyword_str(token)); + } else { + dbgmsg("%s: look ahead to token kind %d", __func__, token); + } + lookahead_token_kind = token; + return lookahead_token_kind; + } + + bool had_lookahead() { + int kind = 0; + std::swap(kind, lookahead_token_kind); + return 0 < kind; + } } -#define USED_TOKEN(T) cdf::used_token(parser::token::YDF_ ##T) +#define SAVE_LOOKAHEAD cdf::lookahead(yyla.kind()) %} %code requires { namespace cdf { - class parser; + class parser; } } @@ -102,10 +111,24 @@ int used_token() { return current_token; } // https://learnmoderncpp.com/2020/12/17/generating-c-programs-with-flex-and- bison-2/ namespace cdf { - int cdflex( parser::semantic_type *value, cbl_loc_t *loc ); + int cdflex( parser::semantic_type *value, cbl_loc_t *loc ); } } +%code provides { + namespace cdf + { + template <typename F> + int token_of( int kind, F xlate ) { + for( int tok = -2; tok < parser::token::YDF_NEG + 1; tok++ ) { + int token = xlate(tok); + if( token == kind ) return tok; + } + assert(false); + } + } +} + %define api.location.type {cbl_loc_t} %{ @@ -292,9 +315,9 @@ apply_cdf_turn( const exception_turn_t& turn ) { %token IN 606 %token NAME 286 %token NUMSTR 306 "numeric literal" -%token OF 687 -%token PSEUDOTEXT 722 -%token REPLACING 744 +%token OF 689 +%token PSEUDOTEXT 724 +%token REPLACING 746 %token LITERAL 299 %token SUPPRESS 379 @@ -318,23 +341,23 @@ apply_cdf_turn( const exception_turn_t& turn ) { %token AS 470 CONSTANT 364 DEFINED 366 %type <boolean> DEFINED -%token OTHER 699 PARAMETER_kw 371 "PARAMETER" -%token OFF 688 OVERRIDE 372 -%token THRU 949 -%token TRUE_kw 814 "True" +%token OTHER 701 PARAMETER_kw 371 "PARAMETER" +%token OFF 690 OVERRIDE 372 +%token THRU 947 +%token TRUE_kw 816 "True" %token CALL_COBOL 394 "CALL" %token CALL_VERBATIM 395 "CALL (as C)" -%token TURN 816 CHECKING 498 LOCATION 650 ON 690 WITH 843 +%token TURN 818 CHECKING 498 LOCATION 650 ON 692 WITH 845 -%left OR 950 -%left AND 951 -%right NOT 952 -%left '<' '>' EQ 298 "EQUAL" NE 953 LE 954 GE 955 +%left OR 948 +%left AND 949 +%right NOT 950 +%left '<' '>' EQ 298 "EQUAL" NE 951 LE 952 GE 953 %left '-' '+' %left '*' '/' -%right NEG 957 +%right NEG 955 %require "3.8.2" // for C++ output %language "c++" @@ -345,20 +368,22 @@ apply_cdf_turn( const exception_turn_t& turn ) { %locations %define parse.error verbose %% -top: partials { YYACCEPT; } +top: partials { + SAVE_LOOKAHEAD; YYACCEPT; } | copy '.' { const char *library = copybook.library(); if( !library ) library = "SYSLIB"; const char *source = copybook.source(); dbgmsg("COPY %s from %s", source, library); + SAVE_LOOKAHEAD; YYACCEPT; } | copy error { error_msg(@error, "COPY directive must end in a %<.%>"); YYABORT; } - | completes { YYACCEPT; } + | completes { SAVE_LOOKAHEAD; YYACCEPT; } ; completes: complete @@ -374,10 +399,10 @@ complete: cdf_define ; /* - * To do: read ISO 2022 to see how >>DISPLAY is dictionary! + * To do: read ISO 2023 to see how >>DISPLAY is dictionary! * To do: DISPLAY UPON * To do: decide what to do about newlines, and when; DISPLAY has - * {}... in the specification. + * {}... in the specification. */ cdf_display: CDF_DISPLAY strings { if( scanner_parsing() ) { @@ -388,12 +413,10 @@ cdf_display: CDF_DISPLAY strings { } ; strings: LITERAL { - USED_TOKEN(LITERAL); - display_msg = xstrdup($1); + display_msg = xstrdup($1); } | strings LITERAL { - USED_TOKEN(LITERAL); - char *p = display_msg; + char *p = display_msg; display_msg = xasprintf("%s %s", p, $2); free(p); } @@ -401,19 +424,25 @@ strings: LITERAL { partials: partial { - if( ! scanner_parsing() ) YYACCEPT; + if( ! scanner_parsing() ) { + SAVE_LOOKAHEAD; + YYACCEPT; + } } | partials partial { - if( ! scanner_parsing() ) YYACCEPT; + if( ! scanner_parsing() ) { + SAVE_LOOKAHEAD; + YYACCEPT; + } } ; partial: cdf_if /* text */ - | CDF_ELSE { USED_TOKEN(CDF_ELSE); scanner_parsing_toggle(); } - | CDF_END_IF { USED_TOKEN(CDF_END_IF); scanner_parsing_pop(); } + | CDF_ELSE { scanner_parsing_toggle(); } + | CDF_END_IF { scanner_parsing_pop(); } | cdf_evaluate /* text */ | cdf_eval_when /* text */ - | CDF_END_EVALUATE { USED_TOKEN(CDF_END_EVALUATE); scanner_parsing_pop(); } + | CDF_END_EVALUATE { scanner_parsing_pop(); } ; cdf_define: CDF_DEFINE cdf_constant NAME as cdf_expr[value] override @@ -445,8 +474,7 @@ cdf_define: CDF_DEFINE cdf_constant NAME as cdf_expr[value] override } | CDF_DEFINE cdf_constant NAME as OFF { - USED_TOKEN(OFF); - cdfval_off( $NAME); + cdfval_off( $NAME); } | CDF_DEFINE cdf_constant NAME as PARAMETER_kw override /* @@ -464,16 +492,14 @@ cdf_define: CDF_DEFINE cdf_constant NAME as cdf_expr[value] override } } | CDF_DEFINE FEATURE as ON { - USED_TOKEN(ON); - auto feature = cbl_gcobol_feature_t($2); + auto feature = cbl_gcobol_feature_t($2); if( ! cobol_gcobol_feature_set(feature, true) ) { error_msg(@FEATURE, "%<>>DEFINE %%EBCDIC-MODE%> is invalid within program body"); } } | CDF_DEFINE FEATURE as OFF { - USED_TOKEN(OFF); - auto feature = cbl_gcobol_feature_t($2); + auto feature = cbl_gcobol_feature_t($2); if( ! cobol_gcobol_feature_set(feature, false) ) { error_msg(@FEATURE, "%<>>DEFINE %%EBCDIC-MODE%> is invalid within program body"); @@ -481,12 +507,10 @@ cdf_define: CDF_DEFINE cdf_constant NAME as cdf_expr[value] override } ; cdf_constant: %empty - | CONSTANT { - USED_TOKEN(CONSTANT); - } + | CONSTANT ; override: %empty { $$ = false; } - | OVERRIDE { $$ = true; USED_TOKEN(OVERRIDE); } + | OVERRIDE { $$ = true; } ; cdf_turn: TURN except_names except_check @@ -498,11 +522,9 @@ cdf_turn: TURN except_names except_check cdf_call_convention: CALL_COBOL { - USED_TOKEN(CALL_COBOL); current_call_convention(cbl_call_cobol_e); } | CALL_VERBATIM { - USED_TOKEN(CALL_VERBATIM); current_call_convention(cbl_call_verbatim_e); } ; @@ -553,13 +575,11 @@ except_name: EXCEPTION_NAME[ec] { except_check: CHECKING on { $$ = exception_turn.enable(true); } | CHECKING OFF { - USED_TOKEN(OFF); $$ = exception_turn.enable(false); } | CHECKING on with LOCATION { - USED_TOKEN(LOCATION); - $$ = exception_turn.enable(true, true); + $$ = exception_turn.enable(true, true); } ; @@ -579,7 +599,6 @@ filenames: filename { ; filename: NAME { - USED_TOKEN(NAME); if( ($$ = cdf_file(PROGRAM, $1)) == nullptr ) { error_msg(@NAME, "invalid file name '%s'", $NAME); YYERROR; @@ -590,17 +609,16 @@ filename: NAME cdf_if: CDF_IF cdf_cond_expr { scanner_parsing(parser::token::YDF_CDF_IF, $2); } - | CDF_IF error { - } CDF_END_IF { // not pushed, don't pop - USED_TOKEN(CDF_END_IF); - if( ! scanner_parsing() ) YYACCEPT; + | CDF_IF error CDF_END_IF { // not pushed, don't pop + if( ! scanner_parsing() ) { + SAVE_LOOKAHEAD; + YYACCEPT; + } } ; cdf_evaluate: CDF_EVALUATE cdf_expr - | CDF_EVALUATE TRUE_kw { - USED_TOKEN(TRUE_kw); - } + | CDF_EVALUATE TRUE_kw ; cdf_eval_when: CDF_WHEN cdf_eval_obj @@ -608,17 +626,12 @@ cdf_eval_when: CDF_WHEN cdf_eval_obj cdf_eval_obj: cdf_cond_expr | cdf_expr THRU cdf_expr - | OTHER { - USED_TOKEN(OTHER); - } + | OTHER ; -cdf_cond_expr: BOOL { - USED_TOKEN(BOOL); - } +cdf_cond_expr: BOOL | NAME DEFINED { - USED_TOKEN(DEFINED); cdf_values_t& dictionary( cdf_dictionary() ); auto p = dictionary.find($1); bool found = p != dictionary.end(); @@ -635,8 +648,7 @@ cdf_cond_expr: BOOL { } | cdf_bool_expr { $$ = $1(@1) == 0? false : true; } | FEATURE DEFINED { - USED_TOKEN(DEFINED); - const auto& feature($1); + const auto& feature($1); $$ = (feature == int(feature & cbl_gcobol_features)); dbgmsg("CDF: feature 0x%02x is %s", $1, $$? "ON" : "OFF"); } @@ -703,8 +715,7 @@ cdf_expr: cdf_expr '+' cdf_expr { $$ = $1(@1) + $3(@3); } ; cdf_factor: NAME { - USED_TOKEN(NAME); - cdf_values_t& dictionary( cdf_dictionary() ); + cdf_values_t& dictionary( cdf_dictionary() ); auto that = dictionary.find($1); if( that != dictionary.end() ) { $$ = that->second; @@ -720,16 +731,13 @@ cdf_factor: NAME { } | NUMBER { - USED_TOKEN(NUMBER); - $$ = cdfval_t($1); + $$ = cdfval_t($1); } | LITERAL { - USED_TOKEN(LITERAL); - $$ = cdfval_t($1); + $$ = cdfval_t($1); } | NUMSTR { - USED_TOKEN(NUMSTR); - auto value = integer_literal($NUMSTR); + auto value = integer_literal($NUMSTR); if( !value.second ) { error_msg(@1, "CDF error: parsed %qs as %lld", $NUMSTR, value.first); @@ -801,21 +809,18 @@ replace_by: name_any[a] BY name_any[b] suppress: %empty | SUPPRESS { - USED_TOKEN(SUPPRESS); - copybook.suppress(); + copybook.suppress(); } ; name_any: namelit | PSEUDOTEXT { - USED_TOKEN(PSEUDOTEXT); - $$ = cdf_arg_t{parser::token::YDF_PSEUDOTEXT, $1}; + $$ = cdf_arg_t{parser::token::YDF_PSEUDOTEXT, $1}; } ; name_one: NAME { - USED_TOKEN(NAME); cdf_values_t& dictionary( cdf_dictionary() ); cdf_arg_t arg = { parser::token::YDF_NAME, $1 }; auto p = dictionary.find($1); @@ -826,12 +831,10 @@ name_one: NAME $$ = arg; } | NUMSTR { - USED_TOKEN(NUMSTR); - $$ = cdf_arg_t{parser::token::YDF_NUMSTR, $1}; + $$ = cdf_arg_t{parser::token::YDF_NUMSTR, $1}; } | LITERAL { - USED_TOKEN(LITERAL); - $$ = cdf_arg_t{parser::token::YDF_LITERAL, $1}; + $$ = cdf_arg_t{parser::token::YDF_LITERAL, $1}; } ; @@ -859,13 +862,10 @@ namelit: name | LITERAL { $$ = cdf_arg_t{parser::token::YDF_LITERAL, $1}; } ; -name: NAME { - USED_TOKEN(NAME); - } +name: NAME | name inof NAME { - USED_TOKEN(NAME); - char *s = xasprintf( "%s %s %s", $1, $2, $3 ); + char *s = xasprintf( "%s %s %s", $1, $2, $3 ); assert($$ == $1); free(const_cast<char*>($1)); free(const_cast<char*>($3)); @@ -873,12 +873,10 @@ name: NAME { } ; inof: IN { - USED_TOKEN(IN); - static const char in[] = "IN"; $$ = in; + static const char in[] = "IN"; $$ = in; } | OF { - USED_TOKEN(OF); - static const char of[] = "OF"; $$ = of; + static const char of[] = "OF"; $$ = of; } ; @@ -891,13 +889,10 @@ subscripts: subscript $$ = s; } ; -subscript: SUBSCRIPT { - USED_TOKEN(SUBSCRIPT); - } +subscript: SUBSCRIPT | LSUB subscript RSUB { - USED_TOKEN(RSUB); - char *s = xasprintf( "%s%s%s", $1, $2, $3 ); + char *s = xasprintf( "%s%s%s", $1, $2, $3 ); free(const_cast<char*>($1)); free(const_cast<char*>($2)); free(const_cast<char*>($3)); @@ -906,21 +901,15 @@ subscript: SUBSCRIPT { ; as: %empty - | AS { - USED_TOKEN(AS); - } + | AS ; on: %empty - | ON { - USED_TOKEN(ON); - } + | ON ; with: %empty - | WITH { - USED_TOKEN(WITH); - } + | WITH ; %% diff --git a/gcc/cobol/gengen.cc b/gcc/cobol/gengen.cc index e41302c5ef7..8dddf8f0103 100644 --- a/gcc/cobol/gengen.cc +++ b/gcc/cobol/gengen.cc @@ -918,9 +918,68 @@ gg_declare_variable(tree type_decl, return var_decl; } +class DECL_COUNTER + { + public: + unsigned long total; + unsigned long artificial; + unsigned long addressable; + unsigned long art_addr; + + DECL_COUNTER() + : total( 0 ), + artificial( 0 ), + addressable( 0 ), + art_addr( 0 ) + { + } + + void + count( tree var_decl ) + { + if( TREE_CODE( var_decl ) == VAR_DECL ) + { + total += 1; + + bool is_artificial = DECL_ARTIFICIAL( var_decl ); + bool is_addressable = TREE_ADDRESSABLE( var_decl ); + + if( is_artificial ) + { + artificial += 1; + } + + if( is_addressable ) + { + addressable += 1; + } + + if( is_artificial && is_addressable ) + { + art_addr += 1; + } + } + } + + ~DECL_COUNTER() + { + if( getenv("VAR_DECL_COUNT") ) + { + fprintf( stderr, "decl_counter VAR_DECL = %lu\n", total ); + fprintf( stderr, "decl_counter ARTIFICIAL = %lu\n", artificial ); + fprintf( stderr, "decl_counter ADDRESSABLE = %lu\n", addressable ); + fprintf( stderr, "decl_counter BOTH = %lu\n", art_addr ); + } + } + }; + +static DECL_COUNTER decl_counter; + tree gg_define_from_declaration(tree var_decl) { + decl_counter.count(var_decl); + // Append the var_decl to either the chain for the current function or for // the translation_unit, depending on the var_decl's context: gg_append_var_decl(var_decl); diff --git a/gcc/cobol/parse.y b/gcc/cobol/parse.y index 928380729b8..20fa4971389 100644 --- a/gcc/cobol/parse.y +++ b/gcc/cobol/parse.y @@ -588,8 +588,8 @@ class locale_tgt_t { MINN "Min" MULTIPLE MOD MODE MODULE_NAME "MODULE-NAME " - NAMED NAT NATIONAL - NATIONAL_EDITED "NATIONAL-EDITED" + NAMED NAMESPACE NAMESPACE_PREFIX "NAMESPACE-PREFIX" + NAT NATIONAL NATIONAL_EDITED "NATIONAL-EDITED" NATIONAL_OF "NATIONAL-OF" NATIVE NESTED NEXT NO NOTE @@ -702,6 +702,7 @@ class locale_tgt_t { UNDERLINE UNSIGNED_kw UTF_16 "UTF-16" UTF_8 "UTF-8" + XML_DECLARATION "XML-DECLARATION" XMLGENERATE "XML GENERATE" XMLPARSE "XML PARSE" @@ -737,7 +738,8 @@ class locale_tgt_t { %type <number> star_cbl_opt close_how %type <number> test_before usage_clause1 might_be alphanational -%type <boolean> all optional sign_leading on_off initialized strong is_signed +%type <boolean> all filler initialized is_signed + on_off optional sign_leading strong %type <number> count data_clauses data_clause %type <number> nine nines nps relop spaces_etc reserved_value signed %type <number> variable_type binary_type @@ -1281,7 +1283,7 @@ class locale_tgt_t { MIGHT_BE MINN MULTIPLE MOD MODE MODULE_NAME - NAMED NAMESPACE NAMESPACE_PREFIX "NAMESPACE-PREFIX" + NAMED NAMESPACE NAMESPACE_PREFIX NAT NATIONAL NATIONAL_EDITED NATIONAL_OF @@ -1355,7 +1357,7 @@ class locale_tgt_t { VALUE VARIANCE VARYING VOLATILE WHEN_COMPILED WITH WORKING_STORAGE - XML_DECLARATION "XML-DECLARATION" + XML_DECLARATION YEAR_TO_YYYY YYYYDDD YYYYMMDD ZERO @@ -4033,7 +4035,7 @@ level_name: LEVEL ctx_name } current_field($$); // make available for data_clauses } - | LEVEL + | LEVEL filler { switch($LEVEL) { case 66: @@ -4051,8 +4053,13 @@ level_name: LEVEL ctx_name struct cbl_field_t field = { FldInvalid, capacity_cast($LEVEL), @LEVEL.first_line }; - - $$ = field_add(@1, &field); + cbl_loc_t loc(@LEVEL); + if( $filler ) { + loc.last_column += 7; + field.set_attr(filler_e); + strcpy(field.name, "FILLER"); + } + $$ = field_add(loc, &field); if( !$$ ) { YYERROR; } @@ -12254,6 +12261,10 @@ file: %empty | FILE_KW ; +filler: %empty { $$ = false; } + | FILLER_kw { $$ = true; } + ; + first_last: %empty { $$ = 0; } | FIRST { $$ = 'F'; } | LAST { $$ = 'L'; } @@ -13095,10 +13106,11 @@ current_tokens_t::tokenset_t::find( const cbl_name_t name, bool include_intrinsi * 1. an intrinsic function name (OK if include_intrinsics) * 2. an ISO/GCC reserved word or context-sensitive word (OK) * 3. a token in our token list for convenience, such as BINARY_INTEGER (bzzt) - */ - + */ cbl_name_t lname; std::transform(name, name + strlen(name) + 1, lname, ftolower); + dbgmsg("current_tokens_t::tokenset_t::find: scanning %lu tokens for '%s'", + (unsigned long)tokens.size(), lname); auto p = tokens.find(lname); if( p == tokens.end() ) return 0; int token = p->second; diff --git a/gcc/cobol/scan.l b/gcc/cobol/scan.l index 1258b9ac877..27de05d2ac7 100644 --- a/gcc/cobol/scan.l +++ b/gcc/cobol/scan.l @@ -79,6 +79,7 @@ nonseq (([''][[:alnum:]]+][''])|([""][[:alnum:]]+[""])) INTEGER 0*[1-9][[:digit:]]* INTEGERZ [[:digit:]]+ +NOTWORD [^[:alnum:]$_-] NONWORD [^[:alnum:]$_-]+ SPC [[:space:]]+ @@ -167,10 +168,9 @@ NAMEQUALS {NAMEQUAL}({SPC}{NAMEQUAL})* STRING [^\r\n""]+ STRING1 [^\r\n'']+ - QSTRING1 ['']{STRING1}(['']{2}{STRING1})*[''] QSTRING2 [""]{STRING}([""]{2}{STRING})*[""] - + /* comma & semicolon must be followed by a space */ COMMA [,;][[:blank:]]* @@ -199,7 +199,7 @@ LINE_DIRECTIVE ^[#]line{SPC}[[:alnum:]]+{SPC}[""''].+\n %x procedure_div ident_state addr_of function classify %x program_id_state comment_entries -%x date_state field_level field_state dot_state +%x date_state field_state fd_state field_data_item dot_state %x numeric_state name_state %x quoted1 quoted2 quoteq %x picture picture_count integer_count @@ -258,7 +258,7 @@ LOCAL-STORAGE{SPC}SECTION { yy_push_state(field_state); return LOCAL_STORAGE_SECT; } WORKING-STORAGE { return WORKING_STORAGE; } -LOCAL-STORAGE { return LOCAL_STORAGE; } +LOCAL-STORAGE { return LOCAL_STORAGE; } SCREEN { return SCREEN; } LINKAGE{SPC}SECTION { @@ -273,8 +273,11 @@ PROGRAM-ID{OSPC}{DOTSEP}? { yy_push_state(ident_state); yy_push_state(program_id_state); yy_push_state(name_state); return PROGRAM_ID; } -PROCEDURE{SPC}DIVISION { yy_push_state(procedure_div); +<*>{ + PROCEDURE{SPC}DIVISION { yy_push_state(procedure_div); return PROCEDURE_DIV; } +} + <comment_entries>{ (ENVIRONMENT|DATA|PROCEDURE){SPC}DIVISION { myless(0); yy_pop_state(); } {BLANK_EOL} @@ -813,13 +816,10 @@ FINAL { return FINAL; } FILE-LIMIT { return FILE_LIMIT; } FILE-CONTROL { return FILE_CONTROL; } -FILE{SPC}SECTION { return FILE_SECT; } +FILE{SPC}SECTION { BEGIN(field_state); return FILE_SECT; } FILE { return FILE_KW; } -FD { return FD; } -SD { return SD; } - EXTERNAL { return EXTERNAL; } EXIT { return EXIT; } EXHIBIT { return EXHIBIT; } @@ -896,7 +896,10 @@ DATE-WRITTEN { return DATE_WRITTEN; } DATE-COMPILED { return DATE_COMPILED; } DAY-OF-WEEK { return DAY_OF_WEEK; } -DATA{SPC}DIVISION{DOTEOL} { return DATA_DIV; } +DATA{SPC}DIVISION{DOTEOL} { + yy_push_state(field_state); + return DATA_DIV; } + DATA { return DATA; } CURRENCY { return CURRENCY; } @@ -977,72 +980,221 @@ USE({SPC}FOR)? { return USE; } } -<field_level>{ - 66/{SPC}(\f#)?{NAME} { yy_pop_state(); - if( !parsing.on() ) orig_picture[0] = '\0'; - if( level_needed() ) { - level_found(); - yylval.number = level_of(yytext); return LEVEL66; - } else { - return numstr_of(yytext); - } - } - 78/{SPC}(\f#)?{NAME} { yy_pop_state(); - if( !parsing.on() ) orig_picture[0] = '\0'; - if( level_needed() ) { - level_found(); - yylval.number = level_of(yytext); return LEVEL78; - } else { - return numstr_of(yytext); - } - } - 88/{SPC}(\f#)?{NAME} { yy_pop_state(); - if( !parsing.on() ) orig_picture[0] = '\0'; - if( level_needed() ) { - level_found(); - yylval.number = level_of(yytext); return LEVEL88; - } else { - return numstr_of(yytext); - } - } - [[:digit:]]{1,2}/[[:space:]] { yy_pop_state(); - if( !parsing.on() ) orig_picture[0] = '\0'; - if( level_needed() ) { - level_found(); - yylval.number = level_of(yytext); return LEVEL; - } else { - return numstr_of(yytext); - } - } - - . { cbl_errx( "failed to parse field level on line %d", yylineno); } +<field_state,field_data_item>{ + WORKING-STORAGE{SPC}SECTION { return WORKING_STORAGE_SECT; } + LOCAL-STORAGE{SPC}SECTION { return LOCAL_STORAGE_SECT; } + LINKAGE{SPC}SECTION { return LINKAGE_SECT; } + SCREEN{SPC}/SECTION { return SCREEN; } + SECTION{OSPC}/{DOTSEP} { yylval.string = NULL; return SECTION; } + + ALL { return ALL; } + AS { return AS; } + ASCENDING { return ASCENDING; } + BLANK { return BLANK; } + BLOCK { return BLOCK_kw; } + BY { return BY; } + BYTE-LENGTH { return BYTE_LENGTH; } + CHARACTER { return CHARACTER; } + CHARACTERS { return CHARACTERS; } + CODE-SET { return CODESET; } + CONSTANT { return CONSTANT; } + CONTAINS { return CONTAINS; } + DATA { return DATA; } + DEPENDING { return DEPENDING; } + DESCENDING { return DESCENDING; } + DISPLAY { return DISPLAY; } + EBCDIC { return EBCDIC; } + EJECT{DOTEOL}? { + dialect_ok(yylloc, IbmEjectE, "EJECT"); + auto len = yyleng - 1; + if( yytext[len] == '\f' ) myless(--len); + } + EXTERNAL { return EXTERNAL; } + FALSE { return FALSE_kw; } + FROM { return FROM; } + GLOBAL { return GLOBAL; } + IN { return IN; } + INDEXED { return INDEXED; } + IS { return IS; } + JUST(IFIED)?({SPC}RIGHT)? { return JUSTIFIED; } + KEY { return KEY; } + LABEL { return LABEL; } + LEADING { return LEADING; } + LEFT { return LEFT; } + MODE { return MODE; } + NATIVE { return NATIVE; } + NO { return NO; } + OCCURS/{SPC}{NAME} { return OCCURS; } + OCCURS { yy_push_state(integer_count); return OCCURS; } + OF { return OF; } + OMITTED { return OMITTED; } + ON { return ON; } + RECORD { return RECORD; } + RECORDING { return RECORDING; } + RECORDS { return RECORDS; } + RECORDS{SPC}ARE { return RECORDS; } + RECORD{SPC}IS { return RECORD; } + REDEFINES { return REDEFINES; } + RENAMES { return RENAMES; } + RIGHT { return RIGHT; } + SEPARATE { return SEPARATE; } + SET { return SET; } + SAME { return SAME; } + SIGN { return SIGN; } + SIZE { return SIZE; } + STANDARD { return STANDARD; } + STANDARD{SPC}ALPHABET { return STANDARD_ALPHABET; } + STRONG { return STRONG; } + SYNC(HRONIZED)? { return SYNCHRONIZED; } + TIMES { return TIMES; } + TIMES[[:space::]]+DEPENDING { return DEPENDING; } + TO { return TO; } + TRAILING { return TRAILING; } + TRUE { return TRUE_kw; } + TYPE { return TYPE; } + TYPEDEF { return TYPEDEF; } + VARYING { return VARYING; } + VOLATILE { return VOLATILE; } + WHEN { return WHEN; } + WITH { return WITH; } + + COPY { + yy_push_state(copy_state); + myless(0); + } + + PROCEDURE{SPC}DIVISION { BEGIN(procedure_div); return PROCEDURE_DIV; } } +<fd_state>{ + ALL { return ALL; } + AS { return AS; } + ASCENDING { return ASCENDING; } + BLANK { return BLANK; } + BLOCK { return BLOCK_kw; } + BY { return BY; } + BYTE-LENGTH { return BYTE_LENGTH; } + CHARACTER { return CHARACTER; } + CHARACTERS { return CHARACTERS; } + CODE-SET { return CODESET; } + CONSTANT { return CONSTANT; } + CONTAINS { return CONTAINS; } + DATA { return DATA; } + DEPENDING { return DEPENDING; } + DESCENDING { return DESCENDING; } + DISPLAY { return DISPLAY; } + EBCDIC { return EBCDIC; } + EJECT{DOTEOL}? { + dialect_ok(yylloc, IbmEjectE, "EJECT"); + auto len = yyleng - 1; + if( yytext[len] == '\f' ) myless(--len); + } + EXTERNAL { return EXTERNAL; } + FALSE { return FALSE_kw; } + FROM { return FROM; } + GLOBAL { return GLOBAL; } + IN { return IN; } + INDEXED { return INDEXED; } + IS { return IS; } + JUST(IFIED)?({SPC}RIGHT)? { return JUSTIFIED; } + KEY { return KEY; } + LABEL { return LABEL; } + LEADING { return LEADING; } + LEFT { return LEFT; } + MODE { return MODE; } + NATIVE { return NATIVE; } + NO { return NO; } + OCCURS { return OCCURS; } + OF { return OF; } + OMITTED { return OMITTED; } + ON { return ON; } + RECORD { return RECORD; } + RECORDING { return RECORDING; } + RECORDS { return RECORDS; } + RECORDS{SPC}ARE { return RECORDS; } + RECORD{SPC}IS { return RECORD; } + REDEFINES { return REDEFINES; } + RENAMES { return RENAMES; } + RIGHT { return RIGHT; } + SEPARATE { return SEPARATE; } + SET { return SET; } + SAME { return SAME; } + SIGN { return SIGN; } + SIZE { return SIZE; } + STANDARD { return STANDARD; } + STANDARD{SPC}ALPHABET { return STANDARD_ALPHABET; } + STRONG { return STRONG; } + SYNC(HRONIZED)? { return SYNCHRONIZED; } + TIMES { return TIMES; } + TIMES[[:space::]]+DEPENDING { return DEPENDING; } + TO { return TO; } + TRAILING { return TRAILING; } + TRUE { return TRUE_kw; } + TYPE { return TYPE; } + TYPEDEF { return TYPEDEF; } + VARYING { return VARYING; } + VOLATILE { return VOLATILE; } + WHEN { return WHEN; } + WITH { return WITH; } + + {INTEGERZ} { return numstr_of(yytext); } + + {NAME} { + yylval.string = xstrdup(yytext); + return NAME; + } + {QSTRING1} | + {QSTRING2} { + static const char prefix[] = ""; + char *s = unquote(); + yylval.literal.set(strlen(s), s, prefix); + return LITERAL; + } + + {DOTSEP} { BEGIN(field_state); return *yytext; } +} + + + /* + * Field state begins with a Data Division section and ends with + * Procedure Division. The work is done by field_data_item. + */ <field_state>{ - ^[[:blank:]]*[[:digit:]]{1,2}{OSPC}/[.] { - if( !parsing.on() ) orig_picture[0] = '\0'; - level_found(); - yylval.number = level_of(yytext); - return LEVEL; - } - - ^[[:blank:]]+ {} - ^[[:digit:]]{1,2}[[:space:]] { yy_push_state(field_level); } - [[:blank:]]*/[[:digit:]]{1,2}{SPC}(\f#)?{NAME} { yy_push_state(field_level); } - - [+-]?{INTEGERZ} { return numstr_of(yytext); } - [+-]?{dfc} { char *s = xstrdup(yytext); - // "The decimal point can appear anywhere within - // the literal except as the rightmost character." - size_t len = strlen(s); - assert(len); - if( s[--len] == '.' ) { - s[len] = '\0'; - myless(len); - } - numstr_of(s); free(s); - return NUMSTR; - } + FD { yy_push_state(fd_state); return FD; } + SD { yy_push_state(fd_state); return SD; } + {NAME} { + ydflval.string = yylval.string = xstrdup(yytext); + return NAME; + } + + [[:digit:]]{1,2}/{NOTWORD} { + yy_push_state(field_data_item); + yylval.number = level_of(yytext); + + switch(yylval.number) { + case 66: return LEVEL66; + case 78: return LEVEL78; + case 88: return LEVEL88; + } + return LEVEL; + } +} +<field_data_item>{ + {DOTSEP} { yy_pop_state(); return '.'; } + + [+-]?{INTEGERZ} { return numstr_of(yytext); } + [+-]?{dfc} { char *s = xstrdup(yytext); + // "The decimal point can appear anywhere within + // the literal except as the rightmost character." + size_t len = strlen(s); + assert(len); + if( s[--len] == '.' ) { + s[len] = '\0'; + myless(len); + } + numstr_of(s); free(s); + return NUMSTR; + } PIC(TURE)?({SPC}IS)?{SPC}{PICTURE} { auto pos = validate_picture(); @@ -1131,85 +1283,7 @@ USE({SPC}FOR)? { return USE; } yy_push_state(numeric_state); return THRU; } - ALL { return ALL; } - AS { return AS; } - ASCENDING { return ASCENDING; } - BLANK { return BLANK; } - BLOCK { return BLOCK_kw; } - BY { return BY; } - BYTE-LENGTH { return BYTE_LENGTH; } - CHARACTER { return CHARACTER; } - CHARACTERS { return CHARACTERS; } - CODE-SET { return CODESET; } - CONSTANT { return CONSTANT; } - CONTAINS { return CONTAINS; } - DATA { return DATA; } - DEPENDING { return DEPENDING; } - DESCENDING { return DESCENDING; } - DISPLAY { return DISPLAY; } - EBCDIC { return EBCDIC; } - EJECT{DOTEOL}? { - dialect_ok(yylloc, IbmEjectE, "EJECT"); - auto len = yyleng - 1; - if( yytext[len] == '\f' ) myless(--len); - } - EXTERNAL { return EXTERNAL; } - FALSE { return FALSE_kw; } - FROM { return FROM; } - GLOBAL { return GLOBAL; } - IN { return IN; } - INDEXED { return INDEXED; } - IS { return IS; } - JUST(IFIED)?({SPC}RIGHT)? { return JUSTIFIED; } - KEY { return KEY; } - LABEL { return LABEL; } - LEADING { return LEADING; } - LEFT { return LEFT; } - MODE { return MODE; } - NATIVE { return NATIVE; } - NO { return NO; } - OCCURS/{SPC}{NAME} { return OCCURS; } - OCCURS { yy_push_state(integer_count); return OCCURS; } - OF { return OF; } - OMITTED { return OMITTED; } - ON { return ON; } - RECORD { return RECORD; } - RECORDING { return RECORDING; } - RECORDS { return RECORDS; } - RECORDS{SPC}ARE { return RECORDS; } - RECORD{SPC}IS { return RECORD; } - REDEFINES { return REDEFINES; } - RENAMES { return RENAMES; } - RIGHT { return RIGHT; } - SEPARATE { return SEPARATE; } - SET { return SET; } - SAME { return SAME; } - SIGN { return SIGN; } - SIZE { return SIZE; } - STANDARD { return STANDARD; } - STANDARD{SPC}ALPHABET { return STANDARD_ALPHABET; } - STRONG { return STRONG; } - SYNC(HRONIZED)? { return SYNCHRONIZED; } - TIMES { return TIMES; } - TIMES[[:space::]]+DEPENDING { return DEPENDING; } - TO { return TO; } - TRAILING { return TRAILING; } - TRUE { return TRUE_kw; } - TYPE { return TYPE; } - TYPEDEF { return TYPEDEF; } - VARYING { return VARYING; } - VOLATILE { return VOLATILE; } - WHEN { return WHEN; } - WITH { return WITH; } - - COPY { - yy_push_state(copy_state); - myless(0); - } - - FD/[[:blank:]]+ { parsing.need_level(false); return FD; } - SD/[[:blank:]]+ { parsing.need_level(false); return SD; } - + FILLER { return FILLER_kw; } {NAME} { // NAME here might be a token alias int token; if( 0 != (token = binary_integer_usage(yytext)) ) { @@ -1229,17 +1303,6 @@ USE({SPC}FOR)? { return USE; } yy_push_state(hex_state); } N?X{nonseq} { dbgmsg("invalid hexadecimal value: %s", yytext); return NO_CONDITION; } - [[:blank:]]*{EOL} {} - - WORKING-STORAGE{SPC}SECTION { return WORKING_STORAGE_SECT; } - LOCAL-STORAGE{SPC}SECTION { return LOCAL_STORAGE_SECT; } - LINKAGE{SPC}SECTION { return LINKAGE_SECT; } - SCREEN{SPC}/SECTION { return SCREEN; } - SECTION{OSPC}/{DOTSEP} { yylval.string = NULL; return SECTION; } - - PROCEDURE{SPC}DIVISION { BEGIN(procedure_div); return PROCEDURE_DIV; } - - [*]>.*$ // ignore inline comment } <numstr_state>{ @@ -1287,9 +1350,8 @@ USE({SPC}FOR)? { return USE; } {COMMA} | [[:blank:]]*{EOL} | [[:blank:]]+{EOL}? { yy_pop_state(); /* embedded/trailing blank */ } - {DOTSEP}[[:blank:].]+$ { yy_pop_state(); return '.'; } - {DOTSEP} { yy_pop_state(); return '.'; } - + {DOTSEP}[[:blank:].]+$ { yy_pop_state(); myless(0); } + {DOTSEP} { yy_pop_state(); myless(0); } [[:blank:]]+[-+]/{EDITED} { return picset(yytext[yyleng-1]); } @@ -1749,6 +1811,14 @@ USE({SPC}FOR)? { return USE; } ZERO | ZEROES | ZEROS { return ZERO; } + + CONFIGURATION{SPC}SECTION { return CONFIGURATION_SECT; } + FILE{SPC}SECTION { return FILE_SECT; } + INPUT-OUTPUT{SPC}SECTION { return INPUT_OUTPUT_SECT; } + LINKAGE{SPC}SECTION { return LINKAGE_SECT; } + LOCAL-STORAGE{SPC}SECTION { return LOCAL_STORAGE_SECT; } + WORKING-STORAGE{SPC}SECTION { return WORKING_STORAGE_SECT; } + } <*>{ @@ -1787,13 +1857,16 @@ USE({SPC}FOR)? { return USE; } {NAME} { ydflval.string = yylval.string = xstrdup(yytext); return NAME; } + + {QSTRING1} | + {QSTRING2} { ydflval.string = unquote(); return LITERAL; } + %EBCDIC-MODE { ydflval.number = feature_internal_ebcdic_e; return FEATURE; } %64-BIT-POINTER { ydflval.number = feature_embiggen_e; return FEATURE; } [[:blank:]]+ {BLANK_EOL} - . { myless(0); yy_pop_state(); } // not a CDF token } <program_id_state>{ @@ -2434,12 +2507,12 @@ BASIS { yy_push_state(basis); return BASIS; } // If WITH DEBUGGING MODE, drop the D, else drop the line. if( include_debug() ) myless(7); } - ^[ ]*>>{OBLANK}IF { yy_push_state(cdf_state); return CDF_IF; } + ^[ ]*>>{OBLANK}IF { PUSH_CDF_STATE; return CDF_IF; } ^[ ]*>>{OBLANK}ELSE { return CDF_ELSE; } ^[ ]*>>{OBLANK}END-IF { return CDF_END_IF; } ^[ ]*[$]{OBLANK}IF { dialect_ok(yylloc, MfCdfDollar, yytext); - yy_push_state(cdf_state); return CDF_IF; } + PUSH_CDF_STATE; return CDF_IF; } ^[ ]*[$]{OBLANK}ELSE { dialect_ok(yylloc, MfCdfDollar, yytext); return CDF_ELSE; } ^[ ]*[$]{OBLANK}END(-IF)? { dialect_ok(yylloc, MfCdfDollar, yytext); @@ -2447,7 +2520,7 @@ BASIS { yy_push_state(basis); return BASIS; } ^[ ]*[$]{OBLANK}SET({SPC}CONSTANT)? { dialect_ok(yylloc, MfCdfDollar, yytext); - yy_push_state(cdf_state); return CDF_DEFINE; } + PUSH_CDF_STATE; return CDF_DEFINE; } ^[ ]*>>{OBLANK}EVALUATE { return CDF_EVALUATE; } ^[ ]*>>{OBLANK}WHEN { return CDF_WHEN; } @@ -2457,9 +2530,10 @@ BASIS { yy_push_state(basis); return BASIS; } ^[ ]*>>{OBLANK}CALL-CONVENTION{BLANK}COBOL { return CALL_COBOL; } ^[ ]*>>{OBLANK}CALL-CONVENTION{BLANK}VERBATIM { return CALL_VERBATIM; } - ^[ ]*>>{OBLANK}DEFINE { yy_push_state(cdf_state); return CDF_DEFINE; } - ^[ ]*>>{OBLANK}DISPLAY { return CDF_DISPLAY; } - ^[ ]*>>{OBLANK}TURN { yy_push_state(exception); return TURN; } + ^[ ]*>>{OBLANK}DEFINE { PUSH_CDF_STATE; return CDF_DEFINE; } + ^[ ]*>>{OBLANK}DISPLAY { PUSH_CDF_STATE; return CDF_DISPLAY; } + + ^[ ]*>>{OBLANK}TURN { yy_push_state(exception); return TURN; } ^[ ]*>>{OBLANK}COBOL-WORDS { yy_push_state(cobol_words); return COBOL_WORDS; } ^[ ]*>>{OBLANK}SOURCE{BLANK}FORMAT { return SOURCE_FORMAT; } @@ -2555,17 +2629,6 @@ BASIS { yy_push_state(basis); return BASIS; } <*>{ {NAME} { - int token = keyword_tok(yytext); - if( token ) { - if(yy_flex_debug && YY_START) { - dbgmsg("missed token %s in start condition %d", - yytext, YY_START); - } - // Do not return "token" because it may have been excluded - // by a start condition. For example, REM might be a name, - // but is the name of an intrinsic function, which would - // appear only after FUNCTION. - } if( is_integer_token() ) return numstr_of(yytext); ydflval.string = yylval.string = xstrdup(yytext); return typed_name(yytext); diff --git a/gcc/cobol/scan_ante.h b/gcc/cobol/scan_ante.h index 4c147cb994e..74b5ddd5d30 100644 --- a/gcc/cobol/scan_ante.h +++ b/gcc/cobol/scan_ante.h @@ -351,9 +351,7 @@ static bool level_needed() { return scanner_normal() && parsing.need_level(); } -static void level_found() { - if( scanner_normal() ) parsing.need_level(false); -} +#define PUSH_CDF_STATE if( YY_START != cdf_state ) yy_push_state(cdf_state) /* * Return all but the first N characters, to be rescanned by the nexst yylex. @@ -539,7 +537,7 @@ trim_location( int nkeep) { gcc_assert( yylloc.first_line <= yylloc.last_line ); gcc_assert( 0 < yylloc.last_column ); - location_dump(__func__, __LINE__, "yylloc", yylloc, true); + ////location_dump(__func__, __LINE__, "yylloc", yylloc, true); } static void @@ -1145,8 +1143,31 @@ symbol_function_token( const char name[] ) { bool in_procedure_division(void); bool in_environment_division(void ); +static const char * +symbol_lower_name( const symbol_elem_t *e, cbl_name_t lname ) { + const char *pname = nullptr; + + switch(e->type) { + case SymFile: + pname = cbl_file_of(e)->name; + break; + case SymField: + pname = cbl_field_of(e)->name; + break; + case SymLabel: + pname = cbl_label_of(e)->name; + break; + default: + return nullptr; + } + gcc_assert(pname); + std::fill(lname, lname + sizeof(cbl_name_t), '\0'); + std::transform( pname, pname + strlen(pname) + 1, lname, tolower ); + return lname; +} + static symbol_elem_t * -symbol_exists( const char name[] ) { +symbol_exists( const char name[], const symbol_elem_t *constant = nullptr ) { typedef std::map <std::string, size_t> name_cache_t; static std::map <size_t, name_cache_t> cachemap; @@ -1154,33 +1175,40 @@ symbol_exists( const char name[] ) { std::transform( name, name + strlen(name) + 1, lname, tolower ); auto& cache = cachemap[PROGRAM]; - if( in_procedure_division() && cache.empty() ) { - for( auto e = symbols_begin(PROGRAM) + 1; - e < symbols_end() && PROGRAM == e->program; e++ ) { - if( e->type == SymFile ) { - cbl_file_t *f(cbl_file_of(e)); - cbl_name_t lname; - std::transform( f->name, f->name + strlen(f->name) + 1, lname, tolower ); - cache[lname] = symbol_index(e); - continue; - } - if( e->type == SymField ) { - auto f(cbl_field_of(e)); - cbl_name_t lname; - std::transform( f->name, f->name + strlen(f->name) + 1, lname, tolower ); - cache[lname] = symbol_index(e); + if( in_procedure_division() ) { + if( cache.empty() ) { + for( auto e = symbols_begin(PROGRAM) + 1; + e < symbols_end() && PROGRAM == e->program; e++ ) { + cbl_name_t name; + const char *pname = symbol_lower_name(e, name); + if( pname ) { + cache[pname] = symbol_index(e); + } } + cache.erase(""); // remove unnamed symbols + } + if( constant ) { // Johnny-come-lately named literal from CDF. + gcc_assert(constant->type == SymField); + cache[lname] = symbol_index(constant); } - cache.erase(""); + auto p = cache.find(lname); + if( p == cache.end() ) return nullptr; + return symbol_at(p->second); } - auto p = cache.find(lname); + /* + * Before data division has been defined and the cache populated, search the + * symbol table in case of named literal. + */ + symbol_elem_t *e = symbol_field( PROGRAM, 0, name ); + return e; +} - if( p == cache.end() ) { - symbol_elem_t * e = symbol_field( PROGRAM, 0, name ); - return e; +void +scanner_cache_update( const symbol_elem_t *elem ) { + if( elem->type == SymField ) { + auto f = cbl_field_of(elem); + symbol_exists(f->name, elem); } - - return symbol_at(p->second); } static int @@ -1419,3 +1447,29 @@ is_refmod() { int(yystr.size()), yystr.c_str(), colon_at_depth1? "" : "not "); return colon_at_depth1; } + +/* + * Given a QSTRING in yytext, remove leading and trailing quotes, and undouble + * any in the body of the string. + */ +static char* +unquote() { + char *p = yytext, *pend = yytext + yyleng; + char *output = xstrdup(yytext), quote = *p; + char prior = '\0'; + + pend = std::copy_if( ++p, --pend, output, + [quote, &prior]( char ch ) { + if( ch == quote ) { + if( ch == prior ) { + prior = '\0'; + return false; + } + prior = ch; + } + return true; + } ); + *pend = '\0'; + return output; +} + diff --git a/gcc/cobol/scan_post.h b/gcc/cobol/scan_post.h index 67a484ef80c..3055650c1f2 100644 --- a/gcc/cobol/scan_post.h +++ b/gcc/cobol/scan_post.h @@ -45,7 +45,7 @@ start_condition_str( int sc ) { case datetime_fmt: state = "datetime_fmt"; break; case dot_state: state = "dot_state"; break; case exception: state = "exception"; break; - case field_level: state = "field_level"; break; + case field_data_item: state = "field_data_item"; break; case field_state: state = "field_state"; break; case function: state = "function"; break; case hex_state: state = "hex_state"; break; @@ -234,8 +234,6 @@ run_cdf( int token ) { parsing.parser_restore(); - if( YY_START == cdf_state ) yy_pop_state(); - if( yy_flex_debug ) { dbgmsg("%s: CDF parser returned %d, scanner SC <%s>", __func__, erc, start_condition_is()); @@ -244,7 +242,6 @@ run_cdf( int token ) { return 0 == erc; } -#include <queue> struct recent_token_t { int token; YYSTYPE value; @@ -255,93 +252,16 @@ struct recent_token_t { recent_token_t( int token, cbl_loc_t loc ) : token(token), loc(loc) {} }; -#define RECENT(T) recent_token_t( (T), yylval ) namespace cdf { - int used_token(); + bool had_lookahead(); cbl_loc_t location(); + int last_lexed; } -static bool -same_end(const cbl_loc_t& a, const cbl_loc_t& b ) { - return a.last_line == b.last_line - && a.last_column == b.last_column; -} - -/* - * Two tokens are one and the same if they have the type and end at the same - * place. The scanner's idea of the current location might embrace more - * characters. - */ -static bool -same_token( const recent_token_t& a, const recent_token_t& b ) { - return a.token == b.token && same_end(a.loc, b.loc); -} - -/* - * Each time the generated scanner returns a token from lexer(), it's pushed on - * the recent_tokens queue. The queue is 2 deep, representing the CDF parser's - * current and lookahead tokens. - */ -static struct recent_tokens_t : protected std::queue<recent_token_t> -{ - const recent_token_t& operator<<(int token) { - while( 1 < size() ) pop(); - push( recent_token_t(token, yylval, yylloc) ); - return front(); - } - - bool eof() const { - return size() == 1 && front().token == YYEOF; - } - - int next() { - int end_token = cdf::used_token(); - - switch( size() ) { - default: gcc_unreachable(); - case 0: - dbgmsg("%s:%d: 0 recent tokens []", __func__, __LINE__); - break; - case 1: - dbgmsg("%s:%d: 1 recent token [%s]", __func__, __LINE__, - keyword_str(back().token)); - break; - case 2: - dbgmsg("%s:%d: 2 recent tokens [%s, %s]", __func__, __LINE__, - keyword_str(front().token), - keyword_str(back().token)); - break; - } - - /* - * Clear any used tokens. Return the unused token, if extant, which was - * the lookahead token. Because it was the last one returned by lexer(), - * the location is accurate. Else return the next token. - */ - recent_token_t ending(end_token, cdf::location()); - if( ! empty() && same_token(front(), ending) ) pop(); - if( ! empty() && same_token(back(), ending) ) c.clear(); - - end_token = 0; - - if( ! empty() ) { - end_token = front().token; - if( ! eof() ) { - pop(); - assert(empty()); - } - } - - dbgmsg("%s:%d: returning %s", __func__, __LINE__, keyword_str(end_token)); - return end_token; - } -} recent_tokens; - static int next_token() { - int token = lexer(); - recent_tokens << token; - return token; + cdf::last_lexed = lexer(); + return cdf::last_lexed; } // act on CDF tokens @@ -360,70 +280,86 @@ prelex() { while( is_cdf_token(token) ) { if( ! run_cdf(token) ) { - dbgmsg( ">>CDF parser failed, ydfchar %d", ydfchar() ); - } - - token = recent_tokens.next(); - - if( recent_tokens.eof() ) { - dbgmsg("lexer() returned EOF (%d)", token); - return YYEOF; + dbgmsg( ">>CDF parser failed, ydfchar %d", cdf::last_lexed ); } - if( token == 0 ) token = next_token(); + // The most recently lexed token was sent to the CDF. If it was a + // lookahead token, it now goes to the main parser, else we get the next + // one. + if( cdf::had_lookahead() ) { + token = cdf::last_lexed; + dbgmsg("%s: recovered lookahead token %s", __func__, keyword_str(token)); + } else { + token = next_token(); + } if( token == NO_CONDITION && parsing.at_eof() ) { dbgmsg("scanner at EOF, apparently (%d)", token); return YYEOF; } - dbgmsg("next token %s", keyword_str(token)); + dbgmsg("%s: next token %s", __func__, keyword_str(token)); // Reenter cdf parser only if next token could affect parsing state. if( ! parsing.on() && ! is_cdf_condition_token(token) ) break; } - if( yy_flex_debug ) { - dbgmsg("scanner SC <%s>", start_condition_is()); - } + // Do not disturb lexer state while we're not parsing the COBOL. + if( parsing.on() ) { + if( YY_START == cdf_state ) yy_pop_state(); - if( YY_START == copy_state || YY_START == cdf_state ) { - if( token == NAME ) { - auto tok = keyword_tok(ydflval.string); - if( tok ) token = tok; + if( yy_flex_debug ) { + dbgmsg("scanner SC <%s>", start_condition_is()); } - yy_pop_state(); - dbgmsg("scanner SC <%s>, token now %s", - start_condition_is(), keyword_str(token)); - } - /* - * The final, rejected CDF token might be a LEVEL number. - */ - if( YY_START == field_state && level_needed() ) { - switch( token ) { - case NUMSTR: - dbgmsg("final token is NUMSTR"); - yylval.number = level_of(yylval.numstr.string); - token = LEVEL; - break; - case cdf::parser::token::YDF_NUMBER: - dbgmsg("final token is YDF_NUMBER"); - yylval.number = ydflval.number; - token = LEVEL; - break; + switch(YY_START) { + case FILE_SECT: + case WORKING_STORAGE_SECT: + case LOCAL_STORAGE_SECT: + case LINKAGE_SECT: + BEGIN(field_state); + if( yy_flex_debug ) { + dbgmsg("scanner SC now <%s>", start_condition_is()); + } } - if( token == LEVEL ) { - switch(yylval.number) { - case 66: - token = LEVEL66; - break; - case 78: - token = LEVEL78; + + if( YY_START == copy_state || YY_START == cdf_state ) { + if( token == NAME ) { + auto tok = keyword_tok(ydflval.string); + if( tok ) token = tok; + } + yy_pop_state(); + dbgmsg("%s: scanner SC <%s>, token now %s", __func__, + start_condition_is(), keyword_str(token)); + } + + /* + * The unused CDF token might be a LEVEL number. + */ + if( YY_START == field_state && level_needed() ) { + switch( token ) { + case NUMSTR: + yylval.number = level_of(yylval.numstr.string); + token = LEVEL; break; - case 88: - token = LEVEL88; + case NUMBER: + yylval.number = ydflval.number; + token = LEVEL; break; } + if( token == LEVEL ) { + yy_push_state(field_data_item); // do as <field_state> does for LEVEL + switch(yylval.number) { + case 66: + token = LEVEL66; + break; + case 78: + token = LEVEL78; + break; + case 88: + token = LEVEL88; + break; + } + } } } @@ -454,30 +390,18 @@ prelex() { * Before invoking the CDF parser, run_cdf() stashes the CDF token with * inject_token(). When the CDF parser turns around and calls yylex, the first * token returned is that stashed one, because it is the reason the CDF parser - * was invoked. - * - * While the CDF parser is active, the scanner saves each token returned by - * lexer() in the recent_tokens queue. As the CDF parser uses a token, it - * records it such that it can be fetched with used_token(). A token fetched - * and *not* used was a lookahead token, and will be directed to the main - * parser. - * - * When the CDF parser returns, recent_tokens.next() compares the last two - * tokens fetched with the last one one used. If there's a recent token in the - * queue after the last one used, that's a lookahead token. It is returned to - * the main parser, which started the whole thing by calling yylex(). + * was invoked. The scanner saves each token returned by lexer() in + * cdf::last_lexed. * - * It's not really necessary to keep the last two tokens for the post-parsing - * logic to work. It's enough to compare the last fetched token with the last - * used token. + * When the CDF parser exits it records whether or not there was a lookahead + * token (else it was YYEOF or YYEMPTY). * - * It *is* really necessary to for the CDF parser to record the tokens it - * actually uses, at least as things stand. The Bison C++ parser (which the - * CDF parser uses) records any token values that it doesn't define as -2, - * "invalid token". If the lookahead token is, say, MOVE, it cannot be - * retrieved from the CDF parser. That's why it's kept in the recent_tokens - * queue, where it's preceded by the last valid, used token, as recorded by the - * CDF parser. + * When the CDF parser returns, the lexer calls cdf::had_lookahead() to see if + * there was a lookahead. (It's true only once because the function clears the + * condition.) If so, the lexer returns cdf::last_lexed to the main parser, + * which started the whole thing by calling yylex(). prelex() also compensates + * haphazardly for special tokens that depend on start condition, such as LEVEL + * versus NUMBER. */ int yylex(void) { @@ -504,6 +428,7 @@ yylex(void) { produce_next_sentence_target = false; } + int ntoken = -1; do { token = prelex(); if( yy_flex_debug ) { @@ -515,9 +440,14 @@ yylex(void) { keyword_str(token) ); } } - + ntoken++; // increment debug count here; might not loop to skip the token } while( token && ! parsing.feed_a_parser() ); + if( 0 < ntoken ) { + dbgmsg("%s:%d: ate %d tokens, now returning %s", __func__, __LINE__, + ntoken, keyword_str(token)); + } + if( next_sentence && token == '.' ) { produce_next_sentence_target = true; } @@ -525,7 +455,7 @@ yylex(void) { if( token == YYEOF && parsing.in_cdf() ) { if( yy_flex_debug) dbgmsg("deflecting EOF"); parsing.at_eof(true); - return NO_CONDITION; + token = NO_CONDITION; } return token; diff --git a/gcc/cobol/symbols.cc b/gcc/cobol/symbols.cc index 3ba1f84b735..1a223d71085 100644 --- a/gcc/cobol/symbols.cc +++ b/gcc/cobol/symbols.cc @@ -104,7 +104,7 @@ static struct symbol_table_t { struct symbol_elem_t *elems; - std::map<elem_key_t, size_t> specials; + std::map<elem_key_t, size_t> specials, programs, functions; std::map<elem_key_t, std::list<size_t>> labels; std::vector<symbol_pair_t> mappings; @@ -136,6 +136,15 @@ static struct symbol_table_t { const char *name = cbl_label_of(e)->name; labels[ elem_key_t(e->program, name) ].push_back( symbol_index(e) ); } + void program_add( size_t isym ) { label_add(programs, isym); } + void function_add( size_t isym ) { label_add(functions, isym); } + protected: + void label_add( std::map<elem_key_t, size_t>& M, size_t isym ) { + auto e = elems + isym; + auto L = cbl_label_of(e); + auto key( elem_key_t(e->program, L->name) ); + M[key] = isym; + } } symbols; static symbol_table_t& @@ -630,6 +639,13 @@ symbol_label_id( const cbl_label_t *label ) { struct cbl_label_t * symbol_program( size_t parent, const char name[], bool prototype ) { +#if 1 + auto key( elem_key_t(parent, name) ); + auto p = symbols.programs.find(key); + if( p == symbols.programs.end() ) return nullptr; + auto L = cbl_label_of(symbol_at(p->second)); + return L->prototype == prototype? L : nullptr; +#else cbl_label_t label = {}; label.type = LblProgram; label.parent = parent; @@ -644,6 +660,7 @@ symbol_program( size_t parent, const char name[], bool prototype ) &symbols.nelem, sizeof(key), symbol_elem_cmp ) ); return e? cbl_label_of(e) : NULL; +#endif } extern int yydebug; @@ -660,6 +677,16 @@ enum protoreq_t { static struct symbol_elem_t * symbol_function_impl( size_t parent, const char name[], protoreq_t protoreq ) { +#if 1 + auto key( elem_key_t(parent, name) ); + auto p = symbols.functions.find(key); + if( p == symbols.functions.end() ) return nullptr; + auto e = symbol_at(p->second); + const cbl_label_t *L = cbl_label_of(e); + if( protoreq == proto_required_e && !L->prototype ) return nullptr; + if( protoreq == proto_disallowed_e && L->prototype ) return nullptr; + return e; +#else auto p = std::find_if( symbols_begin(), symbols_end(), [parent, name, protoreq]( const auto& elem ) { if( elem.type == SymLabel ) { @@ -681,6 +708,7 @@ symbol_function_impl( size_t parent, const char name[], protoreq_t protoreq ) if( yydebug && p == symbols_end() ) symbols_dump( symbols.first_program, true); return p == symbols_end()? NULL : p; +#endif } struct symbol_elem_t * @@ -3646,46 +3674,69 @@ cbl_alphabet_t::also( const cbl_loc_t& loc, size_t ch ) { error_msg(loc, "ALSO value %zu is unknown", ch); } -static symbol_temporaries_t program_temporaries; +static class program_temporaries_t : private symbol_temporaries_t { + std::vector<symbol_temporaries_t::iterator> alphas; + static bool is_alpha_intermediate( const cbl_field_t *f ) { + switch(f->type) { + case FldAlphaEdited: + case FldAlphanumeric: + return f->has_attr(intermediate_e); + default: + break; + } + return false; + } + public: + program_temporaries_t() {} + symbol_temporaries_t::iterator dnubegin() { return symbol_temporaries_t::begin(); } + symbol_temporaries_t::iterator dnuend() { return symbol_temporaries_t::end(); } + + void push_back( cbl_field_t *field ) { + symbol_temporaries_t::push_back(field); + if( is_alpha_intermediate(field) ) { + auto p(end()); + p--; + alphas.push_back(p); + } + } + + // return the alphanumeric temporaries and remove them from the list + symbol_temporaries_t alphanumerics() { + symbol_temporaries_t output; + std::transform( alphas.begin(), alphas.end(), + std::back_inserter(output), + []( auto p ) { + return *p; + } ); + erase_alphas(); + return output; + } + symbol_temporaries_t& as_list() { + return *this; + } + + protected: + void erase_alphas() { + for( auto p : alphas ) { + this->erase(p); + } + alphas.clear(); + } +} program_temporaries; + /* * Supply a reference to the current list of temporaries for use by codegen to free * the memory if it decides to return to the caller. */ symbol_temporaries_t& symbol_temporaries() { - return program_temporaries; + return program_temporaries.as_list(); } symbol_temporaries_t symbol_temporary_alphanumerics() { - symbol_temporaries_t output; - std::copy_if( program_temporaries.begin(), - program_temporaries.end(), - std::back_inserter(output), - []( auto f ) { - switch(f->type) { - case FldAlphaEdited: - case FldAlphanumeric: - return f->has_attr(intermediate_e); - case FldFloat: - case FldNumericBin5: - case FldNumericBinary: - case FldNumericDisplay: - case FldNumericEdited: - case FldPacked: - default: - break; - } - return false; - } ); - for( cbl_field_t *f : output ) { - auto p = std::find( program_temporaries.begin(), - program_temporaries.end(), - f ); - program_temporaries.erase(p); - } - return output; + return program_temporaries.alphanumerics(); } /* @@ -4647,6 +4698,17 @@ symbol_program_add( size_t program, cbl_label_t *input ) e = symbol_append(elem); + switch( input->type ) { + case LblProgram: + symbols.program_add( symbol_index(e) ); + break; + case LblFunction: + symbols.function_add( symbol_index(e) ); + break; + default: + gcc_unreachable(); + } + common_callables_update( symbol_index(e) ); return cbl_label_of(e); diff --git a/gcc/cobol/token_names.h b/gcc/cobol/token_names.h index 4505d45609c..1431eee6e9b 100644 --- a/gcc/cobol/token_names.h +++ b/gcc/cobol/token_names.h @@ -1,14 +1,14 @@ // generated by token_names.h.gen tokens = { - { "identification", IDENTIFICATION_DIV }, // 258 - { "environment", ENVIRONMENT_DIV }, // 259 - { "procedure", PROCEDURE_DIV }, // 260 - { "data", DATA_DIV }, // 261 - { "file", FILE_SECT }, // 262 - { "input-output", INPUT_OUTPUT_SECT }, // 263 - { "linkage", LINKAGE_SECT }, // 264 - { "local-storage", LOCAL_STORAGE_SECT }, // 265 - { "working-storage", WORKING_STORAGE_SECT }, // 266 + { "identification division", IDENTIFICATION_DIV }, // 258 + { "environment division", ENVIRONMENT_DIV }, // 259 + { "procedure division", PROCEDURE_DIV }, // 260 + { "data division", DATA_DIV }, // 261 + { "file section", FILE_SECT }, // 262 + { "input_output section", INPUT_OUTPUT_SECT }, // 263 + { "linkage section", LINKAGE_SECT }, // 264 + { "local_storage section", LOCAL_STORAGE_SECT }, // 265 + { "working_storage section", WORKING_STORAGE_SECT }, // 266 { "object-computer", OBJECT_COMPUTER }, // 267 { "display-of", DISPLAY_OF }, // 268 { "end-function", END_FUNCTION }, // 269 @@ -258,7 +258,7 @@ tokens = { { "common", COMMON }, // 510 { "concat", CONCAT }, // 511 { "condition", CONDITION }, // 512 - { "configuration", CONFIGURATION_SECT }, // 513 + { "configuration section", CONFIGURATION_SECT }, // 513 { "contains", CONTAINS }, // 514 { "content", CONTENT }, // 515 { "control", CONTROL }, // 516 @@ -415,295 +415,295 @@ tokens = { { "mode", MODE }, // 667 { "module-name", MODULE_NAME }, // 668 { "named", NAMED }, // 669 - { "nat", NAT }, // 670 - { "national", NATIONAL }, // 671 - { "national-edited", NATIONAL_EDITED }, // 672 - { "national-of", NATIONAL_OF }, // 673 - { "native", NATIVE }, // 674 - { "nested", NESTED }, // 675 - { "next", NEXT }, // 676 - { "no", NO }, // 677 - { "note", NOTE }, // 678 - { "nulls", NULLS }, // 679 - { "null", NULLS }, // 679 - { "nullptr", NULLPTR }, // 680 - { "numeric", NUMERIC }, // 681 - { "numeric-edited", NUMERIC_EDITED }, // 682 - { "numval", NUMVAL }, // 683 - { "numval-c", NUMVAL_C }, // 684 - { "numval-f", NUMVAL_F }, // 685 - { "occurs", OCCURS }, // 686 - { "of", OF }, // 687 - { "off", OFF }, // 688 - { "omitted", OMITTED }, // 689 - { "on", ON }, // 690 - { "only", ONLY }, // 691 - { "optional", OPTIONAL }, // 692 - { "options", OPTIONS }, // 693 - { "ord", ORD }, // 694 - { "order", ORDER }, // 695 - { "ord-max", ORD_MAX }, // 696 - { "ord-min", ORD_MIN }, // 697 - { "organization", ORGANIZATION }, // 698 - { "other", OTHER }, // 699 - { "otherwise", OTHERWISE }, // 700 - { "output", OUTPUT }, // 701 - { "packed-decimal", PACKED_DECIMAL }, // 702 - { "padding", PADDING }, // 703 - { "page", PAGE }, // 704 - { "page-counter", PAGE_COUNTER }, // 705 - { "pf", PF }, // 706 - { "ph", PH }, // 707 - { "pi", PI }, // 708 - { "pic", PIC }, // 709 - { "picture", PICTURE }, // 710 - { "plus", PLUS }, // 711 - { "present-value", PRESENT_VALUE }, // 712 - { "print-switch", PRINT_SWITCH }, // 713 - { "procedure", PROCEDURE }, // 714 - { "procedures", PROCEDURES }, // 715 - { "proceed", PROCEED }, // 716 - { "processing", PROCESSING }, // 717 - { "program-id", PROGRAM_ID }, // 718 - { "program", PROGRAM_kw }, // 719 - { "property", PROPERTY }, // 720 - { "prototype", PROTOTYPE }, // 721 - { "pseudotext", PSEUDOTEXT }, // 722 - { "quotes", QUOTES }, // 723 - { "quote", QUOTES }, // 723 - { "random", RANDOM }, // 724 - { "random-seed", RANDOM_SEED }, // 725 - { "range", RANGE }, // 726 - { "raise", RAISE }, // 727 - { "raising", RAISING }, // 728 - { "rd", RD }, // 729 - { "record", RECORD }, // 730 - { "recording", RECORDING }, // 731 - { "records", RECORDS }, // 732 - { "recursive", RECURSIVE }, // 733 - { "redefines", REDEFINES }, // 734 - { "reel", REEL }, // 735 - { "reference", REFERENCE }, // 736 - { "relative", RELATIVE }, // 737 - { "rem", REM }, // 738 - { "remainder", REMAINDER }, // 739 - { "remarks", REMARKS }, // 740 - { "removal", REMOVAL }, // 741 - { "renames", RENAMES }, // 742 - { "replace", REPLACE }, // 743 - { "replacing", REPLACING }, // 744 - { "report", REPORT }, // 745 - { "reporting", REPORTING }, // 746 - { "reports", REPORTS }, // 747 - { "repository", REPOSITORY }, // 748 - { "rerun", RERUN }, // 749 - { "reserve", RESERVE }, // 750 - { "restricted", RESTRICTED }, // 751 - { "resume", RESUME }, // 752 - { "retry", RETRY }, // 753 - { "reverse", REVERSE }, // 754 - { "reversed", REVERSED }, // 755 - { "rewind", REWIND }, // 756 - { "rf", RF }, // 757 - { "rh", RH }, // 758 - { "right", RIGHT }, // 759 - { "rounded", ROUNDED }, // 760 - { "run", RUN }, // 761 - { "same", SAME }, // 762 - { "screen", SCREEN }, // 763 - { "sd", SD }, // 764 - { "seconds-from-formatted-time", SECONDS_FROM_FORMATTED_TIME }, // 765 - { "seconds-past-midnight", SECONDS_PAST_MIDNIGHT }, // 766 - { "security", SECURITY }, // 767 - { "separate", SEPARATE }, // 768 - { "sequence", SEQUENCE }, // 769 - { "sequential", SEQUENTIAL }, // 770 - { "sharing", SHARING }, // 771 - { "simple-exit", SIMPLE_EXIT }, // 772 - { "sign", SIGN }, // 773 - { "sin", SIN }, // 774 - { "size", SIZE }, // 775 - { "smallest-algebraic", SMALLEST_ALGEBRAIC }, // 776 - { "source", SOURCE }, // 777 - { "source-computer", SOURCE_COMPUTER }, // 778 - { "special-names", SPECIAL_NAMES }, // 779 - { "sqrt", SQRT }, // 780 - { "stack", STACK }, // 781 - { "standard", STANDARD }, // 782 - { "standard-1", STANDARD_1 }, // 783 - { "standard-deviation", STANDARD_DEVIATION }, // 784 - { "standard-compare", STANDARD_COMPARE }, // 785 - { "status", STATUS }, // 786 - { "strong", STRONG }, // 787 - { "substitute", SUBSTITUTE }, // 788 - { "sum", SUM }, // 789 - { "symbol", SYMBOL }, // 790 - { "symbolic", SYMBOLIC }, // 791 - { "synchronized", SYNCHRONIZED }, // 792 - { "tallying", TALLYING }, // 793 - { "tan", TAN }, // 794 - { "terminate", TERMINATE }, // 795 - { "test", TEST }, // 796 - { "test-date-yyyymmdd", TEST_DATE_YYYYMMDD }, // 797 - { "test-day-yyyyddd", TEST_DAY_YYYYDDD }, // 798 - { "test-formatted-datetime", TEST_FORMATTED_DATETIME }, // 799 - { "test-numval", TEST_NUMVAL }, // 800 - { "test-numval-c", TEST_NUMVAL_C }, // 801 - { "test-numval-f", TEST_NUMVAL_F }, // 802 - { "than", THAN }, // 803 - { "time", TIME }, // 804 - { "times", TIMES }, // 805 - { "to", TO }, // 806 - { "top", TOP }, // 807 - { "top-level", TOP_LEVEL }, // 808 - { "tracks", TRACKS }, // 809 - { "track-area", TRACK_AREA }, // 810 - { "trailing", TRAILING }, // 811 - { "transform", TRANSFORM }, // 812 - { "trim", TRIM }, // 813 - { "true", TRUE_kw }, // 814 - { "try", TRY }, // 815 - { "turn", TURN }, // 816 - { "type", TYPE }, // 817 - { "typedef", TYPEDEF }, // 818 - { "ulength", ULENGTH }, // 819 - { "unbounded", UNBOUNDED }, // 820 - { "unit", UNIT }, // 821 - { "units", UNITS }, // 822 - { "unit-record", UNIT_RECORD }, // 823 - { "until", UNTIL }, // 824 - { "up", UP }, // 825 - { "upon", UPON }, // 826 - { "upos", UPOS }, // 827 - { "upper-case", UPPER_CASE }, // 828 - { "usage", USAGE }, // 829 - { "using", USING }, // 830 - { "usubstr", USUBSTR }, // 831 - { "usupplementary", USUPPLEMENTARY }, // 832 - { "utility", UTILITY }, // 833 - { "uuid4", UUID4 }, // 834 - { "uvalid", UVALID }, // 835 - { "uwidth", UWIDTH }, // 836 - { "validating", VALIDATING }, // 837 - { "value", VALUE }, // 838 - { "variance", VARIANCE }, // 839 - { "varying", VARYING }, // 840 - { "volatile", VOLATILE }, // 841 - { "when-compiled", WHEN_COMPILED }, // 842 - { "with", WITH }, // 843 - { "working-storage", WORKING_STORAGE }, // 844 - { "year-to-yyyy", YEAR_TO_YYYY }, // 845 - { "yyyyddd", YYYYDDD }, // 846 - { "yyyymmdd", YYYYMMDD }, // 847 - { "arithmetic", ARITHMETIC }, // 848 - { "attribute", ATTRIBUTE }, // 849 - { "auto", AUTO }, // 850 - { "automatic", AUTOMATIC }, // 851 - { "away-from-zero", AWAY_FROM_ZERO }, // 852 - { "background-color", BACKGROUND_COLOR }, // 853 - { "bell", BELL }, // 854 - { "binary-encoding", BINARY_ENCODING }, // 855 - { "blink", BLINK }, // 856 - { "capacity", CAPACITY }, // 857 - { "center", CENTER }, // 858 - { "classification", CLASSIFICATION }, // 859 - { "cycle", CYCLE }, // 860 - { "decimal-encoding", DECIMAL_ENCODING }, // 861 - { "entry-convention", ENTRY_CONVENTION }, // 862 - { "eol", EOL }, // 863 - { "eos", EOS }, // 864 - { "erase", ERASE }, // 865 - { "expands", EXPANDS }, // 866 - { "float-binary", FLOAT_BINARY }, // 867 - { "float-decimal", FLOAT_DECIMAL }, // 868 - { "foreground-color", FOREGROUND_COLOR }, // 869 - { "forever", FOREVER }, // 870 - { "full", FULL }, // 871 - { "highlight", HIGHLIGHT }, // 872 - { "high-order-left", HIGH_ORDER_LEFT }, // 873 - { "high-order-right", HIGH_ORDER_RIGHT }, // 874 - { "ignoring", IGNORING }, // 875 - { "implements", IMPLEMENTS }, // 876 - { "initialized", INITIALIZED }, // 877 - { "intermediate", INTERMEDIATE }, // 878 - { "lc-all", LC_ALL_kw }, // 879 - { "lc-collate", LC_COLLATE_kw }, // 880 - { "lc-ctype", LC_CTYPE_kw }, // 881 - { "lc-messages", LC_MESSAGES_kw }, // 882 - { "lc-monetary", LC_MONETARY_kw }, // 883 - { "lc-numeric", LC_NUMERIC_kw }, // 884 - { "lc-time", LC_TIME_kw }, // 885 - { "lowlight", LOWLIGHT }, // 886 - { "nearest-away-from-zero", NEAREST_AWAY_FROM_ZERO }, // 887 - { "nearest-even", NEAREST_EVEN }, // 888 - { "nearest-toward-zero", NEAREST_TOWARD_ZERO }, // 889 - { "none", NONE }, // 890 - { "normal", NORMAL }, // 891 - { "numbers", NUMBERS }, // 892 - { "prefixed", PREFIXED }, // 893 - { "previous", PREVIOUS }, // 894 - { "prohibited", PROHIBITED }, // 895 - { "relation", RELATION }, // 896 - { "required", REQUIRED }, // 897 - { "reverse-video", REVERSE_VIDEO }, // 898 - { "rounding", ROUNDING }, // 899 - { "seconds", SECONDS }, // 900 - { "secure", SECURE }, // 901 - { "short", SHORT }, // 902 - { "signed", SIGNED_kw }, // 903 - { "standard-binary", STANDARD_BINARY }, // 904 - { "standard-decimal", STANDARD_DECIMAL }, // 905 - { "statement", STATEMENT }, // 906 - { "step", STEP }, // 907 - { "structure", STRUCTURE }, // 908 - { "toward-greater", TOWARD_GREATER }, // 909 - { "toward-lesser", TOWARD_LESSER }, // 910 - { "truncation", TRUNCATION }, // 911 - { "ucs-4", UCS_4 }, // 912 - { "underline", UNDERLINE }, // 913 - { "unsigned", UNSIGNED_kw }, // 914 - { "utf-16", UTF_16 }, // 915 - { "utf-8", UTF_8 }, // 916 - { "xmlgenerate", XMLGENERATE }, // 917 - { "xmlparse", XMLPARSE }, // 918 - { "address", ADDRESS }, // 919 - { "end-accept", END_ACCEPT }, // 920 - { "end-add", END_ADD }, // 921 - { "end-call", END_CALL }, // 922 - { "end-compute", END_COMPUTE }, // 923 - { "end-delete", END_DELETE }, // 924 - { "end-display", END_DISPLAY }, // 925 - { "end-divide", END_DIVIDE }, // 926 - { "end-evaluate", END_EVALUATE }, // 927 - { "end-multiply", END_MULTIPLY }, // 928 - { "end-perform", END_PERFORM }, // 929 - { "end-read", END_READ }, // 930 - { "end-return", END_RETURN }, // 931 - { "end-rewrite", END_REWRITE }, // 932 - { "end-search", END_SEARCH }, // 933 - { "end-start", END_START }, // 934 - { "end-string", END_STRING }, // 935 - { "end-subtract", END_SUBTRACT }, // 936 - { "end-unstring", END_UNSTRING }, // 937 - { "end-write", END_WRITE }, // 938 - { "end-xml", END_XML }, // 939 - { "end-if", END_IF }, // 940 - { "attributes", ATTRIBUTES }, // 941 - { "element", ELEMENT }, // 942 - { "namespace", NAMESPACE }, // 943 - { "namespace-prefix", NAMESPACE_PREFIX }, // 944 + { "namespace", NAMESPACE }, // 670 + { "namespace-prefix", NAMESPACE_PREFIX }, // 671 + { "nat", NAT }, // 672 + { "national", NATIONAL }, // 673 + { "national-edited", NATIONAL_EDITED }, // 674 + { "national-of", NATIONAL_OF }, // 675 + { "native", NATIVE }, // 676 + { "nested", NESTED }, // 677 + { "next", NEXT }, // 678 + { "no", NO }, // 679 + { "note", NOTE }, // 680 + { "nulls", NULLS }, // 681 + { "null", NULLS }, // 681 + { "nullptr", NULLPTR }, // 682 + { "numeric", NUMERIC }, // 683 + { "numeric-edited", NUMERIC_EDITED }, // 684 + { "numval", NUMVAL }, // 685 + { "numval-c", NUMVAL_C }, // 686 + { "numval-f", NUMVAL_F }, // 687 + { "occurs", OCCURS }, // 688 + { "of", OF }, // 689 + { "off", OFF }, // 690 + { "omitted", OMITTED }, // 691 + { "on", ON }, // 692 + { "only", ONLY }, // 693 + { "optional", OPTIONAL }, // 694 + { "options", OPTIONS }, // 695 + { "ord", ORD }, // 696 + { "order", ORDER }, // 697 + { "ord-max", ORD_MAX }, // 698 + { "ord-min", ORD_MIN }, // 699 + { "organization", ORGANIZATION }, // 700 + { "other", OTHER }, // 701 + { "otherwise", OTHERWISE }, // 702 + { "output", OUTPUT }, // 703 + { "packed-decimal", PACKED_DECIMAL }, // 704 + { "padding", PADDING }, // 705 + { "page", PAGE }, // 706 + { "page-counter", PAGE_COUNTER }, // 707 + { "pf", PF }, // 708 + { "ph", PH }, // 709 + { "pi", PI }, // 710 + { "pic", PIC }, // 711 + { "picture", PICTURE }, // 712 + { "plus", PLUS }, // 713 + { "present-value", PRESENT_VALUE }, // 714 + { "print-switch", PRINT_SWITCH }, // 715 + { "procedure", PROCEDURE }, // 716 + { "procedures", PROCEDURES }, // 717 + { "proceed", PROCEED }, // 718 + { "processing", PROCESSING }, // 719 + { "program-id", PROGRAM_ID }, // 720 + { "program", PROGRAM_kw }, // 721 + { "property", PROPERTY }, // 722 + { "prototype", PROTOTYPE }, // 723 + { "pseudotext", PSEUDOTEXT }, // 724 + { "quotes", QUOTES }, // 725 + { "quote", QUOTES }, // 725 + { "random", RANDOM }, // 726 + { "random-seed", RANDOM_SEED }, // 727 + { "range", RANGE }, // 728 + { "raise", RAISE }, // 729 + { "raising", RAISING }, // 730 + { "rd", RD }, // 731 + { "record", RECORD }, // 732 + { "recording", RECORDING }, // 733 + { "records", RECORDS }, // 734 + { "recursive", RECURSIVE }, // 735 + { "redefines", REDEFINES }, // 736 + { "reel", REEL }, // 737 + { "reference", REFERENCE }, // 738 + { "relative", RELATIVE }, // 739 + { "rem", REM }, // 740 + { "remainder", REMAINDER }, // 741 + { "remarks", REMARKS }, // 742 + { "removal", REMOVAL }, // 743 + { "renames", RENAMES }, // 744 + { "replace", REPLACE }, // 745 + { "replacing", REPLACING }, // 746 + { "report", REPORT }, // 747 + { "reporting", REPORTING }, // 748 + { "reports", REPORTS }, // 749 + { "repository", REPOSITORY }, // 750 + { "rerun", RERUN }, // 751 + { "reserve", RESERVE }, // 752 + { "restricted", RESTRICTED }, // 753 + { "resume", RESUME }, // 754 + { "retry", RETRY }, // 755 + { "reverse", REVERSE }, // 756 + { "reversed", REVERSED }, // 757 + { "rewind", REWIND }, // 758 + { "rf", RF }, // 759 + { "rh", RH }, // 760 + { "right", RIGHT }, // 761 + { "rounded", ROUNDED }, // 762 + { "run", RUN }, // 763 + { "same", SAME }, // 764 + { "screen", SCREEN }, // 765 + { "sd", SD }, // 766 + { "seconds-from-formatted-time", SECONDS_FROM_FORMATTED_TIME }, // 767 + { "seconds-past-midnight", SECONDS_PAST_MIDNIGHT }, // 768 + { "security", SECURITY }, // 769 + { "separate", SEPARATE }, // 770 + { "sequence", SEQUENCE }, // 771 + { "sequential", SEQUENTIAL }, // 772 + { "sharing", SHARING }, // 773 + { "simple-exit", SIMPLE_EXIT }, // 774 + { "sign", SIGN }, // 775 + { "sin", SIN }, // 776 + { "size", SIZE }, // 777 + { "smallest-algebraic", SMALLEST_ALGEBRAIC }, // 778 + { "source", SOURCE }, // 779 + { "source-computer", SOURCE_COMPUTER }, // 780 + { "special-names", SPECIAL_NAMES }, // 781 + { "sqrt", SQRT }, // 782 + { "stack", STACK }, // 783 + { "standard", STANDARD }, // 784 + { "standard-1", STANDARD_1 }, // 785 + { "standard-deviation", STANDARD_DEVIATION }, // 786 + { "standard-compare", STANDARD_COMPARE }, // 787 + { "status", STATUS }, // 788 + { "strong", STRONG }, // 789 + { "substitute", SUBSTITUTE }, // 790 + { "sum", SUM }, // 791 + { "symbol", SYMBOL }, // 792 + { "symbolic", SYMBOLIC }, // 793 + { "synchronized", SYNCHRONIZED }, // 794 + { "tallying", TALLYING }, // 795 + { "tan", TAN }, // 796 + { "terminate", TERMINATE }, // 797 + { "test", TEST }, // 798 + { "test-date-yyyymmdd", TEST_DATE_YYYYMMDD }, // 799 + { "test-day-yyyyddd", TEST_DAY_YYYYDDD }, // 800 + { "test-formatted-datetime", TEST_FORMATTED_DATETIME }, // 801 + { "test-numval", TEST_NUMVAL }, // 802 + { "test-numval-c", TEST_NUMVAL_C }, // 803 + { "test-numval-f", TEST_NUMVAL_F }, // 804 + { "than", THAN }, // 805 + { "time", TIME }, // 806 + { "times", TIMES }, // 807 + { "to", TO }, // 808 + { "top", TOP }, // 809 + { "top-level", TOP_LEVEL }, // 810 + { "tracks", TRACKS }, // 811 + { "track-area", TRACK_AREA }, // 812 + { "trailing", TRAILING }, // 813 + { "transform", TRANSFORM }, // 814 + { "trim", TRIM }, // 815 + { "true", TRUE_kw }, // 816 + { "try", TRY }, // 817 + { "turn", TURN }, // 818 + { "type", TYPE }, // 819 + { "typedef", TYPEDEF }, // 820 + { "ulength", ULENGTH }, // 821 + { "unbounded", UNBOUNDED }, // 822 + { "unit", UNIT }, // 823 + { "units", UNITS }, // 824 + { "unit-record", UNIT_RECORD }, // 825 + { "until", UNTIL }, // 826 + { "up", UP }, // 827 + { "upon", UPON }, // 828 + { "upos", UPOS }, // 829 + { "upper-case", UPPER_CASE }, // 830 + { "usage", USAGE }, // 831 + { "using", USING }, // 832 + { "usubstr", USUBSTR }, // 833 + { "usupplementary", USUPPLEMENTARY }, // 834 + { "utility", UTILITY }, // 835 + { "uuid4", UUID4 }, // 836 + { "uvalid", UVALID }, // 837 + { "uwidth", UWIDTH }, // 838 + { "validating", VALIDATING }, // 839 + { "value", VALUE }, // 840 + { "variance", VARIANCE }, // 841 + { "varying", VARYING }, // 842 + { "volatile", VOLATILE }, // 843 + { "when-compiled", WHEN_COMPILED }, // 844 + { "with", WITH }, // 845 + { "working-storage", WORKING_STORAGE }, // 846 + { "year-to-yyyy", YEAR_TO_YYYY }, // 847 + { "yyyyddd", YYYYDDD }, // 848 + { "yyyymmdd", YYYYMMDD }, // 849 + { "arithmetic", ARITHMETIC }, // 850 + { "attribute", ATTRIBUTE }, // 851 + { "auto", AUTO }, // 852 + { "automatic", AUTOMATIC }, // 853 + { "away-from-zero", AWAY_FROM_ZERO }, // 854 + { "background-color", BACKGROUND_COLOR }, // 855 + { "bell", BELL }, // 856 + { "binary-encoding", BINARY_ENCODING }, // 857 + { "blink", BLINK }, // 858 + { "capacity", CAPACITY }, // 859 + { "center", CENTER }, // 860 + { "classification", CLASSIFICATION }, // 861 + { "cycle", CYCLE }, // 862 + { "decimal-encoding", DECIMAL_ENCODING }, // 863 + { "entry-convention", ENTRY_CONVENTION }, // 864 + { "eol", EOL }, // 865 + { "eos", EOS }, // 866 + { "erase", ERASE }, // 867 + { "expands", EXPANDS }, // 868 + { "float-binary", FLOAT_BINARY }, // 869 + { "float-decimal", FLOAT_DECIMAL }, // 870 + { "foreground-color", FOREGROUND_COLOR }, // 871 + { "forever", FOREVER }, // 872 + { "full", FULL }, // 873 + { "highlight", HIGHLIGHT }, // 874 + { "high-order-left", HIGH_ORDER_LEFT }, // 875 + { "high-order-right", HIGH_ORDER_RIGHT }, // 876 + { "ignoring", IGNORING }, // 877 + { "implements", IMPLEMENTS }, // 878 + { "initialized", INITIALIZED }, // 879 + { "intermediate", INTERMEDIATE }, // 880 + { "lc-all", LC_ALL_kw }, // 881 + { "lc-collate", LC_COLLATE_kw }, // 882 + { "lc-ctype", LC_CTYPE_kw }, // 883 + { "lc-messages", LC_MESSAGES_kw }, // 884 + { "lc-monetary", LC_MONETARY_kw }, // 885 + { "lc-numeric", LC_NUMERIC_kw }, // 886 + { "lc-time", LC_TIME_kw }, // 887 + { "lowlight", LOWLIGHT }, // 888 + { "nearest-away-from-zero", NEAREST_AWAY_FROM_ZERO }, // 889 + { "nearest-even", NEAREST_EVEN }, // 890 + { "nearest-toward-zero", NEAREST_TOWARD_ZERO }, // 891 + { "none", NONE }, // 892 + { "normal", NORMAL }, // 893 + { "numbers", NUMBERS }, // 894 + { "prefixed", PREFIXED }, // 895 + { "previous", PREVIOUS }, // 896 + { "prohibited", PROHIBITED }, // 897 + { "relation", RELATION }, // 898 + { "required", REQUIRED }, // 899 + { "reverse-video", REVERSE_VIDEO }, // 900 + { "rounding", ROUNDING }, // 901 + { "seconds", SECONDS }, // 902 + { "secure", SECURE }, // 903 + { "short", SHORT }, // 904 + { "signed", SIGNED_kw }, // 905 + { "standard-binary", STANDARD_BINARY }, // 906 + { "standard-decimal", STANDARD_DECIMAL }, // 907 + { "statement", STATEMENT }, // 908 + { "step", STEP }, // 909 + { "structure", STRUCTURE }, // 910 + { "toward-greater", TOWARD_GREATER }, // 911 + { "toward-lesser", TOWARD_LESSER }, // 912 + { "truncation", TRUNCATION }, // 913 + { "ucs-4", UCS_4 }, // 914 + { "underline", UNDERLINE }, // 915 + { "unsigned", UNSIGNED_kw }, // 916 + { "utf-16", UTF_16 }, // 917 + { "utf-8", UTF_8 }, // 918 + { "xml-declaration", XML_DECLARATION }, // 919 + { "xmlgenerate", XMLGENERATE }, // 920 + { "xmlparse", XMLPARSE }, // 921 + { "address", ADDRESS }, // 922 + { "end-accept", END_ACCEPT }, // 923 + { "end-add", END_ADD }, // 924 + { "end-call", END_CALL }, // 925 + { "end-compute", END_COMPUTE }, // 926 + { "end-delete", END_DELETE }, // 927 + { "end-display", END_DISPLAY }, // 928 + { "end-divide", END_DIVIDE }, // 929 + { "end-evaluate", END_EVALUATE }, // 930 + { "end-multiply", END_MULTIPLY }, // 931 + { "end-perform", END_PERFORM }, // 932 + { "end-read", END_READ }, // 933 + { "end-return", END_RETURN }, // 934 + { "end-rewrite", END_REWRITE }, // 935 + { "end-search", END_SEARCH }, // 936 + { "end-start", END_START }, // 937 + { "end-string", END_STRING }, // 938 + { "end-subtract", END_SUBTRACT }, // 939 + { "end-unstring", END_UNSTRING }, // 940 + { "end-write", END_WRITE }, // 941 + { "end-xml", END_XML }, // 942 + { "end-if", END_IF }, // 943 + { "attributes", ATTRIBUTES }, // 944 + { "element", ELEMENT }, // 945 { "nonnumeric", NONNUMERIC }, // 946 - { "xml-declaration", XML_DECLARATION }, // 947 - { "thru", THRU }, // 949 - { "through", THRU }, // 949 - { "or", OR }, // 950 - { "and", AND }, // 951 - { "not", NOT }, // 952 - { "ne", NE }, // 953 - { "le", LE }, // 954 - { "ge", GE }, // 955 - { "pow", POW }, // 956 - { "neg", NEG }, // 957 + { "thru", THRU }, // 947 + { "through", THRU }, // 947 + { "or", OR }, // 948 + { "and", AND }, // 949 + { "not", NOT }, // 950 + { "ne", NE }, // 951 + { "le", LE }, // 952 + { "ge", GE }, // 953 + { "pow", POW }, // 954 + { "neg", NEG }, // 955 }; // cppcheck-suppress useInitializationList @@ -1120,290 +1120,290 @@ token_names = { "MODE", // 409 (667) "MODULE-NAME", // 410 (668) "NAMED", // 411 (669) - "NAT", // 412 (670) - "NATIONAL", // 413 (671) - "NATIONAL-EDITED", // 414 (672) - "NATIONAL-OF", // 415 (673) - "NATIVE", // 416 (674) - "NESTED", // 417 (675) - "NEXT", // 418 (676) - "NO", // 419 (677) - "NOTE", // 420 (678) - "NULLS", // 421 (679) - "NULLPTR", // 422 (680) - "NUMERIC", // 423 (681) - "NUMERIC-EDITED", // 424 (682) - "NUMVAL", // 425 (683) - "NUMVAL-C", // 426 (684) - "NUMVAL-F", // 427 (685) - "OCCURS", // 428 (686) - "OF", // 429 (687) - "OFF", // 430 (688) - "OMITTED", // 431 (689) - "ON", // 432 (690) - "ONLY", // 433 (691) - "OPTIONAL", // 434 (692) - "OPTIONS", // 435 (693) - "ORD", // 436 (694) - "ORDER", // 437 (695) - "ORD-MAX", // 438 (696) - "ORD-MIN", // 439 (697) - "ORGANIZATION", // 440 (698) - "OTHER", // 441 (699) - "OTHERWISE", // 442 (700) - "OUTPUT", // 443 (701) - "PACKED-DECIMAL", // 444 (702) - "PADDING", // 445 (703) - "PAGE", // 446 (704) - "PAGE-COUNTER", // 447 (705) - "PF", // 448 (706) - "PH", // 449 (707) - "PI", // 450 (708) - "PIC", // 451 (709) - "PICTURE", // 452 (710) - "PLUS", // 453 (711) - "PRESENT-VALUE", // 454 (712) - "PRINT-SWITCH", // 455 (713) - "PROCEDURE", // 456 (714) - "PROCEDURES", // 457 (715) - "PROCEED", // 458 (716) - "PROCESSING", // 459 (717) - "PROGRAM-ID", // 460 (718) - "PROGRAM", // 461 (719) - "PROPERTY", // 462 (720) - "PROTOTYPE", // 463 (721) - "PSEUDOTEXT", // 464 (722) - "QUOTES", // 465 (723) - "RANDOM", // 466 (724) - "RANDOM-SEED", // 467 (725) - "RANGE", // 468 (726) - "RAISE", // 469 (727) - "RAISING", // 470 (728) - "RD", // 471 (729) - "RECORD", // 472 (730) - "RECORDING", // 473 (731) - "RECORDS", // 474 (732) - "RECURSIVE", // 475 (733) - "REDEFINES", // 476 (734) - "REEL", // 477 (735) - "REFERENCE", // 478 (736) - "RELATIVE", // 479 (737) - "REM", // 480 (738) - "REMAINDER", // 481 (739) - "REMARKS", // 482 (740) - "REMOVAL", // 483 (741) - "RENAMES", // 484 (742) - "REPLACE", // 485 (743) - "REPLACING", // 486 (744) - "REPORT", // 487 (745) - "REPORTING", // 488 (746) - "REPORTS", // 489 (747) - "REPOSITORY", // 490 (748) - "RERUN", // 491 (749) - "RESERVE", // 492 (750) - "RESTRICTED", // 493 (751) - "RESUME", // 494 (752) - "RETRY", // 495 (753) - "REVERSE", // 496 (754) - "REVERSED", // 497 (755) - "REWIND", // 498 (756) - "RF", // 499 (757) - "RH", // 500 (758) - "RIGHT", // 501 (759) - "ROUNDED", // 502 (760) - "RUN", // 503 (761) - "SAME", // 504 (762) - "SCREEN", // 505 (763) - "SD", // 506 (764) - "SECONDS-FROM-FORMATTED-TIME", // 507 (765) - "SECONDS-PAST-MIDNIGHT", // 508 (766) - "SECURITY", // 509 (767) - "SEPARATE", // 510 (768) - "SEQUENCE", // 511 (769) - "SEQUENTIAL", // 512 (770) - "SHARING", // 513 (771) - "SIMPLE-EXIT", // 514 (772) - "SIGN", // 515 (773) - "SIN", // 516 (774) - "SIZE", // 517 (775) - "SMALLEST-ALGEBRAIC", // 518 (776) - "SOURCE", // 519 (777) - "SOURCE-COMPUTER", // 520 (778) - "SPECIAL-NAMES", // 521 (779) - "SQRT", // 522 (780) - "STACK", // 523 (781) - "STANDARD", // 524 (782) - "STANDARD-1", // 525 (783) - "STANDARD-DEVIATION", // 526 (784) - "STANDARD-COMPARE", // 527 (785) - "STATUS", // 528 (786) - "STRONG", // 529 (787) - "SUBSTITUTE", // 530 (788) - "SUM", // 531 (789) - "SYMBOL", // 532 (790) - "SYMBOLIC", // 533 (791) - "SYNCHRONIZED", // 534 (792) - "TALLYING", // 535 (793) - "TAN", // 536 (794) - "TERMINATE", // 537 (795) - "TEST", // 538 (796) - "TEST-DATE-YYYYMMDD", // 539 (797) - "TEST-DAY-YYYYDDD", // 540 (798) - "TEST-FORMATTED-DATETIME", // 541 (799) - "TEST-NUMVAL", // 542 (800) - "TEST-NUMVAL-C", // 543 (801) - "TEST-NUMVAL-F", // 544 (802) - "THAN", // 545 (803) - "TIME", // 546 (804) - "TIMES", // 547 (805) - "TO", // 548 (806) - "TOP", // 549 (807) - "TOP-LEVEL", // 550 (808) - "TRACKS", // 551 (809) - "TRACK-AREA", // 552 (810) - "TRAILING", // 553 (811) - "TRANSFORM", // 554 (812) - "TRIM", // 555 (813) - "TRUE", // 556 (814) - "TRY", // 557 (815) - "TURN", // 558 (816) - "TYPE", // 559 (817) - "TYPEDEF", // 560 (818) - "ULENGTH", // 561 (819) - "UNBOUNDED", // 562 (820) - "UNIT", // 563 (821) - "UNITS", // 564 (822) - "UNIT-RECORD", // 565 (823) - "UNTIL", // 566 (824) - "UP", // 567 (825) - "UPON", // 568 (826) - "UPOS", // 569 (827) - "UPPER-CASE", // 570 (828) - "USAGE", // 571 (829) - "USING", // 572 (830) - "USUBSTR", // 573 (831) - "USUPPLEMENTARY", // 574 (832) - "UTILITY", // 575 (833) - "UUID4", // 576 (834) - "UVALID", // 577 (835) - "UWIDTH", // 578 (836) - "VALIDATING", // 579 (837) - "VALUE", // 580 (838) - "VARIANCE", // 581 (839) - "VARYING", // 582 (840) - "VOLATILE", // 583 (841) - "WHEN-COMPILED", // 584 (842) - "WITH", // 585 (843) - "WORKING-STORAGE", // 586 (844) - "YEAR-TO-YYYY", // 587 (845) - "YYYYDDD", // 588 (846) - "YYYYMMDD", // 589 (847) - "ARITHMETIC", // 590 (848) - "ATTRIBUTE", // 591 (849) - "AUTO", // 592 (850) - "AUTOMATIC", // 593 (851) - "AWAY-FROM-ZERO", // 594 (852) - "BACKGROUND-COLOR", // 595 (853) - "BELL", // 596 (854) - "BINARY-ENCODING", // 597 (855) - "BLINK", // 598 (856) - "CAPACITY", // 599 (857) - "CENTER", // 600 (858) - "CLASSIFICATION", // 601 (859) - "CYCLE", // 602 (860) - "DECIMAL-ENCODING", // 603 (861) - "ENTRY-CONVENTION", // 604 (862) - "EOL", // 605 (863) - "EOS", // 606 (864) - "ERASE", // 607 (865) - "EXPANDS", // 608 (866) - "FLOAT-BINARY", // 609 (867) - "FLOAT-DECIMAL", // 610 (868) - "FOREGROUND-COLOR", // 611 (869) - "FOREVER", // 612 (870) - "FULL", // 613 (871) - "HIGHLIGHT", // 614 (872) - "HIGH-ORDER-LEFT", // 615 (873) - "HIGH-ORDER-RIGHT", // 616 (874) - "IGNORING", // 617 (875) - "IMPLEMENTS", // 618 (876) - "INITIALIZED", // 619 (877) - "INTERMEDIATE", // 620 (878) - "LC-ALL", // 621 (879) - "LC-COLLATE", // 622 (880) - "LC-CTYPE", // 623 (881) - "LC-MESSAGES", // 624 (882) - "LC-MONETARY", // 625 (883) - "LC-NUMERIC", // 626 (884) - "LC-TIME", // 627 (885) - "LOWLIGHT", // 628 (886) - "NEAREST-AWAY-FROM-ZERO", // 629 (887) - "NEAREST-EVEN", // 630 (888) - "NEAREST-TOWARD-ZERO", // 631 (889) - "NONE", // 632 (890) - "NORMAL", // 633 (891) - "NUMBERS", // 634 (892) - "PREFIXED", // 635 (893) - "PREVIOUS", // 636 (894) - "PROHIBITED", // 637 (895) - "RELATION", // 638 (896) - "REQUIRED", // 639 (897) - "REVERSE-VIDEO", // 640 (898) - "ROUNDING", // 641 (899) - "SECONDS", // 642 (900) - "SECURE", // 643 (901) - "SHORT", // 644 (902) - "SIGNED", // 645 (903) - "STANDARD-BINARY", // 646 (904) - "STANDARD-DECIMAL", // 647 (905) - "STATEMENT", // 648 (906) - "STEP", // 649 (907) - "STRUCTURE", // 650 (908) - "TOWARD-GREATER", // 651 (909) - "TOWARD-LESSER", // 652 (910) - "TRUNCATION", // 653 (911) - "UCS-4", // 654 (912) - "UNDERLINE", // 655 (913) - "UNSIGNED", // 656 (914) - "UTF-16", // 657 (915) - "UTF-8", // 658 (916) - "XMLGENERATE", // 659 (917) - "XMLPARSE", // 660 (918) - "ADDRESS", // 661 (919) - "END-ACCEPT", // 662 (920) - "END-ADD", // 663 (921) - "END-CALL", // 664 (922) - "END-COMPUTE", // 665 (923) - "END-DELETE", // 666 (924) - "END-DISPLAY", // 667 (925) - "END-DIVIDE", // 668 (926) - "END-EVALUATE", // 669 (927) - "END-MULTIPLY", // 670 (928) - "END-PERFORM", // 671 (929) - "END-READ", // 672 (930) - "END-RETURN", // 673 (931) - "END-REWRITE", // 674 (932) - "END-SEARCH", // 675 (933) - "END-START", // 676 (934) - "END-STRING", // 677 (935) - "END-SUBTRACT", // 678 (936) - "END-UNSTRING", // 679 (937) - "END-WRITE", // 680 (938) - "END-XML", // 681 (939) - "END-IF", // 682 (940) - "ATTRIBUTES", // 683 (941) - "ELEMENT", // 684 (942) - "NAMESPACE", // 685 (943) - "NAMESPACE-PREFIX", // 686 (944) + "NAMESPACE", // 412 (670) + "NAMESPACE-PREFIX", // 413 (671) + "NAT", // 414 (672) + "NATIONAL", // 415 (673) + "NATIONAL-EDITED", // 416 (674) + "NATIONAL-OF", // 417 (675) + "NATIVE", // 418 (676) + "NESTED", // 419 (677) + "NEXT", // 420 (678) + "NO", // 421 (679) + "NOTE", // 422 (680) + "NULLS", // 423 (681) + "NULLPTR", // 424 (682) + "NUMERIC", // 425 (683) + "NUMERIC-EDITED", // 426 (684) + "NUMVAL", // 427 (685) + "NUMVAL-C", // 428 (686) + "NUMVAL-F", // 429 (687) + "OCCURS", // 430 (688) + "OF", // 431 (689) + "OFF", // 432 (690) + "OMITTED", // 433 (691) + "ON", // 434 (692) + "ONLY", // 435 (693) + "OPTIONAL", // 436 (694) + "OPTIONS", // 437 (695) + "ORD", // 438 (696) + "ORDER", // 439 (697) + "ORD-MAX", // 440 (698) + "ORD-MIN", // 441 (699) + "ORGANIZATION", // 442 (700) + "OTHER", // 443 (701) + "OTHERWISE", // 444 (702) + "OUTPUT", // 445 (703) + "PACKED-DECIMAL", // 446 (704) + "PADDING", // 447 (705) + "PAGE", // 448 (706) + "PAGE-COUNTER", // 449 (707) + "PF", // 450 (708) + "PH", // 451 (709) + "PI", // 452 (710) + "PIC", // 453 (711) + "PICTURE", // 454 (712) + "PLUS", // 455 (713) + "PRESENT-VALUE", // 456 (714) + "PRINT-SWITCH", // 457 (715) + "PROCEDURE", // 458 (716) + "PROCEDURES", // 459 (717) + "PROCEED", // 460 (718) + "PROCESSING", // 461 (719) + "PROGRAM-ID", // 462 (720) + "PROGRAM", // 463 (721) + "PROPERTY", // 464 (722) + "PROTOTYPE", // 465 (723) + "PSEUDOTEXT", // 466 (724) + "QUOTES", // 467 (725) + "RANDOM", // 468 (726) + "RANDOM-SEED", // 469 (727) + "RANGE", // 470 (728) + "RAISE", // 471 (729) + "RAISING", // 472 (730) + "RD", // 473 (731) + "RECORD", // 474 (732) + "RECORDING", // 475 (733) + "RECORDS", // 476 (734) + "RECURSIVE", // 477 (735) + "REDEFINES", // 478 (736) + "REEL", // 479 (737) + "REFERENCE", // 480 (738) + "RELATIVE", // 481 (739) + "REM", // 482 (740) + "REMAINDER", // 483 (741) + "REMARKS", // 484 (742) + "REMOVAL", // 485 (743) + "RENAMES", // 486 (744) + "REPLACE", // 487 (745) + "REPLACING", // 488 (746) + "REPORT", // 489 (747) + "REPORTING", // 490 (748) + "REPORTS", // 491 (749) + "REPOSITORY", // 492 (750) + "RERUN", // 493 (751) + "RESERVE", // 494 (752) + "RESTRICTED", // 495 (753) + "RESUME", // 496 (754) + "RETRY", // 497 (755) + "REVERSE", // 498 (756) + "REVERSED", // 499 (757) + "REWIND", // 500 (758) + "RF", // 501 (759) + "RH", // 502 (760) + "RIGHT", // 503 (761) + "ROUNDED", // 504 (762) + "RUN", // 505 (763) + "SAME", // 506 (764) + "SCREEN", // 507 (765) + "SD", // 508 (766) + "SECONDS-FROM-FORMATTED-TIME", // 509 (767) + "SECONDS-PAST-MIDNIGHT", // 510 (768) + "SECURITY", // 511 (769) + "SEPARATE", // 512 (770) + "SEQUENCE", // 513 (771) + "SEQUENTIAL", // 514 (772) + "SHARING", // 515 (773) + "SIMPLE-EXIT", // 516 (774) + "SIGN", // 517 (775) + "SIN", // 518 (776) + "SIZE", // 519 (777) + "SMALLEST-ALGEBRAIC", // 520 (778) + "SOURCE", // 521 (779) + "SOURCE-COMPUTER", // 522 (780) + "SPECIAL-NAMES", // 523 (781) + "SQRT", // 524 (782) + "STACK", // 525 (783) + "STANDARD", // 526 (784) + "STANDARD-1", // 527 (785) + "STANDARD-DEVIATION", // 528 (786) + "STANDARD-COMPARE", // 529 (787) + "STATUS", // 530 (788) + "STRONG", // 531 (789) + "SUBSTITUTE", // 532 (790) + "SUM", // 533 (791) + "SYMBOL", // 534 (792) + "SYMBOLIC", // 535 (793) + "SYNCHRONIZED", // 536 (794) + "TALLYING", // 537 (795) + "TAN", // 538 (796) + "TERMINATE", // 539 (797) + "TEST", // 540 (798) + "TEST-DATE-YYYYMMDD", // 541 (799) + "TEST-DAY-YYYYDDD", // 542 (800) + "TEST-FORMATTED-DATETIME", // 543 (801) + "TEST-NUMVAL", // 544 (802) + "TEST-NUMVAL-C", // 545 (803) + "TEST-NUMVAL-F", // 546 (804) + "THAN", // 547 (805) + "TIME", // 548 (806) + "TIMES", // 549 (807) + "TO", // 550 (808) + "TOP", // 551 (809) + "TOP-LEVEL", // 552 (810) + "TRACKS", // 553 (811) + "TRACK-AREA", // 554 (812) + "TRAILING", // 555 (813) + "TRANSFORM", // 556 (814) + "TRIM", // 557 (815) + "TRUE", // 558 (816) + "TRY", // 559 (817) + "TURN", // 560 (818) + "TYPE", // 561 (819) + "TYPEDEF", // 562 (820) + "ULENGTH", // 563 (821) + "UNBOUNDED", // 564 (822) + "UNIT", // 565 (823) + "UNITS", // 566 (824) + "UNIT-RECORD", // 567 (825) + "UNTIL", // 568 (826) + "UP", // 569 (827) + "UPON", // 570 (828) + "UPOS", // 571 (829) + "UPPER-CASE", // 572 (830) + "USAGE", // 573 (831) + "USING", // 574 (832) + "USUBSTR", // 575 (833) + "USUPPLEMENTARY", // 576 (834) + "UTILITY", // 577 (835) + "UUID4", // 578 (836) + "UVALID", // 579 (837) + "UWIDTH", // 580 (838) + "VALIDATING", // 581 (839) + "VALUE", // 582 (840) + "VARIANCE", // 583 (841) + "VARYING", // 584 (842) + "VOLATILE", // 585 (843) + "WHEN-COMPILED", // 586 (844) + "WITH", // 587 (845) + "WORKING-STORAGE", // 588 (846) + "YEAR-TO-YYYY", // 589 (847) + "YYYYDDD", // 590 (848) + "YYYYMMDD", // 591 (849) + "ARITHMETIC", // 592 (850) + "ATTRIBUTE", // 593 (851) + "AUTO", // 594 (852) + "AUTOMATIC", // 595 (853) + "AWAY-FROM-ZERO", // 596 (854) + "BACKGROUND-COLOR", // 597 (855) + "BELL", // 598 (856) + "BINARY-ENCODING", // 599 (857) + "BLINK", // 600 (858) + "CAPACITY", // 601 (859) + "CENTER", // 602 (860) + "CLASSIFICATION", // 603 (861) + "CYCLE", // 604 (862) + "DECIMAL-ENCODING", // 605 (863) + "ENTRY-CONVENTION", // 606 (864) + "EOL", // 607 (865) + "EOS", // 608 (866) + "ERASE", // 609 (867) + "EXPANDS", // 610 (868) + "FLOAT-BINARY", // 611 (869) + "FLOAT-DECIMAL", // 612 (870) + "FOREGROUND-COLOR", // 613 (871) + "FOREVER", // 614 (872) + "FULL", // 615 (873) + "HIGHLIGHT", // 616 (874) + "HIGH-ORDER-LEFT", // 617 (875) + "HIGH-ORDER-RIGHT", // 618 (876) + "IGNORING", // 619 (877) + "IMPLEMENTS", // 620 (878) + "INITIALIZED", // 621 (879) + "INTERMEDIATE", // 622 (880) + "LC-ALL", // 623 (881) + "LC-COLLATE", // 624 (882) + "LC-CTYPE", // 625 (883) + "LC-MESSAGES", // 626 (884) + "LC-MONETARY", // 627 (885) + "LC-NUMERIC", // 628 (886) + "LC-TIME", // 629 (887) + "LOWLIGHT", // 630 (888) + "NEAREST-AWAY-FROM-ZERO", // 631 (889) + "NEAREST-EVEN", // 632 (890) + "NEAREST-TOWARD-ZERO", // 633 (891) + "NONE", // 634 (892) + "NORMAL", // 635 (893) + "NUMBERS", // 636 (894) + "PREFIXED", // 637 (895) + "PREVIOUS", // 638 (896) + "PROHIBITED", // 639 (897) + "RELATION", // 640 (898) + "REQUIRED", // 641 (899) + "REVERSE-VIDEO", // 642 (900) + "ROUNDING", // 643 (901) + "SECONDS", // 644 (902) + "SECURE", // 645 (903) + "SHORT", // 646 (904) + "SIGNED", // 647 (905) + "STANDARD-BINARY", // 648 (906) + "STANDARD-DECIMAL", // 649 (907) + "STATEMENT", // 650 (908) + "STEP", // 651 (909) + "STRUCTURE", // 652 (910) + "TOWARD-GREATER", // 653 (911) + "TOWARD-LESSER", // 654 (912) + "TRUNCATION", // 655 (913) + "UCS-4", // 656 (914) + "UNDERLINE", // 657 (915) + "UNSIGNED", // 658 (916) + "UTF-16", // 659 (917) + "UTF-8", // 660 (918) + "XML-DECLARATION", // 661 (919) + "XMLGENERATE", // 662 (920) + "XMLPARSE", // 663 (921) + "ADDRESS", // 664 (922) + "END-ACCEPT", // 665 (923) + "END-ADD", // 666 (924) + "END-CALL", // 667 (925) + "END-COMPUTE", // 668 (926) + "END-DELETE", // 669 (927) + "END-DISPLAY", // 670 (928) + "END-DIVIDE", // 671 (929) + "END-EVALUATE", // 672 (930) + "END-MULTIPLY", // 673 (931) + "END-PERFORM", // 674 (932) + "END-READ", // 675 (933) + "END-RETURN", // 676 (934) + "END-REWRITE", // 677 (935) + "END-SEARCH", // 678 (936) + "END-START", // 679 (937) + "END-STRING", // 680 (938) + "END-SUBTRACT", // 681 (939) + "END-UNSTRING", // 682 (940) + "END-WRITE", // 683 (941) + "END-XML", // 684 (942) + "END-IF", // 685 (943) + "ATTRIBUTES", // 686 (944) + "ELEMENT", // 687 (945) "NONNUMERIC", // 688 (946) - "XML-DECLARATION", // 689 (947) - "THRU", // 691 (949) - "OR", // 692 (950) - "AND", // 693 (951) - "NOT", // 694 (952) - "NE", // 695 (953) - "LE", // 696 (954) - "GE", // 697 (955) - "POW", // 698 (956) - "NEG", // 699 (957) + "THRU", // 689 (947) + "OR", // 690 (948) + "AND", // 691 (949) + "NOT", // 692 (950) + "NE", // 693 (951) + "LE", // 694 (952) + "GE", // 695 (953) + "POW", // 696 (954) + "NEG", // 697 (955) }; diff --git a/gcc/cobol/util.cc b/gcc/cobol/util.cc index c4927a9c459..b3633723f62 100644 --- a/gcc/cobol/util.cc +++ b/gcc/cobol/util.cc @@ -345,12 +345,14 @@ cdf_file_name( const cbl_file_t *file ) { return file->name; } - void cdf_field_add( const cbl_loc_t& loc, const std::string& name, const cdfval_t& value ) { + void scanner_cache_update( const symbol_elem_t *elem ); + if( symbols_begin() < symbols_end() ) { cbl_field_t field = cdf_literalize(loc, name, value); - symbol_field_add(current_program_index(), &field); + auto e = symbol_field_add(current_program_index(), &field); + scanner_cache_update( e ); } } diff --git a/libgcobol/posix/README.md b/libgcobol/posix/README.md index 2cd02f921bf..14c789edba7 100644 --- a/libgcobol/posix/README.md +++ b/libgcobol/posix/README.md @@ -103,3 +103,44 @@ the OS, each in their own way. GnuCOBOL implements some of those functions. The UDF produced by `udf-gen` is pure ISO COBOL. The code should be compilable by any ISO COBOL compiler. + +# TODO + +In the real world we know GnuCOBOL applications use: + +- clock_gettime +- getgrnam +- getpgid +- getpid +- getppid +- getpwnam +- getpwuid +- getuid +- gmtime_r +- kill +- localtime +- localtime_r +- MD5Final +- MD5Init +- MD5Update +- memcpy +- mktime +- msgctl +- msgget +- msgrcv +- msgsnd +- nanosleep +- rename +- semctl +- semget +- semop +- semtimedop +- setsid +- shmat +- shmctl +- shmdt +- shmget +- strftime +- system + +- diff --git a/libgcobol/posix/shim/open.cc b/libgcobol/posix/shim/open.cc index ed2a9d131b4..c449fae10a7 100644 --- a/libgcobol/posix/shim/open.cc +++ b/libgcobol/posix/shim/open.cc @@ -35,7 +35,6 @@ posix_open(const char *pathname, int cbl_flags, int cbl_mode) { { cbl::PSX_O_APPEND, O_APPEND }, { cbl::PSX_O_NONBLOCK, O_NONBLOCK }, { cbl::PSX_O_DSYNC, O_DSYNC }, - { cbl::PSX_O_LARGEFILE, O_LARGEFILE }, { cbl::PSX_O_SYNC, O_SYNC }, /* * The O_CLOEXEC, O_DIRECTORY, and O_NOFOLLOW flags are not specified in @@ -56,11 +55,13 @@ posix_open(const char *pathname, int cbl_flags, int cbl_mode) { */ #if defined(_GNU_SOURCE) { cbl::PSX_O_DIRECT, O_DIRECT }, + { cbl::PSX_O_LARGEFILE, O_LARGEFILE }, { cbl::PSX_O_NOATIME, O_NOATIME }, { cbl::PSX_O_PATH, O_PATH }, { cbl::PSX_O_TMPFILE, O_TMPFILE }, #else { cbl::PSX_O_DIRECT, 0 }, + { cbl::PSX_O_LARGEFILE, 0 }, { cbl::PSX_O_NOATIME, 0 }, { cbl::PSX_O_PATH, 0 }, { cbl::PSX_O_TMPFILE, 0 }, -- 2.34.1