Java Fixes
Edward John Steere <[email protected]> Sat, 05 Nov 2016 12:38:44 +0200
| Newsgroups | gmane.emacs.semantic |
|---|---|
| Message-ID | <[email protected]> |
--=-=-= Content-Type: text/plain Hi All, Today I'm submitting a series of five patches. These provide fixes for: - Annotations in java. There was a problem with the syntax of `@' in java buffers. It should now correctly recognise annotations as extra modifiers on tags in java buffers and not cause a parser error. You should find that CEDET now finds function, class and member declarations which it previously wasn't -- in cases where they were annotated. - Diamond syntax in java. This is a fix to prevent the parser from bombing out when no type arguments are present inside of diamonds. Again this will help the parser to recognise more modern java files correctly where it wasn't before. - Starry imports in java. There was support for starry imports in java but it was broken. My final fix returns it to working order. You should now be able to complete types imported with star imports. I've also improved the testing for java via changes made in `cit-javaroot.el'. --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Recognise-annotations-in-java-grammar.patch Content-Description: Part 1: Regonise Annotations >From c15f4d13d78393e0cb874b172068e313aad257bc Mon Sep 17 00:00:00 2001 From: Edward John Steere <[email protected]> Date: Sun, 11 Sep 2016 12:40:52 +0200 Subject: [PATCH 1/5] Recognise annotations in java grammar * lisp/cedet/semantic/wisent/java.el (wisent-java-default-setup): change syntax of @ to punctuation in java * lisp/cedet/semantic/wisent/java.wy (annotation): AT name -> AT IDENTIFIER * tests/cit-javaroot.el (cit-src-javaroot-main-tags): added annotation to list of recognised tags * tests/integ_src/javaroot/TestMain.java (main): added annotation --- lisp/cedet/semantic/wisent/java.el | 1 + lisp/cedet/semantic/wisent/java.wy | 10 +++++----- tests/cit-javaroot.el | 6 +++--- tests/integ_src/javaroot/TestMain.java | 2 +- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/lisp/cedet/semantic/wisent/java.el b/lisp/cedet/semantic/wisent/java.el index 0a38545..f039696 100644 --- a/lisp/cedet/semantic/wisent/java.el +++ b/lisp/cedet/semantic/wisent/java.el @@ -56,6 +56,7 @@ names in scope." 'wisent-java-init-parser-context nil t) (setq ;; Lexical analysis + semantic-lex-syntax-modifications '((?\@ ".")) semantic-lex-number-expression semantic-java-number-regexp semantic-lex-depth nil semantic-lex-analyzer 'wisent-java-lexer diff --git a/lisp/cedet/semantic/wisent/java.wy b/lisp/cedet/semantic/wisent/java.wy index 3abedd5..187857c 100644 --- a/lisp/cedet/semantic/wisent/java.wy +++ b/lisp/cedet/semantic/wisent/java.wy @@ -2070,15 +2070,15 @@ default_value ;; SingleMemberAnnotation: ;; @ TypeName ( MemberValue ) annotation - : AT name + : AT IDENTIFIER (concat $1 $2) - | AT name LPAREN member_value RPAREN + | AT IDENTIFIER LPAREN member_value RPAREN (concat $1 $2) - | AT name LPAREN member_value_pairs RPAREN + | AT IDENTIFIER LPAREN member_value_pairs RPAREN (concat $1 $2) - | AT name LPAREN RPAREN + | AT IDENTIFIER LPAREN RPAREN (concat $1 $2) - | AT name LPAREN error + | AT IDENTIFIER LPAREN error (prog1 (concat $1 $2) ;; On error, skip current block and try to continue. (SKIP-BLOCK $3)) diff --git a/tests/cit-javaroot.el b/tests/cit-javaroot.el index d5e187e..12e5454 100644 --- a/tests/cit-javaroot.el +++ b/tests/cit-javaroot.el @@ -64,8 +64,8 @@ "main" "void" (list (semantic-tag-new-variable "args" "String" nil :dereference 1)) - :typemodifiers '("public" "static")) - ) + :typemodifiers '("@Deprecated" "public" "static")) + ) nil ;; parents :typemodifiers '("public")) ) @@ -90,7 +90,7 @@ ;; Copy source files into the javaroot directory (condition-case nil ;; Emacs 24.2 - (copy-directory (file-name-as-directory cit-integ-javaroot-srcdir) cit-integ-target-javaroot t t t) + (copy-directory (file-name-as-directory cit-integ-javaroot-srcdir) cit-integ-target-javaroot t t t) ;; Emacs 23 (error diff --git a/tests/integ_src/javaroot/TestMain.java b/tests/integ_src/javaroot/TestMain.java index 508bc30..e545922 100644 --- a/tests/integ_src/javaroot/TestMain.java +++ b/tests/integ_src/javaroot/TestMain.java @@ -30,7 +30,7 @@ import java.io.InputStream; public class TestMain { - public static void main(String[] args) { + @Deprecated public static void main(String[] args) { System.out.println("Hello, World"); } -- 2.9.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0002-Added-support-for-diamond-syntax-in-java-grammar.patch Content-Description: Part 2: Prevent parser from bombing out on diamond syntax >From fa0eb58274bba34c901f7f6e2fd78c169fb35c05 Mon Sep 17 00:00:00 2001 From: Edward John Steere <[email protected]> Date: Sun, 11 Sep 2016 12:43:46 +0200 Subject: [PATCH 2/5] Added support for diamond syntax in java grammar Grammar recognises the open close diamond as an empty type parameter specifier. It doesn't account for the diamond appearing on the LHS of an expression (which can be incorrect.) * lisp/cedet/semantic/wisent/java.wy (type_argument_list_1): Added case for empty list of arguments. * tests/cit-javaroot.el (cit-ede-javaroot-test): Ensure that no tags go unrecognised. * tests/integ_src/javaroot/TestMain.java (TestMain): Added generic type imports and construct a generic type without listing the type arguments in the constructor. --- lisp/cedet/semantic/wisent/java.wy | 1 + tests/cit-javaroot.el | 4 ++++ tests/integ_src/javaroot/TestMain.java | 4 ++++ 3 files changed, 9 insertions(+) diff --git a/lisp/cedet/semantic/wisent/java.wy b/lisp/cedet/semantic/wisent/java.wy index 187857c..0fe33cd 100644 --- a/lisp/cedet/semantic/wisent/java.wy +++ b/lisp/cedet/semantic/wisent/java.wy @@ -569,6 +569,7 @@ type_argument_list_1 : type_argument_1 | type_argument_list COMMA type_argument_1 (concat $1 $2 $3) + | GT ;; EMPTY for type inference ; type_argument_list_2 diff --git a/tests/cit-javaroot.el b/tests/cit-javaroot.el index 12e5454..2dd2d46 100644 --- a/tests/cit-javaroot.el +++ b/tests/cit-javaroot.el @@ -114,6 +114,10 @@ ;; Validate found tags based on project provided macros. (cit-srecode-verify-tags (semantic-fetch-tags) cit-src-javaroot-main-tags) + + ;; Ensure that the file contains no unmatched syntax (checks the use + ;; of diamond notation) + (null (semantic-unmatched-syntax-tokens)) ;; Test out the include paths by checking the discovered file names for the includes. (let ((itag (semantic-find-tags-included (current-buffer))) diff --git a/tests/integ_src/javaroot/TestMain.java b/tests/integ_src/javaroot/TestMain.java index e545922..9da208e 100644 --- a/tests/integ_src/javaroot/TestMain.java +++ b/tests/integ_src/javaroot/TestMain.java @@ -28,10 +28,14 @@ import syslib.TestSysJar; import java.io.InputStream; +import java.util.Map; +import java.util.HashMap; + public class TestMain { @Deprecated public static void main(String[] args) { System.out.println("Hello, World"); + final Map<String, Integer> map = new HashMap<>(); } } // TestMain -- 2.9.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0003-Fixed-tests.patch Content-Description: Part 3: Fixed tests >From 6ca2db9a515b6029fcfd99860d304b3e8bcbd71f Mon Sep 17 00:00:00 2001 From: Edward John Steere <[email protected]> Date: Fri, 4 Nov 2016 21:20:12 +0200 Subject: [PATCH 3/5] Fixed tests * tests/cit-javaroot.el (cit-src-javaroot-main-tags): Updated for changes in 4df1aec. * tests/cit-javaroot.el (cit-javaroot-depfiles): Updated for changes in 4df1aec. * tests/cit-javaroot.el (cit-ede-javaroot-test): Use the wisent parser for java in tests. --- tests/cit-javaroot.el | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/tests/cit-javaroot.el b/tests/cit-javaroot.el index 2dd2d46..1194760 100644 --- a/tests/cit-javaroot.el +++ b/tests/cit-javaroot.el @@ -56,14 +56,16 @@ (semantic-tag-new-include "testproj.TestInJar" nil) (semantic-tag-new-include "syslib.TestSysJar" nil) (semantic-tag-new-include "java.io.InputStream" nil) + (semantic-tag-new-include "java.util.Map" nil) + (semantic-tag-new-include "java.util.HashMap" nil) - (semantic-tag-new-type + (semantic-tag-new-type "TestMain" "class" (list (semantic-tag-new-function "main" "void" (list (semantic-tag-new-variable "args" "String" nil - :dereference 1)) + :dereference 1)) :typemodifiers '("@Deprecated" "public" "static")) ) nil ;; parents @@ -72,18 +74,24 @@ "List of tags we need to be able to to find in main.java") (defvar cit-javaroot-depfiles - (list + (list (list 'semanticdb-table (expand-file-name "TestLib.java" cit-integ-target-javaroot)) (list 'semanticdb-table (expand-file-name "test/TestTest.java" cit-integ-target-javaroot)) (list 'semanticdb-table-jar-file "testproj/TestInJar.class") (list 'semanticdb-table-jar-file "syslib/TestSysJar.class") (list 'semanticdb-table-jar-file "java/io/InputStream.class") + (list 'semanticdb-table-jar-file "java/util/Map.class") + (list 'semanticdb-table-jar-file "java/util/HashMap.class") ;;(expand-file-name "TestLib.h" cit-integ-javaroot-sys-srcdir) ) "List of expected path names to include files found in TestMain.java") (defun cit-ede-javaroot-test () "Test the ede-java-root project type." + ;; Setup the java wisent parser + (autoload 'wisent-java-default-setup "semantic/wisent/java" + "Hook run to setup Semantic in `java-mode'." nil nil) + ;; Create directory for the project (cit-make-dir cit-integ-target-javaroot) -- 2.9.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0004-Fixed-starry-imports-in-db-javap.patch Content-Description: Part 4: Fixed starry imports in db-javap >From 277becb28f7f27f827d8f9aa5f99d15054843483 Mon Sep 17 00:00:00 2001 From: Edward John Steere <[email protected]> Date: Sat, 5 Nov 2016 09:07:41 +0200 Subject: [PATCH 4/5] Fixed starry imports in db-javap * lisp/cedet/semantic/db-javap.el (semanticdb-normalize-tags): use full tag file path to extract tags from jar. * lisp/cedet/semantic/db-javap.el (semanticdb-normalize-tags): find tags by name with the table-jar-directory and the table which could have been found from extracting tags * lisp/cedet/semantic/db-javap.el (semanticdb-java-jar-class-file-regex): a more accurate regular expression for matching class files from a jar * lisp/cedet/semantic/db-javap.el (semanticdb-java-jar-package-files): use the more accurate regular expression when matching class files --- lisp/cedet/semantic/db-javap.el | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/lisp/cedet/semantic/db-javap.el b/lisp/cedet/semantic/db-javap.el index 5404777..a1529c6 100644 --- a/lisp/cedet/semantic/db-javap.el +++ b/lisp/cedet/semantic/db-javap.el @@ -374,15 +374,17 @@ The default tag just has a name, type, and the filename. Normalize by loading in the file it belongs to, and looking up that symbol in the file and returning that tag instead." (let ((tagret nil) - (parentdb (oref obj parent-db))) + (parentdb (oref obj parent-db))) (dolist (T tags) - (let* ((tfn (semantic-tag-file-name T)) - (realtable (semanticdb-jar-extract-and-save-tags obj tfn)) - (foundtags (semanticdb-find-tags-by-name-method - realtable (semantic-tag-name T)))) - (dolist (FT foundtags) - (semantic--tag-put-property FT :filename tfn) - (setq tagret (cons FT tagret))))) + (let* ((tfbn (semantic-tag-file-name T)) + (tfn (concat (semantic--tag-get-property T :packagedir) + tfbn)) + (realtable (semanticdb-jar-extract-and-save-tags obj tfn)) + (foundtags (semanticdb-find-tags-by-name-method + obj (semantic-tag-name T) realtable))) + (dolist (FT foundtags) + (semantic--tag-put-property FT :filename tfbn) + (setq tagret (cons FT tagret))))) tagret)) (defmethod semanticdb-javap-resolve-proxy ((obj semanticdb-table-jar-directory) tag) @@ -715,13 +717,17 @@ If the table for DIR does not exist, create one." ))) newtab))) +(defconst semanticdb-java-jar-class-file-regex + "\\(\\([a-zA-Z_][a-zA-Z_0-9]*\\.class$\\)\\|\\([a-zA-Z_]*\\.class$\\)\\)" + "A regular expression for class files in a folder structure.") + (defmethod semanticdb-java-jar-package-files ((dbc semanticdb-java-jar-database) dir) "Get the file class names from DBC that match DIR." (when (stringp dir) (setq dir (file-name-as-directory dir)) (let ((ans nil)) (dolist (F (oref dbc jarfilecache)) - (when (string-match (concat "^" (regexp-quote dir) "[a-zA-Z_]*\\.class$") + (when (string-match (concat "^" (regexp-quote dir) semanticdb-java-jar-class-file-regex) F) (push F ans))) (nreverse ans)))) -- 2.9.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0005-Test-starry-imports.patch Content-Description: Part 5: Provide tests for starry imports including completion >From e84bd5b2ad210faebad893f61504d162cc18d3a5 Mon Sep 17 00:00:00 2001 From: Edward John Steere <[email protected]> Date: Sat, 5 Nov 2016 12:00:17 +0200 Subject: [PATCH 5/5] Test starry imports * tests/cit-javaroot.el (cit-src-javaroot-main-tags): Added extra tags for verification. * tests/cit-javaroot.el (cit-javaroot-depfiles): Added starry import tag for verification. * tests/cit-javaroot.el (cit-javaroot-expected-completion): Added expected completion from starry import type for verification. * tests/cit-javaroot.el (cit-ede-javaroot-test): Added step to verify completions from type included via starry import. --- tests/cit-javaroot.el | 34 ++++++++++++++++++++++++++++++---- tests/integ_src/javaroot/TestMain.java | 11 +++++++++-- 2 files changed, 39 insertions(+), 6 deletions(-) diff --git a/tests/cit-javaroot.el b/tests/cit-javaroot.el index 1194760..b27b0af 100644 --- a/tests/cit-javaroot.el +++ b/tests/cit-javaroot.el @@ -58,19 +58,25 @@ (semantic-tag-new-include "java.io.InputStream" nil) (semantic-tag-new-include "java.util.Map" nil) (semantic-tag-new-include "java.util.HashMap" nil) + (semantic-tag-new-include "java.text.*" nil) (semantic-tag-new-type "TestMain" "class" (list + '("DATE_FORMAT" variable + (:typemodifiers ("private" "static" "final") :type "String") + (reparse-symbol field_declaration)) (semantic-tag-new-function "main" "void" (list (semantic-tag-new-variable "args" "String" nil :dereference 1)) :typemodifiers '("@Deprecated" "public" "static")) - ) + (semantic-tag-new-function + "testStarryImport" "DateFormat" + nil + :typemodifiers '("private"))) nil ;; parents - :typemodifiers '("public")) - ) + :typemodifiers '("public"))) "List of tags we need to be able to to find in main.java") (defvar cit-javaroot-depfiles @@ -82,10 +88,18 @@ (list 'semanticdb-table-jar-file "java/io/InputStream.class") (list 'semanticdb-table-jar-file "java/util/Map.class") (list 'semanticdb-table-jar-file "java/util/HashMap.class") + (list 'semanticdb-table-jar-directory nil) ;;(expand-file-name "TestLib.h" cit-integ-javaroot-sys-srcdir) ) "List of expected path names to include files found in TestMain.java") +(defvar cit-javaroot-expected-completion + '(("SimpleDateFormat" + type (:type "class") + (:filename "SimpleDateFormat.class" :faux-flag t :packagedir "java/text/" :proxy :tag-proxy) + nil)) + "A tag which we expect to find as the completion of `SimpleDate' in TestMain.java.") + (defun cit-ede-javaroot-test () "Test the ede-java-root project type." ;; Setup the java wisent parser @@ -153,7 +167,19 @@ expected (cdr expected))) (when (or itag expected) (error "Number of found include tags does not match number of expected tags."))) - ) + + ;; Test that completion detects SimpleDateFormat as a completion + (progn + (goto-char (point-min)) + (search-forward "SimpleDateFormat") + (back-to-indentation) + (open-line 2) + (forward-char) + (insert "final DateFormat format = new SimpleDate") + (let ((completion (semantic-analyze-possible-completions (semantic-analyze-current-context)))) + (when (null completion) + (error "No completions found for starry import")) + (cit-srecode-verify-tags cit-javaroot-expected-completion completion)))) (defun cit-file-javaroot (filename) "Return a testing filename. diff --git a/tests/integ_src/javaroot/TestMain.java b/tests/integ_src/javaroot/TestMain.java index 9da208e..1ae72d7 100644 --- a/tests/integ_src/javaroot/TestMain.java +++ b/tests/integ_src/javaroot/TestMain.java @@ -8,12 +8,12 @@ * modify it under the terms of the GNU General Public License as * published by the Free Software Foundation, either version 3 of the * License, or (at your option) any later version. - * + * * This program is distributed in the hope that it will be useful, but * WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU * General Public License for more details. - * + * * You should have received a copy of the GNU General Public License * along with this program. If not, see http://www.gnu.org/licenses/. */ @@ -31,11 +31,18 @@ import java.io.InputStream; import java.util.Map; import java.util.HashMap; +import java.text.*; + public class TestMain { + private static final String DATE_FORMAT = "yyyyMMdd"; @Deprecated public static void main(String[] args) { System.out.println("Hello, World"); final Map<String, Integer> map = new HashMap<>(); } + private DateFormat testStarryImport() { + return new SimpleDateFormat(DATE_FORMAT); + } + } // TestMain -- 2.9.0 --=-=-= Content-Type: text/plain Kind Regards, Edward Steere --=-=-= Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------------ Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi --=-=-= Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ cedet-semantic mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/cedet-semantic --=-=-=--