When using undefined filetypes ...

Adriaan de Groot <adridg-FlD2LfDziEhmR6Xm/[email protected]>
Newsgroups gmane.comp.tools.aap.devel
Message-ID <[email protected]>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

While messing about with recipes for new filetypes - and in particular actions 
for filetypes that have no specific extension, or for which I just haven't 
bothered to introduce a suffix rule - I'm regularly bitten by typos. I 
declare an action :compile ltobject cpp and then :route cpp ltlibobject. So I 
thought it would be useful if AAP would warn the user when he or she uses an 
unknown filetype (one that hasn't appeared anywhere in a :filetype command). 
Then it shows up immediately when your :route command doesn't make sense, or 
there's a typo in your :action.

The attached patches do just that.

action.diff adds the checking part to the :action rule. I don't know how to 
print a sensible warning (there's aap_error(), but no aap_warning()), though.

commands.diff adds the same to :route.

filetype.diff does the main work. Ik adds a ft_known() funtion to the API of 
Filetypes. All filetypes that are declared through :filetype commands are 
added to the _filetype_dict dictionary and are considered "known". The 
predeclared types are added already in __init__. I've done a little work to 
extract the types that the Python snippets detect and add them too - I'm not 
sure that it's complete though.

Because it is possible that a filetype is legitimate but never mentioned in a 
:filetype rule (for instance, 

	:filetype
		python
			type = "otherfiletype"

) I've added an additional filetype rule "declare", which just says that a 
filetype is legitimate without adding any detection rules for it. The above 
example would become

	:filetype
		declare otherfiletype
		python
			type = "otherfiletype"

but using "declare" isn't mandatory - after all, it's just a warning. 

There is one minor issue with this whole patch: ft_known calls __init__ (to 
initialize the lists of filetypes). Previously, __init__ wasn't called until 
the first add_rules or ft_detect was done. Now, in ft_known, there is no 
recdict parameter, so None is passed to __init__. It doesn't look like the 
recdict parameter to __init__ is of great importance -- it's used only to 
pass to add_rules, which only uses it to pass it back to __init__. But if 
there is some secret very important reason for that parameter to be there, 
let me know.


doc.diff documents the additional :filetype declare rule, which might be 
needed for suppressing warnings for filetypes that have no other filetype 
rule at all (such as python-detected types).


- -- 
pub  1024D/FEA2A3FE 2002-06-18 Adriaan de Groot <[email protected]>
     Key fingerprint = 934E 31AA 80A7 723F 54F9  50ED 76AC EE01 FEA2 A3FE
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (FreeBSD)

iD8DBQE/R0YudqzuAf6io/4RAqatAJ9GH9tfP5aJy462Pw421d2pogjuVACdGK+2
S5EsFduq28NNdlYnlOaJOTI=
=pnLf
-----END PGP SIGNATURE-----
action.diff (text/x-diff, 1.8 KB)
--- orig/Action.py	Sat Aug 23 09:36:05 2003
+++ Action.py	Sat Aug 23 12:09:30 2003
@@ -30,7 +30,7 @@
 from Dictlist import listitem2str, str2dictlist, dictlist2str
 from ParsePos import ParsePos
 from Process import Process, recipe_error
-from Filetype import filetype_root
+from Filetype import filetype_root, ft_known
 from Commands import expand
 from Scope import get_build_recdict
 
@@ -59,18 +59,30 @@
     global _action_dict
 
     if len(arglist) == 2:
-        outtypes = "default"
-        intypes = arglist[1]["name"]
+        outtypes = ["default"]
+        intypes = string.split(arglist[1]["name"], ',')
     elif len(arglist) == 3:
-        outtypes = arglist[1]["name"]
-        intypes = arglist[2]["name"]
+        outtypes = string.split(arglist[1]["name"], ',')
+        intypes = string.split(arglist[2]["name"], ',')
     else:
         recipe_error(rpstack, _(':action must have two or three arguments'))
 
     act = Action(rpstack, recdict, arglist[0], commands)
+
+    
+    # Check that the types named exist
+    for i in intypes:
+        if i != "default" and not ft_known(i):
+	    # How to do a decent warning?
+	    print _("WARNING: :action for unknown filetype (%s) in %s line %d") % ( i,rpstack[-1].name,rpstack[-1].line_nr)
+    for i in outtypes:
+        if i != "default" and not ft_known(i):
+	    # How to do a decent warning?
+	    print _("WARNING: :action for unknown filetype (%s) in %s line %d") % ( i,rpstack[-1].name,rpstack[-1].line_nr)
+
     for action in string.split(arglist[0]["name"], ','):
-        for intype in string.split(intypes, ','):
-            for outtype in string.split(outtypes, ','):
+        for intype in intypes:
+            for outtype in outtypes:
                 if not _action_dict.has_key(action):
                     _action_dict[action] = {}
                 if not _action_dict[action].has_key(intype):
commands.diff (text/x-diff, 997 B)
diff -u orig/Commands.py ./Commands.py
--- orig/Commands.py	Sat Aug 23 09:36:05 2003
+++ ./Commands.py	Sat Aug 23 11:24:20 2003
@@ -38,6 +38,7 @@
 from DoRead import read_recipe, recipe_dir, did_read_recipe
 from DoArgs import doargs, add_cmdline_settings
 from Error import *
+from Filetype import ft_known
 import Global
 from Process import assert_var_name, assert_scope_name
 from Process import recipe_error, option_error, get_recipe_msg
@@ -264,6 +265,14 @@
     # it into a list of lists.
     typelist = map(lambda x: string.split(x, ","), typelist)
 
+    # Check that all the filetypes are known
+    for il in typelist:
+	for i in il:
+            if not ft_known(i):
+		# How to do warning properly?
+		print _('WARNING: :route given an unknown filetype (%s) in %s line %d') % (i,rpstack[-1].name,rpstack[-1].line_nr)
+
+    # Now add the routes themselves
     for in_type in typelist[0]:
         for out_type in typelist[-1]:
             route = work.find_route(in_type, out_type)
filetype.diff (text/x-diff, 5.2 KB)
diff -u orig/Filetype.py ./Filetype.py
--- orig/Filetype.py	Sat Aug 23 09:36:05 2003
+++ ./Filetype.py	Sat Aug 23 11:32:02 2003
@@ -26,8 +26,11 @@
 #
 # ft_add_rules(str, lnum [, recdict])
 #                           Add file type detection rules from "str".  See
-#                           the Aap reference manua for the syntax.
+#                           the Aap reference manual for the syntax.
 #
+# ft_known(type)
+#                           Returns true or false depending on whether
+#                           the filetype "type" is a known type.
 
 import string
 import os.path
@@ -1191,12 +1194,111 @@
 # Index in the list is "ignore".
 _cache_dict = [{}, {}]
 
+
+# Dictionary of known filetypes (just stores zeroes, the keys are important)
+_filetype_dict = {}
+
+_filetype_pre_list = [
+### List of types from the builtin python scripts
+### Generated by the following shell command:
+###
+### grep 'type[[:space:]]*=[[:space:]]*"' Filetype.py | \
+###	sed -e 's,""",,' | \
+###	sed -e 's,[^"]*",,' -e 's,".*,,' | \
+###	sort | uniq \
+###	sed -e 's,^,  ",' -e 's/$/",/'
+###
+### This is used to pre-populate _filetype_dict. See comment
+### "### End of the list of types." below for the end. Update
+### this list if the list of builtin Python detected-types changes.
+  "abaqus",
+  "asm",
+  "aspperl",
+  "aspvbs",
+  "automake",
+  "baan",
+  "basic",
+  "cfg",
+  "ch",
+  "change",
+  "cl",
+  "config",
+  "cvs",
+  "cweb",
+  "dcl",
+  "diff",
+  "diva",
+  "dns",
+  "dtd",
+  "eiffel",
+  "form",
+  "ishd",
+  "jargon",
+  "lprolog",
+  "m4",
+  "mail",
+  "mason",
+  "master",
+  "matlab",
+  "mib",
+  "mma",
+  "modsim3",
+  "moo",
+  "nroff",
+  "objc",
+  "pascal",
+  "perl",
+  "php",
+  "postscript",
+  "progress",
+  "prolog",
+  "psf",
+  "purifylog",
+  "rcslog",
+  "rebol",
+  "registry",
+  "rexx",
+  "sendpr",
+  "sgmldecl",
+  "sh",
+  "sicad",
+  "sindaout",
+  "smil",
+  "snnsnet",
+  "snnspat",
+  "snnsres",
+  "specman",
+  "strace",
+  "tads",
+  "takout",
+  "text",
+  "trasys",
+  "valgrind",
+  "vb",
+  "vim",
+  "virata",
+  "web",
+  "winbatch",
+  "xmath",
+  "xml",
+  "xpm",
+  "xpm2",
+  "xxd",
+  "zsh",
+### End of the list of types.
+###
+### The remainder of these types is internal to AAP and cannot be detected.
+  "libobject",
+  "ltlibobject"
+]
+
 _did_init = 0       # non-zero when __init__() did its work
 
 def __init__(recdict = None):
     global _suffix_dict, _regexp_list, _script_list
     global _py_list_before, _py_list_after
     global _did_init
+    global _filetype_dict
 
     # this only needs to be done once
     if _did_init:
@@ -1208,6 +1310,7 @@
     _regexp_list = []
     _script_list = []
     _py_list_after = []
+    _filetype_dict = {}
 
     # Load the built-in detection rules.
     _add_suffixlist(_def_suffix_list)
@@ -1221,6 +1324,9 @@
     for dir in default_dirs():
         ft_check_dir(os.path.join(dir, "afd"))
 
+    # Declare all the filetypes known from the builtin Python bits
+    for i in _filetype_pre_list:
+	_filetype_dict[i]=1
 
 class DetectError(Exception):
     """Error for something gone wrong."""
@@ -1228,6 +1334,9 @@
         Exception.__init__(self)
         self.args = args
 
+def ft_known(type):
+    __init__()
+    return _filetype_dict.has_key(type)
 
 def ft_check_dir(dir, errmsg = 0, recdict = None):
     """Check directory "dir" for *.afd files and load them.
@@ -1337,19 +1446,32 @@
         else:
             arg4 = ''
 
+        if type in ["declare"] and not arg1:
+            raise DetectError, (_('Missing argument in line %d: "%s"')
+                                           % (line_idx + recipe_line_nr, line))
+
         if type in ["suffix", "regexp", "script"] and not arg2:
             raise DetectError, (_('Missing argument in line %d: "%s"')
                                            % (line_idx + recipe_line_nr, line))
-        if type == "suffix":
+
+	# Just declare a filetype
+	if type == "declare":
+	    _filetype_dict[arg1]=1
+
+	# Filetype file file suffix
+        elif type == "suffix":
             _add_suffix(arg1, arg2)
 
+	# Filetype based on a regex match of the filename
         elif type == "regexp":
             _add_regexp(arg1, arg2, arg3 == "tail" or arg4 == "tail",
                                           arg3 == "append" or arg4 == "append")
 
+	# Filetype based on checking the #! line for an interpreter
         elif type == "script":
             _add_script(arg1, arg2, arg3 and arg3 == "append")
 
+	# Magic python-based filetype detection
         elif type == "python":
             append = 0
             after = 0
@@ -1447,12 +1569,14 @@
             del _suffix_dict[suf]
     else:
         _suffix_dict[suf] = type
+        _filetype_dict[type]=1
 
 
 def _add_suffixlist(list):
     """Add suffix rules from a list of suffix-type tuples."""
     for suf, type in list:
         _suffix_dict[suf] = type
+	_filetype_dict[type]=1
 
 
 def _add_regexp(regexp, type, tail, append):
@@ -1470,6 +1594,7 @@
             _regexp_list.append(f)
         else:
             _regexp_list.insert(0, f)
+	_filetype_dict[type]=1
 
 
 def _add_regexplist(list):
@@ -1489,6 +1614,7 @@
                 _script_list.remove(r)
     else:
         f = _Ft_re(regexp, type, 0)
+	_filetype_dict[type]=1
         if append:
             _script_list.append(f)
         else:
doc.diff (text/x-diff, 2.8 KB)
Only in .: .#exec.sgml.1.19
Only in .: CVS
Only in .: aap.mod
Only in .: exec.pdf
Only in .: exec.txt
Only in .: main.aap
Only in .: mysign
Only in .: orig
diff -u --exclude *.html orig/ref-filetype.sgml ./ref-filetype.sgml
--- orig/ref-filetype.sgml	Sat Aug 23 11:29:03 2003
+++ ./ref-filetype.sgml	Sat Aug 23 11:44:43 2003
@@ -303,6 +303,31 @@
     </listitem>
   </varlistentry>
 
+  <varlistentry id="filetype-declare"><term><cmdsynopsis>
+    <command>declare</command>
+    <arg choice="plain"><replaceable>type</replaceable></arg>
+    </cmdsynopsis></term>
+    <listitem>
+    <para>
+    <!-- rather inconsistent use of {} - it looks like an attribute -->
+    Declare {type} to be a recognized filetype.
+    This is needed for filetypes that are recognized through
+    Python code <emphasis>only</emphasis>.
+    All other filetypes (those that appear in suffix,
+    regexp, and script rules) need not be separately declared.
+    </para>
+    <para>
+    When you use an unknown filetype in a recipe,
+    &Aap; prints a warning to alert you to the possibility of
+    a misspelling.
+    The declare rule  is needed
+    because &Aap; cannot tell what filetype the 
+    Python code is capable of detecting,
+    so the declare rule is used to tell &Aap; 
+    specifically that the filetype {type} is a known and recognized type.
+    </para>
+    </listitem>
+  </varlistentry>
 </variablelist>
 
 <para>
diff -u --exclude *.html orig/user-filetype.sgml ./user-filetype.sgml
--- orig/user-filetype.sgml	Sat Aug 23 11:29:03 2003
+++ ./user-filetype.sgml	Sat Aug 23 12:44:41 2003
@@ -58,7 +58,7 @@
 </para>
 
 <para>
-For the syntax of the file see filetype.txt.
+For the syntax of the file see <xref linkend="ref-filetype">.
 </para>
 
 <para>
@@ -74,6 +74,29 @@
             OPTIMIZE = 3
             :do compile $source
 </programlisting>
+
+<!-- <note> -->
+<para>
+<!-- <indexterm><primary>:filetype</primary><secondary>declare</secondary></indexterm> -->
+When you define an action (or a route), &Aap; checks
+that the filetypes you use are known filetypes, i.e.&nbsp;mentioned
+somewhere in a <link linkend="cmd-filetype">:filetype</link>
+command. If you just make up filetypes and use them in actions,
+&Aap; will give you a warning.
+This helps detect misspellings and the like.
+However, for the "optimized C" filetype above,
+this leads to a warning where you do not want one:
+<literal>c_opt</literal> is a proper filetype in this context.
+In order to declare a filetype without giving 
+any rules to detect files of that type,
+use <literal>declare</literal> in a 
+<link linkend="cmd-filetype">:filetype</link> command:
+<programlisting>
+    :filetype
+	declare c_opt
+</programlisting>
+</para>
+<!-- </note> -->
 
 <para>
 The detected filetypes never contain an underscore.  A-A-P knows that the
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.