Search Lookup and classpath semantics

Further to the discussions on lookup and search. Today I've been
thinking some more on it.

Some background on why search and lookup exist...

First, what I call classpath order semantics. In Tapestry at runtime
resources are obtained from the classpath by calls to
Class#getResource(). So a file, X, in package foo in the first jar in
the classpath would hide another file named X in package foo if it was
in a jar later in the classpath. We must duplicate this in behaviour
in Spindle, but not always as will be seen below.

**The fact that classpath order sematics don't always hold is a
clarification of a statement in the porting guide some have already
seen and asked questions about.

Lookup (ICoreResource#lookup(IResourceAcceptor))
=======================================

A  lookup is used to find all the Resources that have the same path
(the same folder) as
the resource that lookup() is called on.

Why is this useful? Consider the Tapestry rule for finding Templates.
One of the rules states that Tapestry will look for templates in the
same folder as the spec file (ignore for now the case where no xml
spec file exists)

using find we would do it this way.

ICoreResource specfile = a Resource pointing to a spec.
final String specName = new
PathUtils(specfile.getName()).removeFileExtension().lastSegment();

IResourceAcceptor templateFinder = new IResourceAcceptor() {

   ArrayList<ICoreResource> results = new ArrayList<ICoreResource>();

  public boolean accept(ICoreResource location) {
    PathUtils utils = new PathUtils(location.getPath());
    if ("html".equals(utils.getFileExtension()) {

      if (specName.equals(utils.removeFileExtension().lastSegment()) {
         result.add(location);
         return false; // stop the lookup!
      }
    }
    return true; // continue the lookup!
  }

  ICoreResource [] getResults() {
     return results.toArray(new ICoreResource[0]);
  }
};

specfile.lookup(templateFinder);

ICoreResource [] templates = templateFinder.getResults();

etc.

THE IMPORTANT BIT

It is important to note that a lookup **must always follow classpath
order semantics**. Files that would be hidden at runtime *must* also
be hidden during the lookup.

END OF IMPORTANT BIT

ISearch#search(ISearchAcceptor acceptor)
=======

The code example for search is pretty much the same except you get the
search from an IResourceRoot.

IResourceRoot myRoot = ...;
ISearch searcher = myRoot.getSearch();

ISearchAcceptor acceptor = new ISearchAcceptor() {...fill in as above};
searcher.search(acceptor);

etc.

Search is used by the builder to collect up all the interesting
Tapestry files (.jwc, .page, .application, .library, templates) in the
project as a build is starting. The found artifacts are stored in the 
BuilderQueue. As the builder navigates though the project (using the
Tapestry lookup semantics) and encounters artifacts, we already know
it's in queue so after we finish parsing/processing/validating that
file we remove it from the BuilderQueue.

At the end of the build the only files left in the BuilderQueue are
files that Tapestry would not have picked up at runtime, so we can
record problems to display to the user ('Tapestry would not see this
file at runtime'). The severity of these problems is end user
configurable.

Up till recently, builder was using search on both roots, classpath
and context, to populate the BuilderQueue. In hindsight this was silly
as the resources in the jars are not files in the filesystem and in
Eclipse it's impossible to add a marker to a resource in a jar file.

And besides, the search in a jar file is extremely expensive using the
JDT api and was consuming up to 40% of the total build time in large
projects with many jars in the classpath.

I tweaked Spindle 3.2.X to skip most of the jars and saw a great
improvement is speed.

But the real improvement would be to restrict the search to the
context root + classpath root (excluding jar files).  I think this is
not an API issue but rather an IDE implementation issue.

The spot where IDE implementors can put in thier optimizations would
be in thier implementation of
AbstractBuildInfrastructure#findAllTapestrySourceFiles(Set<String>
knownTemplateExtensions,
            ArrayList<Resource> found);

which is where a Search would be used. And currently in Spindle this
is the *only* place a search is used.

So a Search is used to visit every Resource (no class or java files)
in every sub folder of a root. Not that classpath ordering semantics
is not an issue. We want to see *all* the resources

THE IMPORTANT BIT

Search *never* adheres to classpath order semantics. It would be
useless to omit files that would be hidden at runtime from a search.
If this was the case no (Tapestry would never see this file at
runtime) problems would ever be recorder againsr the 'hidden' files

few, this message is a lot longer than I though it would be.

hope that helps clear things up.

Geoff




--
The Spindle guy.          http://spindle.sf.net
Get help with Spindle:   
http://lists.sourceforge.net/mailman/listinfo/spindle-user
Blog:                     http://jroller.com/page/glongman
Feature Updates:          http://spindle.sf.net/updates


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid3432&bid#0486&dat1642