Re: debugging remote JAI app

Tom Roche Sun, Jun 6, 2010 at 6:28 PM
>> I get

>> > WARNING: Can't load a service for category "MathTransformProvider". Cause is "NoClassDefFoundError: javax/media/jai/WarpAffine".
>> > Exception in thread "main" java.lang.NullPointerException
>> > at anl.verdi.core.VerdiApplication$1.messageReceived(VerdiApplication.java:86)
>> > at simphony.util.messages.MessageCenter.fireMessageEvent(MessageCenter.java:141)
>> > at simphony.util.messages.MessageCenter.error(MessageCenter.java:126)
>> > at saf.core.runtime.Boot.run(Boot.java:90)
>> > at saf.core.runtime.Boot.main(Boot.java:169)
...
>> what could differ in our environments on the cluster that could
>> cause [VERDI to throw for me but work for an admin? The] first
>> thing that comes to my mind is, java version.

(By which I meant, WarpAffine comes with admin's JRE's classpath, but
not with mine.)

Richard O. Hammer Sun, 06 Jun 2010 20:20:22 -0400 (rearranged)
> 2. The stack trace, being based in Boot.main, suggests to me that
> your call is not being run as a service (a service's thread which
> would start somewhere other than main), but is calling "java
> MainClass" more-or-less directly.

While not itself "a smoking gun," that seems indeed to point to one:

[me@cluster-login1 ~]$ ls -al $(which verdi)
> lrwxrwxrwx 1 mwaldron employee 8 May 19 10:43 /ifs1/apps/verdi-1.2/verdi_1.2/verdi -> verdi.sh
[me@cluster-login1 ~]$ ls -al /ifs1/apps/verdi-1.2/verdi_1.2/verdi.sh
> -rwxr-xr-x 1 mwaldron employee 503 Jun  2 16:30 /ifs1/apps/verdi-1.2/verdi_1.2/verdi.sh
[me@cluster-login1 ~]$ cat /ifs1/apps/verdi-1.2/verdi_1.2/verdi.sh
...
> cd $VERDI_HOME/plugins/bootstrap
> JAVA=${VERDI_HOME}/jre1.6.0_13/bin/java
> JAVAMAXMEM="-Xmx1024M"
> # Limit the number of default spawned threads (eca):
> JAVAOPTS="-XX:+UseParallelGC -XX:ParallelGCThreads=1"
> $JAVA -Duser.home=$HOME $JAVAOPTS $JAVAMAXMEM -classpath "./bootstrap.jar:./lib/saf.core.runtime.jar:./lib/commons-logging.jar:./lib/jpf-boot.jar:./lib/jpf.jar:./lib/log4j-1.2.13.jar" saf.core.runtime.Boot $*

but

[me@cluster-login1 ~]$ find ${VERDI_HOME}/plugins/bootstrap -name '*.jar' | xargs ls -al
> -rw-r--r-- ... ${VERDI_HOME}/plugins/bootstrap/bootstrap.jar
> -rw-r--r-- ... ${VERDI_HOME}/plugins/bootstrap/lib/commons-logging.jar
> -rw-r--r-- ... ${VERDI_HOME}/plugins/bootstrap/lib/j3dcore.jar
> -rw-r--r-- ... ${VERDI_HOME}/plugins/bootstrap/lib/j3dutils.jar
> -rw-r--r-- ... ${VERDI_HOME}/plugins/bootstrap/lib/jai_core.jar
> -rw-r--r-- ... ${VERDI_HOME}/plugins/bootstrap/lib/jai_imageio.jar
> -rw-r--r-- ... ${VERDI_HOME}/plugins/bootstrap/lib/jpf-boot.jar
> -rw-r--r-- ... ${VERDI_HOME}/plugins/bootstrap/lib/jpf.jar
> -rw-r--r-- ... ${VERDI_HOME}/plugins/bootstrap/lib/log4j-1.2.13.jar
> -rw-r--r-- ... ${VERDI_HOME}/plugins/bootstrap/lib/saf.core.runtime.jar
> -rw-r--r-- ... ${VERDI_HOME}/plugins/bootstrap/lib/vecmath.jar

Not all of which are in the classpath, particularly

[me@cluster-login1 ~]$ zipinfo ${VERDI_HOME}/plugins/bootstrap/lib/jai_core.jar | fgrep -e 'WarpAffine'
> -rw----     2.0 fat     4706 bl defN 17-Sep-04 11:03 javax/media/jai/WarpAffine.class

So the smoking gun appears to be that, if I

[me@cluster-login2 ~]$ cp /ifs1/apps/verdi-1.2/verdi_1.2/verdi.sh ~/bin/
[me@cluster-login2 ~]$ emacs -nw ~/bin/verdi.sh 
# Why did it take so long to get classpath wildcards?
[me@cluster-login2 ~]$ diff /ifs1/apps/verdi-1.2/verdi_1.2/verdi.sh ~/bin/
13c13
< $JAVA -Duser.home=$HOME $JAVAOPTS $JAVAMAXMEM -classpath "./bootstrap.jar:./lib/saf.core.runtime.jar:./lib/commons-logging.jar:./lib/jpf-boot.jar:./lib/jpf.jar:./lib/log4j-1.2.13.jar" saf.core.runtime.Boot $*
---
> $JAVA -Duser.home=$HOME $JAVAOPTS $JAVAMAXMEM -classpath "./bootstrap.jar:./lib/*" saf.core.runtime.Boot $*

and then call the original script

[me@cluster-login2 ~]$ verdi -f "${VERDI_DATA}/model/CCTM46_P16.baseO2a.36k.O3MAX -s O3[1] -gtype tile" &
[1] 30729
Jun 6, 2010 11:39:35 PM FactoryRegistry scanForPlugins
WARNING: Can't load a service for category "MathTransformProvider". Cause is "NoClassDefFoundError: javax/media/jai/WarpAffine".
Exception in thread "main" java.lang.NullPointerException
	at anl.verdi.core.VerdiApplication$1.messageReceived(VerdiApplication.java:86)
	at simphony.util.messages.MessageCenter.fireMessageEvent(MessageCenter.java:141)
	at simphony.util.messages.MessageCenter.error(MessageCenter.java:126)
	at saf.core.runtime.Boot.run(Boot.java:90)
	at saf.core.runtime.Boot.main(Boot.java:169)
[1]+  Exit 1                  verdi -f "${VERDI_DATA}/model/CCTM46_P16.baseO2a.36k.O3MAX -s O3[1] -gtype tile"

But if I call the new'n'improved script ...

[me@cluster-login2 ~]$ ~/bin/verdi.sh -f "${VERDI_DATA}/model/CCTM46_P16.baseO2a.36k.O3MAX -s O3[1] -gtype tile" & 
[1]+  Done                    ~/bin/verdi.sh -f "${VERDI_DATA}/model/CCTM46_P16.baseO2a.36k.O3MAX -s O3[1] -gtype tile"

... I get a plot! Thanks, Rich!