Re: [rvm-research] Errors while running a few benchmarks of DaCapo on RVM

Hi,

On 06.04.2013 14:47, Kathiravelu Pradeeban wrote:
> avrora, h2, tomcat, tradebeans, and tradesoap are not available
> (throwing "Unknown benchmark" message, when testing) in the dacapo.jar
> downloaded by the build.

That's right. The set of benchmarks that are available in the DaCapo
benchmark suite generally changes between releases. The downloaded
dacapo.jar is for DaCapo 2006-10-MR2 which does not contain any of those
benchmarks.

> batik gives the below error.
> sunflow gives the below.

Although DaCapo 2006-10-MR2 does not complain about using "batik" or
"sunflow" as arguments, the respective benchmarks aren't contained in
DaCapo 2006-10-MR2: you'll get the same error if you use a product VM
like OpenJDK.

> When testing with dacapo-9.12-bach.jar, these errors were noticed for
> these benchmarks that are unknown to dacapo.jar. Should I not test these
> benchmarks, as they are not included?

Benchmarks that are marked as unknown are not included and can't be tested.


Jikes RVM currently cannot run all of the DaCapo 9.12 benchmarks. It may
be useful to keep around a separate repository that tracks the current,
unmodfied development version. You can then use a build from that
repository to determine if a failure also exists in an unmodified
Jikes RVM or was introduced (or exposed) by your changes.

> tradebeans gives the below error.
> pradeeban@llovizna:~/jikesrvm/dist/BaseBaseMarkSweep_x86_64-linux$ ./rvm
> -jar dacapo-9.12-bach.jar tradebeans
> Using scaled threading model. 2 processors detected, 2 threads used to
> drive the workload, in a possible range of [1,512]
> Exception during initialization:
> java.lang.reflect.InvocationTargetException [...]

This error also occurs on head, so it does not need to concern you with
respect to the Compressor implementation.

> tradesoap gives the below error.
Same as tradebeans.

The rest of the failures don't look like the ones that we normally see
for DaCapo 9.12, so they might be related to the Compressor changes. In
particular, avrora, lusearch and sunflow work fine on my machine using
the development configuration (which uses GenImmix) on an unmodified
Jikes RVM.

The rest of this email contains some general notes about testing. Just
skip them if they aren't of interest you.

You can define your own test-runs (see "Testing the RVM" in the user
guide / on the website) and use the predefined test runs as inspiration.
The pre-commit test run is the test run that needs to succeed for code
that is committed to the main repository.

You'll likely want to start small. You can add more
-test cases (e.g. basic, opttests, dacapo, jni, gctest, ...)
-configs (FastAdaptiveCompressor, ...,
ExtremeAssertionsFullAdaptiveCompressor,
ExtremeAssertionsOptAdaptiveCompressor)
-requirements (e.g. checkstyle, unit tests)

as the stability of the implementation increases.

Don't hestitate to add assertions to the code if they are helpful for
you (but note that we use our own facilities instead of the assert
keyword).

A word about debug statements: It is advisable to guard those with a
static final boolean variable (e.g. DEBUG). This makes it easy to switch
them off and ensures that they won't take space if they aren't used.
Another common pattern for debug outputs is to use ints to control the
verbosity level.

Note that I'm not working in the area of garbage collection. Other
people will be able to give you better advice. You can also ask your
mentor if you're accepted. In particular, you may want to ask about
using the MMTk harness and about the general requirements for a new
collector in the Jikes RVM.


Kind regards,

Erik Brangs

------------------------------------------------------------------------------
Minimize network downtime and maximize team effectiveness.
Reduce network management and security costs.Learn how to hire 
the most talented Cisco Certified professionals. Visit the 
Employer Resources Portal
http://www.cisco.com/web/learning/employer_resources/index.html