Re: [VOTE] Release Candidate JMeter 2.3 RC3

sebb <[email protected]>
Newsgroups gmane.comp.jakarta.general
Message-ID <[email protected]> 
On 09/07/07, Jörg Schaible <[email protected]> wrote:
> Hi Sebastian,
>
> sebb wrote:
>
> > Following on from the helpful feedback on RC1, I've now created JMeter
> > 2.3 RC3 in the directory:
> >
> > http://people.apache.org/~sebb/jmeter-2.3/dist
> >
> > Site documentation is here:
> > http://people.apache.org/~sebb/jmeter-2.3/site
> >
> >
> > It should now build and test OK on Unix (and in non-English Locales).
> >
> > [Please note that the 2 BeanShell tests require the optional beanshell
> > jar to be downloaded from www.beanshell.org and put in the lib/opt
> > directory.]
> >
> > All feedback welcome.
> >
> >
> > [ ]+1 - the release candidate looks OK, proceed with releasing it
> > [ ]-1 - there is a problem (please indicate what it is)
> >
> > There have been quite a few major updates since 2.2, so I think it
> > would be prudent to release 2.3 initially as a release candidate; a
> > full release can follow in a month or so.
> >
> > sebb AT apache DOT org
>
> After using the offical bsh-2.0-beta-4.jar from the website (and not the one
> provided by Gentoo) the tests run much better. I build now RC3 with my
> complete compiler zoo, running "ant all test".

Glad that's sorted.

> One strange error is always present: The ClassFinder detects 72 TestCase
> classes, but 1 cannot be instantiated:
>
> ========= %< ============
>     [java] Creating test suite
>     [java]
> Scanning /home/joehni/java/jakarta-jmeter-2.3RC3/build/test,../lib/ext for
> test cases
>     [java] ClassFinder found: 72 TestCase classes
>     [java] INFO: JMeterGUIComponent: skipping some tests
> org.apache.jmeter.testbeans.gui.TestBeanGUI
>     [java] Error adding test for class
> org.apache.jmeter.protocol.http.sampler.TestHTTPSamplersAgainstHttpMirrorServer
> java.lang.ClassCastException:
> org.apache.jmeter.protocol.http.sampler.TestHTTPSamplersAgainstHttpMirrorServer$1

I changed the test recently to use a one-time TestSetup - this returns
a Test, not a TestSuite.

The changed test worked fine in Eclipse, but I see now that it does
not work in Ant - unfortunately I overlooked that line in the output
... instead of prining a message, perhaps I should add a dummy test
that is bound to fail.

Thanks for finding it.

>     [java] Created: 71 tests including 5 suites
>     [java] Starting test run, test count = 1286
> ========= %< ============
>
> Apart from this following JDKs run all the tests fine:
> - Blackdown 1.4.2_03
> - Sun JDK 1.5.0_12
> - Sun JDK 1.6.0_01
>

Good.

> These JDKs failed:
> - IBM JDK 1.4.2_08 with
> ========= %< ============
>     [java] There were 4 failures:
>     [java] 1)
> testAssertionBadXSDFile(org.apache.jmeter.assertions.XMLSchemaAssertionTest)junit.framework.AssertionFailedError
>     [java]     at
> org.apache.jmeter.assertions.XMLSchemaAssertionTest.testAssertionBadXSDFile(XMLSchemaAssertionTest.java:107)
>     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:85)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:58)
>     [java]     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java(Compiled
> Code))
>     [java]     at org.apache.jorphan.test.AllTests.main(AllTests.java:214)
>     [java] 2)
> testAssertionBlankResult(org.apache.jmeter.assertions.XMLSchemaAssertionTest)junit.framework.AssertionFailedError
>     [java]     at
> org.apache.jmeter.assertions.XMLSchemaAssertionTest.testAssertionBlankResult(XMLSchemaAssertionTest.java:151)
>     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:85)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:58)
>     [java]     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java(Compiled
> Code))
>     [java]     at org.apache.jorphan.test.AllTests.main(AllTests.java:214)
>     [java] 3)
> testXMLTrailingcontent(org.apache.jmeter.assertions.XMLSchemaAssertionTest)junit.framework.AssertionFailedError
>     [java]     at
> org.apache.jmeter.assertions.XMLSchemaAssertionTest.testXMLTrailingcontent(XMLSchemaAssertionTest.java:164)
>     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:85)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:58)
>     [java]     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java(Compiled
> Code))
>     [java]     at org.apache.jorphan.test.AllTests.main(AllTests.java:214)
>     [java] 4)
> testAssertionBlankResult(org.apache.jmeter.assertions.XPathAssertionTest)junit.framework.AssertionFailedError
>     [java]     at
> org.apache.jmeter.assertions.XPathAssertionTest.testAssertionBlankResult(XPathAssertionTest.java:183)
>     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:85)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:58)
>     [java]     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java(Compiled
> Code))
>     [java]     at org.apache.jorphan.test.AllTests.main(AllTests.java:214)
>     [java]
>     [java] FAILURES!!!
>     [java] Tests run: 1286,  Failures: 4,  Errors: 0
> ========= %< ============
>
> - IBM JDK 1.5.0_05 with
> ========= %< ============
>     [java] There were 4 failures:
>     [java] 1)
> testAssertionBadXSDFile(org.apache.jmeter.assertions.XMLSchemaAssertionTest)junit.framework.AssertionFailedError
>     [java]     at
> org.apache.jmeter.assertions.XMLSchemaAssertionTest.testAssertionBadXSDFile(XMLSchemaAssertionTest.java:107)
>     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
>     [java]     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     [java]     at org.apache.jorphan.test.AllTests.main(AllTests.java:214)
>     [java] 2)
> testAssertionBlankResult(org.apache.jmeter.assertions.XMLSchemaAssertionTest)junit.framework.AssertionFailedError
>     [java]     at
> org.apache.jmeter.assertions.XMLSchemaAssertionTest.testAssertionBlankResult(XMLSchemaAssertionTest.java:151)
>     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
>     [java]     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     [java]     at org.apache.jorphan.test.AllTests.main(AllTests.java:214)
>     [java] 3)
> testXMLTrailingcontent(org.apache.jmeter.assertions.XMLSchemaAssertionTest)junit.framework.AssertionFailedError
>     [java]     at
> org.apache.jmeter.assertions.XMLSchemaAssertionTest.testXMLTrailingcontent(XMLSchemaAssertionTest.java:164)
>     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
>     [java]     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     [java]     at org.apache.jorphan.test.AllTests.main(AllTests.java:214)
>     [java] 4)
> testAssertionBlankResult(org.apache.jmeter.assertions.XPathAssertionTest)junit.framework.AssertionFailedError
>     [java]     at
> org.apache.jmeter.assertions.XPathAssertionTest.testAssertionBlankResult(XPathAssertionTest.java:183)
>     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
>     [java]     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     [java]     at org.apache.jorphan.test.AllTests.main(AllTests.java:214)
>     [java]
>     [java] FAILURES!!!
>     [java] Tests run: 1286,  Failures: 4,  Errors: 0
> ========= %< ============
>
> - BEA JRockit 1.4.2_13 with
> ========= %< ============
>     [java] There was 1 failure:
>     [java] 1)
> testSubResults(org.apache.jmeter.samplers.TestSampleResult)junit.framework.AssertionFailedError:
> expected:<304> but was:<305>

This is a bug in the test; it assumes that there is no gap between
generating two samples, even though there is actually some processing
that occurs. I don't remember seeing this before.

I obviously need to use a range check.

>     [java]     at
> org.apache.jmeter.samplers.TestSampleResult.testSubResults(TestSampleResult.java:159)
>     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [java]     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [java]     at org.apache.jorphan.test.AllTests.main(AllTests.java:214)
>     [java]
>     [java] FAILURES!!!
>     [java] Tests run: 1286,  Failures: 1,  Errors: 0
> ========= %< ============
>
> - BEA JRockit 1.5.0_10 with
> ========= %< ============
>     [java] There was 1 failure:
>     [java] 1)
> testSubResults(org.apache.jmeter.samplers.TestSampleResult)junit.framework.AssertionFailedError:
> expected:<304> but was:<305>

Same as previous.

>     [java]     at
> org.apache.jmeter.samplers.TestSampleResult.testSubResults(TestSampleResult.java:159)
>     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [java]     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [java]     at org.apache.jorphan.test.AllTests.main(AllTests.java:214)
>     [java]
>     [java] FAILURES!!!
>     [java] Tests run: 1286,  Failures: 1,  Errors: 0
> ========= %< ============
>
> - Sun JDK 1.4.2_15 with
> ========= %< ============
>     [java] There was 1 failure:
>     [java] 1)
> testElapsed(org.apache.jmeter.samplers.TestSampleResult)junit.framework.AssertionFailedError:
> Sample time should be >=100, actual 99

This is a timing issue, and will probably disappear if you run the test again.

I ought to change the check - a Thread.sleep(100) may be slightly less
than 100 as shown above.

>     [java]     at
> org.apache.jmeter.samplers.TestSampleResult.testElapsed(TestSampleResult.java:47)
>     [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>     [java]     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     [java]     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     [java]     at org.apache.jorphan.test.AllTests.main(AllTests.java:214)
>     [java]
>     [java] FAILURES!!!
>     [java] Tests run: 1286,  Failures: 1,  Errors: 0
> ========= %< ============
>
> Again, all logs are sent in private. The question is, if really all problems
> have to be addressed. However, they might indicate real problems - depending
> on the tests nature. This is up to you to decide.

Thanks for all the useful information.

I have not yet investigated why the IBM JVMs have problems with the
XMLSchema and XPath tests, but given that none of the other JVMs show
the same errors I'm not too worried.

The other errors are all problems in the test code, rather than errors
in JMeter itself.

I've fixed the Class Cast problem, and the tests now run OK on Windows and Unix.

Given that it is a release candidate rather than a full release, I'm
inclined to leave fixing these errors until the full release.

> - Jörg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.