Re: Test instrumentation using a self.method

FYI,

I have solved this in a different way, as the instrumentation seems to not
work as expect (or I am simply using it incorrectly).

I have moved testing related methods outside of TestRunner and wrapped them
based on this example:

http://open.bekk.no/performance-testing-using-grinder

It would be great to have a large example base, is there any way people can
add to the sourceforge script gallery?

I ended up with instrumentation something like the following:

self.wrapped_createReportTest = tests["createReport"].wrap(createReportTest)
self.wrapped_createTableTest = tests["createTable"].wrap(createTableTest)
self.wrapped_createRecordTest = tests["createRecord"].wrap(createRecordTest)
self.wrapped_readReportTest = tests["readReport"].wrap(readReportTest)
self.wrapped_readTableTest = tests["readTable"].wrap(readTableTest)
self.wrapped_readRecordTest = tests["readRecord"].wrap(readRecordTest)
self.wrapped_updateReportTest = tests["updateReport"].wrap(updateReportTest)
self.wrapped_updateTableTest = tests["updateTable"].wrap(updateTableTest)
self.wrapped_updateRecordTest = tests["updateRecord"].wrap(updateRecordTest)
self.wrapped_deleteReportTest = tests["deleteReport"].wrap(deleteReportTest)
self.wrapped_deleteTableTest = tests["deleteTable"].wrap(deleteTableTest)
self.wrapped_deleteRecordTest = tests["deleteRecord"].wrap(deleteRecordTest)
self.wrapped_addBulkRecordTest =
tests["bulkAddRecord"].wrap(addBulkRecordTest)


On Wed, Sep 16, 2015 at 3:22 PM, Darren Ball <[email protected]> wrote:

> Hi Philip,
>
> I just tried this, with no luck.
> Here is what I have attempted, where cscAppPerf is the java object being
> used to run tests, and CscAppPerfFilter.matches returning my valid test
> methods.
>
> class CscAppPerfFilter(Test.InstrumentationFilter):
>     def matches(self, method):
>         return method.name in ["perfGetTableRecord",
> "perfRetrieveReport","perfRetrieveTable","perfGetTableRecord","perfUpdateTableRecord","perfDeleteTableRecord","perfAddBulkTableRecord"]
>
>
> # Instrumentation for tests
> tests["createReport"].record(cscAppPerf, CscAppPerfFilter())
> tests["createTable"].record(cscAppPerf, CscAppPerfFilter())
> tests["createRecord"].record(cscAppPerf, CscAppPerfFilter())
> tests["readReport"].record(cscAppPerf, CscAppPerfFilter())
> tests["readTable"].record(cscAppPerf, CscAppPerfFilter())
> tests["readRecord"].record(cscAppPerf, CscAppPerfFilter())
> tests["updateReport"].record(cscAppPerf, CscAppPerfFilter())
> tests["updateTable"].record(cscAppPerf, CscAppPerfFilter())
> tests["updateRecord"].record(cscAppPerf, CscAppPerfFilter())
> tests["deleteReport"].record(cscAppPerf, CscAppPerfFilter())
> tests["deleteTable"].record(cscAppPerf, CscAppPerfFilter())
> tests["deleteRecord"].record(cscAppPerf, CscAppPerfFilter())
> tests["bulkAddRecord"].record(cscAppPerf, CscAppPerfFilter())
>
> My stats summary is now somewhat unusual:
>
>
> 2015-09-16 19:06:34,305 INFO  i-20ff63e6-0 : Final statistics for this
> process:
> 2015-09-16 19:06:34,313 INFO  i-20ff63e6-0 :
>
>              Tests        Errors       Mean Test    Test Time    TPS
>
>                                        Time (ms)    Standard
>
>                                                     Deviation
>
>                                                     (ms)
>
>
>
> (Test 1      126          0            744.15       1612.83      1.32)
>     "createReport"
> (Test 2      126          0            744.14       1612.80      1.32)
>     "createTable"
> (Test 3      126          0            744.11       1612.81      1.32)
>     "createRecord"
> (Test 4      126          0            744.10       1612.81      1.32)
>     "readReport"
> (Test 5      126          0            744.08       1612.82      1.32)
>     "readTable"
> (Test 6      126          0            744.06       1612.82      1.32)
>     "readRecord"
> (Test 7      126          0            744.02       1612.76      1.32)
>     "updateReport"
> (Test 8      126          0            744.01       1612.76      1.32)
>     "updateTable"
> (Test 9      126          0            744.00       1612.76      1.32)
>     "updateRecord"
> (Test 10     126          0            743.99       1612.76      1.32)
>     "deleteReport"
> (Test 11     126          0            743.99       1612.76      1.32)
>     "deleteTable"
> (Test 12     126          0            743.96       1612.76      1.32)
>     "deleteRecord"
> Test 13      126          0            743.95       1612.76      1.32
>      "bulkAddRecord"
>
> Totals       126          0            743.95       1612.76      1.32
>
>              (1512)       (0)
>
>
> Note:
> 1.  I am using weighted algorithm for running my tests, and 126 total
> count for each test is wrong.  All statistics are wrong now.
> 2.  Test 13 does not have parentheses around it?  Why is that?
>
> Here is what the results look like when I use the instrumentation of
> javaobject.method like the following:
> tests["createReport"].record(cscAppPerf.perfGetTableRecord)
> tests["createTable"].record(cscAppPerf.perfGetTableRecord)
> tests["createRecord"].record(cscAppPerf.perfGetTableRecord)
> tests["readReport"].record(cscAppPerf.perfRetrieveReport)
> tests["readTable"].record(cscAppPerf.perfRetrieveTable)
> tests["readRecord"].record(cscAppPerf.perfGetTableRecord)
> tests["updateReport"].record(cscAppPerf.perfGetTableRecord)
> tests["updateTable"].record(cscAppPerf.perfGetTableRecord)
> tests["updateRecord"].record(cscAppPerf.perfUpdateTableRecord)
> tests["deleteReport"].record(cscAppPerf.perfGetTableRecord)
> tests["deleteTable"].record(cscAppPerf.perfGetTableRecord)
> tests["deleteRecord"].record(cscAppPerf.perfDeleteTableRecord)
> tests["bulkAddRecord"].record(cscAppPerf.perfAddBulkTableRecord)
>
> Statistics:
> 2015-09-16 19:20:09,472 INFO  i-20ff63e6-1 : Final statistics for this
> process:
> 2015-09-16 19:20:09,480 INFO  i-20ff63e6-1 :
>
>              Tests        Errors       Mean Test    Test Time    TPS
>
>                                        Time (ms)    Standard
>
>                                                     Deviation
>
>                                                     (ms)
>
>
> (Test 1      3            0            255.33       16.74        0.15)
>     "createReport"
> (Test 2      3            0            255.33       16.74        0.15)
>     "createTable"
> (Test 3      3            0            255.33       16.74        0.15)
>     "createRecord"
> Test 4       4            0            47.75        12.75        0.20
>      "readReport"
> Test 5       4            0            4734.25      1530.59      0.20
>      "readTable"
> (Test 6      3            0            255.33       16.74        0.15)
>     "readRecord"
> (Test 7      3            0            255.33       16.74        0.15)
>     "updateReport"
> (Test 8      3            0            255.33       16.74        0.15)
>     "updateTable"
> Test 9       2            0            53.50        3.50         0.10
>      "updateRecord"
> (Test 10     3            0            255.33       16.74        0.15)
>     "deleteReport"
> Test 11      3            0            255.33       16.74        0.15
>      "deleteTable"
> Test 12      2            0            39.50        0.50         0.10
>      "deleteRecord"
> Test 13      1            0            74.00        0.00         0.05
>      "bulkAddRecord"
>
> Totals       16           0            1259.62      2148.51      0.78
>
>              (21)         (0)
>
> This is more realistic and is actually happening - but seemingly not
> possible?
>
> On Wed, Sep 16, 2015 at 1:56 PM, Philip Aston <[email protected]> wrote:
>
>> On 16/09/15 14:29, Darren Ball wrote:
>> > Is it possible in the testrunner init to do something like:
>> >
>> > tests["abc"].record(self.abc)?
>> >
>> > Why is it that a specific java object method can not be instrumented?
>> > Seem like only the whole object, meaning every call within?
>> >
>>
>> You can't instrument a specific method by passing it to record as above
>> is that under the covers because Java does not provide a references to
>> bound methods, and so the Jython expression "self.abc" is actually a
>> Jython wrapper. The wrappers change with each version of Jython so we
>> (no longer) rely on Jython internals to figure out what exactly they
>> refer to.
>>
>> Instead you can use an overloaded version of record that accepts a
>> filter that specifies just the methods you care about. See
>>
>> http://grinder.sourceforge.net/g3/instrumentation.html#Selective+instrumentation
>>
>>
>> - Phil
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
>> Get real-time metrics from all of your servers, apps and tools
>> in one place.
>> SourceForge users - Click here to start your Free Trial of Datadog now!
>> http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140
>> _______________________________________________
>> grinder-use mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/grinder-use
>>
>
>

------------------------------------------------------------------------------
Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140

_______________________________________________
grinder-use mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/grinder-use