Re: Setting MultiSearch and Understanding GridSearch Outputs

> I've been struggling to understand the MultiSearch and GridSearch tools and I desperately need some help.  I'm doing everything through the GUI
> interface.
>
> More specifically, I don't know how I could wrap my classifier into the Search since I would like to use a filtered classifier (C4.5 with AttributeSelectionClassifier + ClassBalancer).  I know I can choose the classifier "Filtered Classifier" when setting GridSearch but I don't know how I could refer the J48 parameters to the X and Y properties then. Another issue is that ClassBalancer must be the last filter to be applied because AttributeSelectionClassifier does not handle doubled instances as far as I understood.

Unfortunately, I can't comment on the "AttributeSelectionClassifier
not liking duplicate instances".

Assuming that you want to perform attribute selection before using the
ClassBalancer you could use the following setup:
FilteredClassifier
- filter: MultiFilter (using AttributeSelection, ClassBalancer)
- classifier: J48

> I don't know either if it is possible to wrap everything into the MultiSearch to include filter parameters into the search and how I would do that.  I didn't understand the MultiSearch setting at all, to be honest.

Have you looked at the examples?
https://github.com/fracpete/multisearch-weka-package
https://github.com/fracpete/multisearch-weka-package-examples

What parameters of what class do you want to optimize?

> Moreover, I don't quite understand the outputs of GridSearch in the Explorer. Where can I see which were the best parameters chosen?

Using the default settings of GridSearch on a toy dataset, you get
something like:

===
weka.classifiers.meta.GridSearch:
Classifier: weka.classifiers.functions.SMOreg -C 100.0 -N 0 -I
"weka.classifiers.functions.supportVector.RegSMOImproved -T 0.001 -V
-P 1.0E-12 -L 0.001 -W 1" -K
"weka.classifiers.functions.supportVector.RBFKernel -C 250007 -G 0.1"

X property: C
Y property: kernel.gamma

Evaluation: Correlation coefficient
Coordinates: [2.0, -1.0]
Values: 100.0 (X coordinate), 0.1 (Y coordinate)
===

Explanation:
- Classifier: the classifier that you supplied to GridSearch for optimization
- X property: the first property path of the classifier property to
optimize ("C" of SMO)
- Y property: the second property path of the classifier property to
optimize (the "gamma" property of the "kernel" property of SMO)
- Evaluation: how to ascertain the performance of a model
- Coordinates: the coordinates in the grid that you defined
- Values: the actual values for you to use for X ("C") and Y ("kernel.gamma")

A "property" is the name of a parameter that you see in Weka's
GenericObjectEditor.

MultiSearch works similarly, but it is not fixed to always optimize
two parameters. Instead, you can optimize an arbitrary number of
parameters. Also, you can specify mathematical expressions (like with
GridSearch), fixed lists with predefined values or parameter groups (=
each defining a sub-search).

Cheers, Peter
-- 
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 577-5304
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [email protected]
Send posts to [email protected]
To unsubscribe send an email to [email protected]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html