Re: Merit of best subset found & Methods

The correlation coefficient is one of the other evaluation metrics that you
can select in the GenericObjectEditor when you configure
WrapperSubsetSelection (assuming you are using a reasonably recent version
of WEKA). By definition it will be in [-1,1].

Every dataset is different. However, considering the dimensions of your
data, it will be quite difficult to avoid detecting spurious correlations,
particularly if you apply a complex search technique. Have you tried
running your configurations in the AttributeSelectedClassifier in the
ClassifyPanel, within a 10-fold cross-validation (which will implement a
"proper" evaluation using nested cross-validation)? Is the root relative
squared error well below 100 so that your model actually outperforms simply
predicting the mean target value from the training data?

Cheers,
Eibe

On Sat, 18 Mar 2023 at 10:37, Francisco Cabrera-Torres <
[email protected]> wrote:

> Thanks for your answer
>
> Which option allows me to constrain the correlation coefficient between
> [-1, 1]?
>
> On the other hand, I will apply PLS later, however, now I am
> characterizing heavy metals and I need to select the best wavelengths based
> on the chemical analyses.
>
> I was telling you that I have 35 soil samples and 16 vegetation samples,
> each with 2051 columns of wavelengths. Based on your extensive experience,
> what method and evaluator could you use for subsets of soils and vegetation?
>
> I have currently used several, however, based on the results obtained, the
> ones that look best visually are:
> 1.    Evaluator: weka.attributeSelection.WrapperSubsetEval -B
> weka.classifiers.rules.M5Rules -F 5 -T 0.01 -R 1 -E DEFAULT -- -
>        M 4.0 -num-decimal-places 4
>        Search: weka.attributeSelection.BestFirst -D 1 -N 5
> 2.    Evaluator: weka.attributeSelection.ClassifierSubsetEval -B
> weka.classifiers.trees.RandomForest -T -H "Click to set hold out
>        or test instances" -E DEFAULT -- -P 100 -I 25 -num-slots 1 -K 0 -M
> 1.0 -V 0.001 -S 1
>        Search: weka.attributeSelection.BestFirst -D 1 -N 5
>
> Regards
> _______________________________________________
> Wekalist mailing list -- [email protected]
> Send posts to [email protected]
> To unsubscribe send an email to [email protected]
> To subscribe, unsubscribe, etc., visit
> https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
> List etiquette:
> http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
>

_______________________________________________
Wekalist mailing list -- [email protected]
Send posts to [email protected]
To unsubscribe send an email to [email protected]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html