Re: Merit of best subset found & Methods

By default, for regression tasks, the "evaluationMeasure" in the Wrapper (specified by the -E option in the command-line interface) is the RMSE. This will be output as the "merit" of the best subset found (even though it's actually an error metric that you will want to minimise, and this is actually what happens internally when the code searches for the best subset). The RMSE can be greater than 1. For example, when running a k-fold cross-validation using linear regression, depending on the data, the RMSE can be greater than 1.

There is an option to switch to the correlation coefficient instead, which will be bounded in [-1,1]. (Also, the word "merit" makes more sense in that case.)

It sounds like you are looking at NIR spectra. As you probably know, the standard method in this case is PLS regression, and there is an optional package for WEKA that you can install to run it in WEKA. Rather than performing attribute selection, it is probably best to instead tune the number of components to use in PLS (e.g., using MultiSearch). (This is assuming you want to maximise predictive performance.)

Cheers,
Eibe

> On 17/03/2023, at 4:29 PM, [email protected] wrote:
> 
> Hi
> 
> I am currently using the selection of WEKA attributes, however, I have a question regarding the "Merit of best subset found" parameter, since in several examples they fluctuate between 0 and 1, instead with my data and using the Wrapper method this value is triggered above 1. Is this OK?
> 
> My data contains 35 soil samples and for each sample I have the reflectivities values for each nm (400 - 2450 nm), in total 2051 columns. The attribute that I selected as class is of numeric type, which corresponds to the different results of the chemical analyzes (for example As, Cd, Cr, etc). What method and evaluator do you recommend I use?
> 
> I have planned to do the same for my vegetation samples, but these are much smaller, there are a total of 16 vegetation samples, I don't know if I would have any problem applying the Wrapper method or which one could apply?
> 
> Regards
> _______________________________________________
> Wekalist mailing list -- [email protected]
> Send posts to [email protected]
> To unsubscribe send an email to [email protected]
> To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [email protected]
Send posts to [email protected]
To unsubscribe send an email to [email protected]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html