Re: percentage split versus re-evaluated complete data set

Ok, that explains a lot.
So, in order to produce a model for production from a single data set, one has to split the dataset into the training and the test set beforehand (e.g. with a filter) instead of using the percentage split in the explorer.
Am I right that this behaviour (final saved model is based on learning on the complete dataset) is also used during usage of the CLI for learning?

That behaviour also explains, why I was never able to reproduce the good results of AutoWeka in the Explorer. The AutoWeka stats are apparently based on learning and predicting on the full dataset but the stats printed in the Explorer are based on learning on the selected subset (e.g percentage split or cross validation) and predicting on the rest.

That info was very valuable!
Thanks!
Jens

-----Ursprüngliche Nachricht-----
Von: Peter Reutemann <[email protected]> 
Gesendet: Dienstag, 29. Juni 2021 22:52
An: Weka machine learning workbench list. <[email protected]>
Betreff: [Wekalist] Re: percentage split versus re-evaluated complete data set


Regardless of what evaluation method you use, Weka will build (and
keep) a final model on the full dataset (unless you've turned this off under "More options") which you can then save.
If you've performed only a "percentage split", then the "Re-evaluate model on current test set" option will be grayed out.
If you then select the original dataset as the test set, then you will basically evaluate a model that was trained on the full dataset on the training set.
This is essentially the same as choosing "Use training set" as the evaluation method.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 577-5304
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [email protected] Send posts to [email protected] To unsubscribe send an email to [email protected] To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
_______________________________________________
Wekalist mailing list -- [email protected]
Send posts to [email protected]
To unsubscribe send an email to [email protected]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html