Re: Weka meta bagging and boosting classifiers
Michael Hall <[email protected]>
| Newsgroups | gmane.comp.ai.weka |
|---|---|
| Message-ID | <[email protected]> |
> On Nov 25, 2021, at 8:33 PM, Felix Mohr <[email protected]> wrote: > > Hi Michael, > > let's see. Sure, in fact we configure meta-classifiers with base learners in ML-Plan (and then configure parameters of both base learner and meta-learner) if I we are talking about the same thing. > Felix, That is what I’m talking about. Although I rarely changed hyper-parameters for the meta-learner. > The question was, from what I understood, whether this also works with regressors. No since regressors also have the classifier interface in WEKA I have no doubt that this syntactically works, and for bagging I don't see any reason why it should not work. For AdaBoost, I guess that one needs to adapt the weighting mechanism in the boosting algorithm, which in uthe variant of boosting i am aware of, is based on the error rate, which you cannot compute in the case of regression. > No, I wasn’t really thinking regression. Sorry for any confusion. > Bagging with RandomForests sounds funny. I currently cannot imagine a theoretical justification for this phenomenon except maybe that you have a kind of inner and outer sampling technique, which might increase robustness to over-fitting even more. Yes, I thought it was funny too when I found out RF bagged. But my own results with Weka seemed to show this almost always resulted in improvement. Not the extensive bench marking as was talked about for the conference but the link I posted somewhat bears this out for a sample of Weka provided datasets Bagging -> RF was best. But even the maybe also funny combination of AdaBoostM1 -> Bagging -> RF was the best for a couple datasets. > > So yes, ML-Plan supports the meta-classifiers of WEKA and also stacking. Stacking I don’t generally use. Putting together a good ensemble from scratch is still not really a skill I have. > > Thank for following up on this, > Felix > Thanks for the reply. _______________________________________________ Wekalist mailing list -- [email protected] Send posts to [email protected] To unsubscribe send an email to [email protected] To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html