Re: Problem of large data sets

Am 17.04.24 um 05:00 schrieb Shu-Ju Tu:
> Hi dear Weka development team staff:
> 
> I have a problem of getting low predictive accuracy when running a large 
> data set.
> 
> Here is the story and thank you for the patient in advance:
> We started a small data set (n=100) last year.
> It is a 2-class supervised data set and the class is evenly distributed 
> 50-50.
> The correctly predictive accuracy on training after feature selection 
> and test data sets is about 85%.
> We have tried RandomForest and AdaBoostM1.
> Then we increased the data set to n=200 (later 300) and were getting 
> about similar predictive results.
> Then recently we increased to n=800 and were getting very low accuracy 
> of 60%.
> 
> Are there something we can do and try to improve on the results?

Maybe your new data is significantly different from the old data. If so, 
you could try to retrain your model on the new data.

I had a situation like that where I was looking at manufacturing data. 
Then they reconfigured / optimised the machine and the data changed 
enough to make my model useless.


_______________________________________________
Wekalist mailing list -- [email protected]
Send posts to [email protected]
To unsubscribe send an email to [email protected]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html