Re: Important bug in percentage split for classification models

The section in the Explorer output with "full training set" refers to the
full dataset loaded into the Explorer, not the training set of your split.
You can see that it will output the same model with 10-fold
cross-validation (cross-validation doesn't produce a model, the models from
each train/test fold pair get discarded after collecting the statistics).

You can turn off this output by unchecking "Output model" in the "More
options" dialog.

Not a bug, merely a feature that allows you to immediately safe the model
trained on the full dataset to disk (and use in production, for instance).
Unfortunately a feature that tends to confuse people new to Weka.

Cheers, Peter

On Sat, Dec 31, 2022 at 4:41 PM Jastrade <[email protected]> wrote:

> Dear sirs,
>
>
>
> I am noticing something wrong in Weka version 3.8.6, that I think it is
> worth to describe and ask for explanation or solution if that is a bug. The
> example shown is with the classifier RepTree but it happens the same with
> the others.
>
>
>
> I am using percentage split 66%, and here you see the results
>
>
> [image: 2022-12-30_11h02_45.png]
>
>
>
>
>    - It stays that the classification has been applied to the full
>    training test, and it shows the results (for example the first) as:
>    (1063/88)[528/53].
>    - Total of instances is 1063+528 = 1588. Then according to that, it
>    splits the data in 66%/33% correctly.
>
>
>
> But look at what happens if I do the same but changing the pertentage
> split to 80%
>
>
> [image: 2022-12-30_11h07_14.png]
>
>
>
> Basically the results are exactly the same, and it also shows in the
> different rules, that it doesn´t split 80/20, but still the same 66/33 as
> before.
>
>
>
> So I have the following questions:
>
>
>
>    1. Does this mean that it is always splitting the 66/33 by defauld,
>    whatever you specify?
>    2. Is it that maybe these rules are the result of using all data as
>    training set in reality?
>
>
>
> I have seen if I press to More options, that I can click on “Output models
> for training splits”, and then it shows also other results, only using the
> % of data indicated in Pertentage split, but again, using those data, it
> shows results again splited 66/33%, as you see here:
>
>
> [image: 2022-12-30_11h09_57.png]ç
>
> [image: 2022-12-30_11h11_11.png]
>
>
>
>
> So it is really confusing for me what is this doing in reality…
>
>
>
> Can you please clarify?
>
>
>
> Thank you very much in advance,
>
>
>
> Best regards,
>
>
> Jastrade
>
>
> _______________________________________________
> Wekalist mailing list -- [email protected]
> Send posts to [email protected]
> To unsubscribe send an email to [email protected]
> To subscribe, unsubscribe, etc., visit
> https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
> List etiquette:
> http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
>


-- 
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
Mobile +64 22 190 2375
https://www.cs.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/

_______________________________________________
Wekalist mailing list -- [email protected]
Send posts to [email protected]
To unsubscribe send an email to [email protected]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html