Re: Weka Experimenter - Classification for multiple Datasets

Thanks a lot for your comments, they make a lot of sense. I have indeed 
an attribute that perfectly predicts the class attribute, which is a 
blunder on my part. I have many files and each contains the filename as 
an attribute. So I concatenated two files together and that led to all 
rows of "filename1" being true and all of "filename2" being false.

So I removed the filename column and the result was the same: another 
attribute (now the timestamp) perfectly predicted the class attribute. 
It's timeseries data and all the timestamps in the first file are 
smaller than those in the second file.

So, after I removed all attributes that were basically constant within a 
file, I got a more sensible result (77% accuracy for J48 and 56% for OneR).

I will now look into multi-instance learning, although I am not 100% 
sure what its benefit would be. It seems that all my data files would be 
combined into one large file. So does that mean this is the only way I 
could train a model using all my data? Meaning that I cannot train a 
model off more than one file?

Kind regards,

Ulrich

Am 11.11.23 um 04:05 schrieb Eibe Frank:
> Yes, in your first experiment, WEKA will treat that two datasets as 
> completely separate entities.
> 
> If I understand you correctly, the result from your second experiment 
> implies that there is an attribute in your data (assuming you are using 
> OneR) that can perfectly separate the "true" and "false" cases in your 
> (one) dataset. To verify this and check which attribute is responsible, 
> try repeating the experiment in the Classify panel of the WEKA Explorer.
> 
> It sounds to me like you might want to look at multi-instance learning 
> (also called "multiple instance" learning), which is designed to 
> classify bags (i.e., "datasets"). WEKA has some learning algorithms that 
> support it and relational (aka "relation-valued") attributes are 
> available whose values can be entire datasets:
> 
> https://waikato.github.io/weka-wiki/formats_and_processing/arff_stable/ 
> <https://waikato.github.io/weka-wiki/formats_and_processing/arff_stable/>
> 
> https://waikato.github.io/weka-wiki/multi_instance_classification/ 
> <https://waikato.github.io/weka-wiki/multi_instance_classification/>
> 
> Cheers,
> Eibe
> Multi instance classification - Weka Wiki 
> <https://waikato.github.io/weka-wiki/multi_instance_classification/>
> Documention on Weka that can also be found in the manual and the example 
> archive that comes with each Weka download (zip or installer).
> waikato.github.io
> 
> 
> Arff stable - Weka Wiki 
> <https://waikato.github.io/weka-wiki/formats_and_processing/arff_stable/>
> Documention on Weka that can also be found in the manual and the example 
> archive that comes with each Weka download (zip or installer).
> waikato.github.io
> 
> 
> 
> ------------------------------------------------------------------------
> *From:* Ulrich Mayring <[email protected]>
> *Sent:* Saturday, November 11, 2023 2:16 AM
> *To:* [email protected] <[email protected]>
> *Subject:* [Wekalist] Weka Experimenter - Classification for multiple 
> Datasets
> [You don't often get email from [email protected]. Learn why this 
> is important at https://aka.ms/LearnAboutSenderIdentification 
> <https://aka.ms/LearnAboutSenderIdentification> ]
> 
> Hi all,
> 
> this is a very high-level question about how the classification in Weka
> works with multiple input files:
> 
> Are they treated as seperate (non-related) datasets or are they combined
> into one dataset? I couldn't tell from experimenting with it.
> 
> Background: I have ARFF files, which contain data like this:
> 
> @attribute sensordata1 numeric
> @attribute sensordata2 numeric
> ...
> @attribute class {true,false}
> 
> So I have many of those files with all different sensordata values, but
> within each file the class attribute is always either true or false for
> all rows. So now I want to train a model in order to classify future
> files (which do not contain the class attribute) as true or false.
> 
> So here's what I did:
> 
> I loaded two files into the experimenter (one with the class attribute
> as true and one with false) and the result was a 100% success rate in
> classification with either OneR or J48 (using ten-fold cross
> validation). But both datasets were listed seperately in the results
> window, which makes me believe that were evaluated seperately and the
> 100% success rate is because the class value is the same for all rows.
> 
> But then I combined both files into one file and loaded that one file
> into the experimenter and the result was the same. I would have expected
> a success rate below 100%, because now half of the rows were now "true"
> and the other half "false".
> 
> So my question is: did I actually run the same experiment twice or does
> Weka behave differently with multiple datasets compared to one dataset
> (even if the actual data is the same in both cases)?
> 
> Many thanks in advance for any pointers,
> 
> Ulrich
> 
> _______________________________________________
> Wekalist mailing list -- [email protected]
> Send posts to [email protected]
> To unsubscribe send an email to [email protected]
> To subscribe, unsubscribe, etc., visit 
> https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz 
> <https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz>
> List etiquette: 
> http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html 
> <http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html>
> 
> _______________________________________________
> Wekalist mailing list -- [email protected]
> Send posts to [email protected]
> To unsubscribe send an email to [email protected]
> To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html


_______________________________________________
Wekalist mailing list -- [email protected]
Send posts to [email protected]
To unsubscribe send an email to [email protected]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html