Re: How to the corresponding instance of a prediction

Hello Peter,

The dataset already contains IDs, although I remove those during the training phase to avoid any weird overfitting due to their unique nature.

The main issue I'm running into is that I cannot tell which record is which when using Evaluation.Predictions.get(i).

My code basicly looks as follows:

BayesNet net = new BayesNet();
Instances data = readData("class", "resources/test.csv");
net.buildClassifier(data);

Evaluation eval = new Evaluation(data);
eval.crossValidateModel(net, data, 10, new Random(1));

If I now compare eval.predictions().get(i).actual() with data.get(i).value(data.attribute("class")) the two values are unequal. Indicating that the prediction in my eval object at i does not belong to the instance in my data object at i.

This mismatch only seems to happen when using crossvalidation.

Since the prediction objects do not contain any identifiers I am struggling to link them back to the original dataset.

Is there a way to identify the predictions at this point?
 
Regards,

Florian

-----Original Message-----
From: Peter Reutemann <[email protected]> 
Sent: vrijdag 3 mei 2024 00:48
To: Weka machine learning workbench list. <[email protected]>
Subject: [Wekalist] Re: How to the corresponding instance of a prediction

Waarschuwing: We kunnen de herkomst van deze mail niet controleren. Pas extra op met links of het openen van bijlagen. Als u deze email niet verwacht, verwijder deze dan direct!


> I am doing an experiment with cancer data where I want to evaluate the performance of a model on various stages of cancer that are present in the dataset.
>
> Consequently after crossvalidating I want to go over the predictions and group them by cancer stage so I can calculate various statistics for each cancer stage.
>
>
> However, after I run eval.crossValidateModel() I no longer know which prediction belongs to which Instance in the dataset.
>
> Since Eval.predictions() does not provide any usefull identifiers to match it back to the instances, I originally assumed the prediction would maintain the same order as the instances in the dataset, but this is not correct.
>
> How do I determine which prediction belongs to which records?

You'd want to use an extra attribute with a unique ID.

Have a look at the following wiki entry:
https://waikato.github.io/weka-wiki/instance_id/

Cheers, Peter
--
My Open Source Blog - http://open.fracpete.org _______________________________________________
Wekalist mailing list -- [email protected] Send posts to [email protected] To unsubscribe send an email to [email protected] To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
_______________________________________________
Wekalist mailing list -- [email protected]
Send posts to [email protected]
To unsubscribe send an email to [email protected]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html