Re: Problem with WEKA

> > I have two files in .csv format. I open the files in WEKA and delete fields from both files and then save them as WEKA files.
> >
> > When I tried to open them, I got Error saying that the format is not recognized.
> >
> > The two files are attached
> >
> > For diabetic_training.csv  file, i need to remove (5 fields)      “encounter_id”,“patient_nbr” , “diag_1”, “diag_2”,  and “diag_3”,   then save it as diabetic_training.arff.
> >
> > For diabetic_test.csv, remove “encounter_id”, “patient_nbr”, “diag_1”, “diag_2”,  and “diag_3” and save it as diabetic_test.arff
> >
> >
> > Thanks much for your help if you either let me know what the problem is and produce two new .arff files
>
> Working with CSV files is always problematic for various reasons:
> - different values in columns
> - different order of values per column
> - different number of values can mean difference between STRING or
> NOMINAL attribute
>
> I was able to load both of the attached CSV files in the Weka Explorer
> with the default settings of the CSV loader (Weka 3.9.6).
>
> Here's what I would do:
> - combine the two CSV files into a third one (that way we will
> encounter ALL values)
> - load each CSV file in the Weka Explorer and save it as ARFF
> - open all three ARFF files in a text editor
> - copy the header section (between @relation and @data) from the
> combined file and replace the ones in your train/test ARFF files with
> that
> - save your train/test files
>
> The two files should be compatible now.

Since there still seem to be problems with this...

Being an ADAMS (https://adams.cms.waikato.ac.nz/) user rather than
"vanilla" Weka, I've used its Spreadsheet file viewer to load and
combine the CSV files. I then saved them all as ARFF. In the training
and test files I then replaced the header sections with the one from
the combined ARFF file. See attached the compressed versions of the
files.

And yes, I could build a J48 classifier (wrapped in FilteredClassifier
and the RemoveType filter for removing string attributes), using the
"readmitted" column as class attribute.

BTW The diag_1 to _3 fields are mainly numeric, but contain some
random values like "V57" or "E885". Not sure whether these are
leftovers from Excel cell references.

Cheers, Peter
-- 
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
https://www.cs.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/

_______________________________________________
Wekalist mailing list -- [email protected]
Send posts to [email protected]
To unsubscribe send an email to [email protected]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html