imageFilters: Strange behaviour

--===============6448207350995656386==
Content-Type: multipart/alternative; boundary="000000000000eb3ab4061bd42ab0"

--000000000000eb3ab4061bd42ab0
Content-Type: text/plain; charset="UTF-8"

Dear all,

When working with "imageFilters" I found the following strange behaviour:

I divided the data into training and test sets. The test set, as usual, is
totally different from the training set. Then, when running the
classification process via FilteredClassifier (classifier: J48; filter:
MultiFilter--ColorLayoutFilter + Remove the first attribute), I have an
error like this:

 ImageFilter Exception: was unable to read file mno001.jpg!

The idea here is that the desired image is not in the training set, it is
only in the test set. But, when adding this image to the training set,
apply the classification process (highlighted up), and use it as well in
the test set (i.e., the desired image existing in both datasets). The
classification process works well and the target label is predicted
normally.

I'm trying to say that the performed prediction procedure is not conducted
fairly when using ImageFilter since all the images in the test set need to
be originally in the training set. Hence, the classifier will see the
images twice--on the training and the test set--resulting in an optimistic
result (overfitting).

Any advice to avoid such an issue is highly appreciated.

Regards,
Peter

--000000000000eb3ab4061bd42ab0
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Dear all,=C2=A0<div><br></div><div>When working with &quot=
;imageFilters&quot; I found the following strange behaviour:</div><div><br>=
</div><div>I divided the data into training and test sets. The test set, as=
 usual, is totally different from the training set. Then, when running the =
classification process via=C2=A0FilteredClassifier (classifier: J48; filter=
: MultiFilter--ColorLayoutFilter + Remove the first attribute), I have an e=
rror like this:</div><div><br></div><div>=C2=A0ImageFilter Exception: was u=
nable to read file mno001.jpg!=C2=A0</div><div><br></div><div>The idea here=
 is that the desired image is not in the training set, it is only in the te=
st set. But, when adding this image to the training set, apply the classifi=
cation process (highlighted up), and use it as well in the test set (i.e., =
the desired image existing in both datasets). The classification process wo=
rks well and the target label is predicted normally.</div><div><br></div><d=
iv>I&#39;m trying to say that the performed prediction procedure is not con=
ducted fairly when using ImageFilter since all the images in the test set n=
eed to be originally in the training set. Hence, the classifier will see th=
e images twice--on the training and the test set--resulting in an optimisti=
c result (overfitting).=C2=A0=C2=A0</div><div><br></div><div>Any advice to =
avoid such an issue is highly appreciated.<br></div><div><br></div><div>Reg=
ards,=C2=A0</div><div>Peter</div></div>

--000000000000eb3ab4061bd42ab0--

--===============6448207350995656386==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Wekalist mailing list -- [email protected]
Send posts to [email protected]
To unsubscribe send an email to [email protected]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

--===============6448207350995656386==--