Re: imageFilters: Strange behaviour

--===============4036377327120593141==
Content-Type: multipart/alternative; boundary="0000000000005de9e7061be1629b"

--0000000000005de9e7061be1629b
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Thank you Eibe, for the prompt reply.

If both training and test images are together available in the same
directory (ARFF of the test set is available in a different directory and
only has neme(s) of the test image(s), the classifier sees both during the
training process. This is not considered cheating while learning the data?

To me, in order to have a healthy learning process, the training set images
and their ARFF file should be available in a different directory from the
test set images and their ARFF file, which is different from the working
mechanism of ImageFilter and your suggestion.

Any clarification would be greatly appreciated.

Regards,
Peter

On Thu, Jun 27, 2024 at 6:06=E2=80=AFAM Eibe Frank <[email protected]=
z> wrote:

> This should work if your training and test images are in the same
> directory, the directory that is given as a parameter in the
> GenericObjectEditor when configuring the filter.
>
>
>
> Of course, the training and test data should not have any overlap.
>
>
>
> Cheers,
>
> Eibe
>
>
>
> *From: *Peter holunaro <[email protected]>
> *Date: *Thursday, 27 June 2024 at 12:26=E2=80=AFPM
> *To: *Weka machine learning workbench list. <[email protected]>
> *Subject: *[Wekalist] imageFilters: Strange behaviour
>
> Some people who received this message don't often get email from
> [email protected]. Learn why this is important
> <https://aka.ms/LearnAboutSenderIdentification>
>
> Dear all,
>
>
>
> When working with "imageFilters" I found the following strange behaviour:
>
>
>
> I divided the data into training and test sets. The test set, as usual, i=
s
> totally different from the training set. Then, when running the
> classification process via FilteredClassifier (classifier: J48; filter:
> MultiFilter--ColorLayoutFilter + Remove the first attribute), I have an
> error like this:
>
>
>
>  ImageFilter Exception: was unable to read file mno001.jpg!
>
>
>
> The idea here is that the desired image is not in the training set, it is
> only in the test set. But, when adding this image to the training set,
> apply the classification process (highlighted up), and use it as well in
> the test set (i.e., the desired image existing in both datasets). The
> classification process works well and the target label is predicted
> normally.
>
>
>
> I'm trying to say that the performed prediction procedure is not conducte=
d
> fairly when using ImageFilter since all the images in the test set need t=
o
> be originally in the training set. Hence, the classifier will see the
> images twice--on the training and the test set--resulting in an optimisti=
c
> result (overfitting).
>
>
>
> Any advice to avoid such an issue is highly appreciated.
>
>
>
> Regards,
>
> Peter
> _______________________________________________
> Wekalist mailing list -- [email protected]
> Send posts to [email protected]
> To unsubscribe send an email to [email protected]
> To subscribe, unsubscribe, etc., visit
> https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
> List etiquette:
> http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
>

--0000000000005de9e7061be1629b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Thank you Eibe, for the prompt reply.<div><br></div><div>I=
f both training and test images are together available in the same director=
y (ARFF of the test set is available in a different directory and only has =
neme(s) of the test image(s), the classifier sees both during the training =
process. This is not considered cheating while learning the data?=C2=A0</di=
v><div><br></div><div>To me, in order to have a healthy learning process, t=
he training set images and their ARFF file should be available in a differe=
nt directory from the test set images and their ARFF file, which is differe=
nt from the working mechanism of ImageFilter and your suggestion.=C2=A0<br>=
</div><div><br></div><div>Any clarification would be greatly appreciated.</=
div><div><br></div><div>Regards,=C2=A0</div><div>Peter</div></div><br><div =
class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Thu, Jun 27,=
 2024 at 6:06=E2=80=AFAM Eibe Frank &lt;<a href=3D"mailto:eibe.frank@waikat=
o.ac.nz">[email protected]</a>&gt; wrote:<br></div><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg=
b(204,204,204);padding-left:1ex"><div class=3D"msg5025650672554470779">





<div lang=3D"EN-NZ" style=3D"overflow-wrap: break-word;">
<div class=3D"m_5025650672554470779WordSection1">
<p class=3D"MsoNormal"><span style=3D"font-size:11pt">This should work if y=
our training and test images are in the same directory, the directory that =
is given as a parameter in the GenericObjectEditor when configuring the fil=
ter.
<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11pt"><u></u>=C2=A0<u></u><=
/span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11pt">Of course, the traini=
ng and test data should not have any overlap.<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11pt"><u></u>=C2=A0<u></u><=
/span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11pt">Cheers,<u></u><u></u>=
</span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11pt">Eibe<u></u><u></u></s=
pan></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11pt"><u></u>=C2=A0<u></u><=
/span></p>
<div id=3D"m_5025650672554470779mail-editor-reference-message-container">
<div>
<div style=3D"border-right:none;border-bottom:none;border-left:none;border-=
top:1pt solid rgb(181,196,223);padding:3pt 0cm 0cm">
<p class=3D"MsoNormal" style=3D"margin-bottom:12pt"><b><span style=3D"color=
:black">From:
</span></b><span style=3D"color:black">Peter holunaro &lt;<a href=3D"mailto=
:[email protected]" target=3D"_blank">[email protected]</a>&gt;=
<br>
<b>Date: </b>Thursday, 27 June 2024 at 12:26</span><span style=3D"font-fami=
ly:Arial,sans-serif;color:black">=E2=80=AF</span><span style=3D"color:black=
">PM<br>
<b>To: </b>Weka machine learning workbench list. &lt;<a href=3D"mailto:weka=
[email protected]" target=3D"_blank">[email protected]</a>&=
gt;<br>
<b>Subject: </b>[Wekalist] imageFilters: Strange behaviour<u></u><u></u></s=
pan></p>
</div>
<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" align=3D"left" widt=
h=3D"100%" style=3D"width:100%;display:table;float:none">
<tbody>
<tr>
<td style=3D"background:rgb(166,166,166);padding:5.25pt 1.5pt"></td>
<td width=3D"100%" style=3D"width:100%;background:rgb(234,234,234);padding:=
5.25pt 3.75pt 5.25pt 11.25pt">
<div>
<p class=3D"MsoNormal">
<span style=3D"font-size:9pt;font-family:&quot;Segoe UI&quot;,sans-serif;co=
lor:rgb(33,33,33)">Some people who received this message don&#39;t often ge=
t email from <a href=3D"mailto:[email protected]" target=3D"_blank">p=
[email protected]</a>.
</span><span style=3D"color:black"><a href=3D"https://aka.ms/LearnAboutSend=
erIdentification" target=3D"_blank"><span style=3D"font-size:9pt;font-famil=
y:&quot;Segoe UI&quot;,sans-serif">Learn why this is important</span></a></=
span><span style=3D"font-size:9pt;font-family:&quot;Segoe UI&quot;,sans-ser=
if;color:rgb(33,33,33)"><u></u><u></u></span></p>
</div>
</td>
<td width=3D"75" style=3D"width:56.25pt;background:rgb(234,234,234);padding=
:5.25pt 3.75pt">
</td>
</tr>
</tbody>
</table>
<div>
<div>
<p class=3D"MsoNormal">Dear all,=C2=A0 </p>
<div>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
</div>
<div>
<p class=3D"MsoNormal">When working with &quot;imageFilters&quot; I found t=
he following strange behaviour:</p>
</div>
<div>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
</div>
<div>
<p class=3D"MsoNormal">I divided the data into training and test sets. The =
test set, as usual, is totally different from the training set. Then, when =
running the classification process via=C2=A0FilteredClassifier (classifier:=
 J48; filter: MultiFilter--ColorLayoutFilter
 + Remove the first attribute), I have an error like this:</p>
</div>
<div>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
</div>
<div>
<p class=3D"MsoNormal">=C2=A0ImageFilter Exception: was unable to read file=
 mno001.jpg!=C2=A0</p>
</div>
<div>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
</div>
<div>
<p class=3D"MsoNormal">The idea here is that the desired image is not in th=
e training set, it is only in the test set. But, when adding this image to =
the training set, apply the classification process (highlighted up), and us=
e it as well in the test set (i.e.,
 the desired image existing in both datasets). The classification process w=
orks well and the target label is predicted normally.</p>
</div>
<div>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
</div>
<div>
<p class=3D"MsoNormal">I&#39;m trying to say that the performed prediction =
procedure is not conducted fairly when using ImageFilter since all the imag=
es in the test set need to be originally in the training set. Hence, the cl=
assifier will see the images twice--on
 the training and the test set--resulting in an optimistic result (overfitt=
ing).=C2=A0=C2=A0</p>
</div>
<div>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
</div>
<div>
<p class=3D"MsoNormal">Any advice to avoid such an issue is highly apprecia=
ted.</p>
</div>
<div>
<p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p>
</div>
<div>
<p class=3D"MsoNormal">Regards,=C2=A0</p>
</div>
<div>
<p class=3D"MsoNormal">Peter</p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>

_______________________________________________<br>
Wekalist mailing list -- <a href=3D"mailto:[email protected]" tar=
get=3D"_blank">[email protected]</a><br>
Send posts to <a href=3D"mailto:[email protected]" target=3D"_bla=
nk">[email protected]</a><br>
To unsubscribe send an email to <a href=3D"mailto:[email protected]=
to.ac.nz" target=3D"_blank">[email protected]</a><br>
To subscribe, unsubscribe, etc., visit <a href=3D"https://list.waikato.ac.n=
z/postorius/lists/wekalist.list.waikato.ac.nz" rel=3D"noreferrer" target=3D=
"_blank">https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.a=
c.nz</a><br>
List etiquette: <a href=3D"http://www.cs.waikato.ac.nz/~ml/weka/mailinglist=
_etiquette.html" rel=3D"noreferrer" target=3D"_blank">http://www.cs.waikato=
.ac.nz/~ml/weka/mailinglist_etiquette.html</a><br>
</div></blockquote></div>

--0000000000005de9e7061be1629b--

--===============4036377327120593141==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Wekalist mailing list -- [email protected]
Send posts to [email protected]
To unsubscribe send an email to [email protected]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

--===============4036377327120593141==--