Questions about Weighting Schemes project

--0000000000001005050585b0c661
Content-Type: text/plain; charset="UTF-8"

Hi,
I was going through the Xapian code base of different weighting schemes. In
the lmweight code, I found out that we are returning non-negative numbers
from get_maxpart, get_sumpart methods. Is this to avoid negative weight?
Also in the Language Model with Jelinek Mercer Smoothing (LM-JM)
implementation, I don't see any idf effect involved in that equation. The
LM-JM equation looks something like this:
 *(LAMBDA)* MLE(t,d) + (1-LAMBDA) * MLE(t,c)*
However, if we bind it with idf, it will look like :

*1 + ((LAMBDA) / (1-LAMBDA) * (MLE(t,d) / MLE(t,c))) *
which is widely used everywhere. I am planning to patch an improved
representation of LM-JM with the idf effect shortly. Kindly let me know for
any concerns.

Thanks and Regards,
-Sourav

--0000000000001005050585b0c661
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div>Hi, <br></div><div>=
I was going through the Xapian code base of different weighting schemes. In=
 the lmweight code, I found out that we are returning non-negative numbers =
from get_maxpart, get_sumpart methods. Is this to avoid negative weight? Al=
so in the Language Model with Jelinek Mercer Smoothing (LM-JM) implementati=
on, I don&#39;t see any idf effect involved in that equation. The LM-JM equ=
ation looks something like this:</div><div><div style=3D"font-family:arial,=
helvetica,sans-serif" class=3D"gmail_default">=C2=A0<b><span style=3D"font-=
family:monospace,monospace">(LAMBDA)* MLE(t,d) + (1-LAMBDA) * MLE(t,c)</spa=
n></b> </div></div><div>However, if we bind it with idf, it will look like =
:</div><div>=C2=A0 <b><span style=3D"font-family:monospace,monospace">1 + (=
(LAMBDA) /<b><span style=3D"font-family:monospace,monospace"> (1-LAMBDA) </=
span></b>* (MLE(t,d) /<b><span style=3D"font-family:monospace,monospace"> M=
LE(t,c)))</span></b> <br></span></b></div><div>which is widely used everywh=
ere. I am planning to patch an improved representation of LM-JM with the id=
f effect shortly. Kindly let me know for any concerns.</div><div><br><div d=
ir=3D"ltr" class=3D"gmail_signature"><div dir=3D"ltr"><div><div dir=3D"ltr"=
><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div d=
ir=3D"ltr"><div dir=3D"ltr"><div style=3D"background-image:none;margin:0cm =
0cm 0pt;background-repeat:repeat"><span style=3D"font-family:&quot;Times Ne=
w Roman&quot;,serif;font-size:12pt;letter-spacing:0.2px">Thanks and Regards=
,</span><br></div><div style=3D"background-image:none;margin:0cm 0cm 0pt;ba=
ckground-repeat:repeat"><span style=3D"font-size:12pt;font-family:&quot;Tim=
es New Roman&quot;,serif">-Sourav</span></div></div></div></div></div></div=
></div></div></div></div></div></div></div></div></div>

--0000000000001005050585b0c661--