Re: PostgreSQL full text search/OpenFTS ranking algorithm?

Robert,

there two ranking functions in tsearch2. One is ts_rank_cd(), which uses
"cover density" approach from paper "Relevance ranking for one to three term
queries", mention by Neophytos. We added various normalizations.
Better, just see source code. Another one is old ts_rank() function, which
uses statistical approach. Again, see source code. 
Some notices:
1. ts_rank_cd generally is better for AND queries
2. ts_rank has some support for OR queries
3. both functions use only local (current document) informations, so
they are very good for combining search results from several machines.

I have several presentations 
http://www.sai.msu.su/~megera/postgres/talks/,

http://www.sai.msu.su/~megera/postgres/talks/fts-pgday-2007.pdf

On Wed, 29 Jul 2009, Neophytos Demetriou wrote:

> Dear Robert,
>
> I just got back from holidays. Attached you may find the paper behind the 
> rank_cd ranking function in tsearch2 (this is the one I had mentioned in my 
> previous reply). You may also want to check out the corresponding code in 
> postgresql/src/backend/adt/tsrank.c (calc_rank_cd and Cover).
>
> IIRC, the ranking functions in OpenFTS were different than the ones in 
> tsearch2. I'm CC-ing Oleg Bartunov which might have more information to 
> share.
>
> If you need further info, please do not hesitate and contact me.
>
> Best wishes,
> Neophytos
>
>
> Robert Koblischke wrote:
>> Dear Neophytos,
>> 
>> This is Robert Koblischke from the New Media Department, WU Wien - the guy 
>> who updated www.objectspot.org for example (afaik you
>> worked on that partly as well).
>> 
>> I got a short question for you and I hope you can help me out with an 
>> answer here:
>> 
>> I have done some retrieval evaluations for my diploma thesis and the text 
>> search ranking algorithm is one I evaluated. However I'd need to know the 
>> formula/process behind this ranking algorithm, so a link to some basic 
>> documentation on this would really help.
>> 
>> After doing some research on the web, I finally landed on your posting here
>> http://openacs.org/forums/message-view?message_id=22013
>> where you write:
>> "The ranking function uses methodology proposed by Andrew Kovalenko and 
>> Nickolay Kharin. Many thanks to Andrew and Nickolay who gave their 
>> permission to use their methodology in the ranking function of OpenFTS."
>> 
>> and the OpenFTS primer:
>> "The ranking function uses methodology proposed by Andrew Kovalenko and 
>> Nickolay Kharin."
>> 
>> 
>> I know that was a long time ago, but maybe you still know where to find the 
>> documentation on that ranking formula if that is publicly available? A 
>> simple link would be enough.
>> 
>> I'd really appreciate any info here,
>> Thanks in advance,
>> Robert Koblischke
>> WU Wien
>> 
>> P.S.: I have also posted in the OpenFTS forum 
>> (https://sourceforge.net/forum/forum.php?thread_id=3343362&forum_id=97445), 
>> but I am not sure if anyone still reads that.
>> 
>
>

 	Regards,
 		Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: [email protected], http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july