ADT.Heap performance poorer than just sorting and slicing
Chris Angelico <[email protected]>
| Newsgroups | gmane.comp.lang.pike.user |
|---|---|
| Message-ID | <CAPTjJmqUbOC4Gzca6T7EKh+1z8D9HZhfe_Q5SYjpTqB6-HyW=w@mail.gmail.com> |
I'm not sure whether I'm making a mess of my timings, or if ADT.Heap
just isn't designed for what I'm trying to do... Here's the situation:
String.fuzzymatch will examine two strings and give a score out of 100
for their similarity. I'd like to employ this to create a simple
spelling suggestion engine, by feeding it a test word and an array of
known words, and getting back an array of the 5 nearest matches. So
far, so good.
The dead simple technique is to build an array of the fuzzymatch
scores, then sort that and sort the strings alongside them:
sort(String.fuzzymatch(words[*], checkme), words);
best_matches = words[<4..];
My brain's telling me, though, that it ought to be possible to do this
with a heap; retain only the current top five, and discard the rest.
In theory, that ought to use less memory and be faster, right?
ADT.Priority_queue looks like exactly what I want. But it's about 50%
slower, and that with the optimization of not putting into the heap
anything we're about to take straight out again (without which it'd be
63% slower):
object heap = ADT.Priority_queue();
int worst = 0;
foreach (words, string word)
{
int score = String.fuzzymatch(word, checkme);
if (sizeof(heap) < 5) {heap->push(score, word); continue;}
if (score > worst) {heap->push(score, word); heap->pop();}
}
best_matches = ({ });
while (sizeof(heap)) best_matches += ({heap->pop()});
Is there a better way to use a heap? Or should I simply accept that
the highly optimized sort() function is generally going to be better
for anything less than billions of rows?
ChrisA