KMeans - Going forward
Richhiey Thomas <[email protected]> Sun, 23 Jul 2017 15:50:51 -0400
| Newsgroups | gmane.comp.search.xapian.devel |
|---|---|
| Message-ID | <CAPHaz=BFyqWMcW3_+fUZwo=wDaufKhArdOUbs7TFyYdgVR54DQ@mail.gmail.com> |
Hello, Now work on stopword removal and stemming is almost ending and the run time for KMeans seem to be getting lesser (around 0.15 s for 100 documents and this increases to around 1.2 s with 500 documents and 2.5 s with 1000 documents). I tried this out on the BBC datasets available with a value k=5, since there were 5 categories in the dataset. Going forward, the next step to optimize KMeans is to use the faster optimized version of KMeans which reduces distance computations developed by Charles Elkan. For this, I will be providing the user an option to specify with the constructor whether they would want the standard algorithm or Elkans algorithm. and write a method within KMeans to implement the triangle inequality optmization. I will also be moving RoundRobin to the testsuite. Thanks.