AW: AW: Sql nested set from dictionary

I now use a static IndexSearcher and I can see a performance improvement. Unless you have any more tips, I think I will leave it like this.

Thank you very much,
Adrian



________________________________
Von: Adrian Vintu <adrianvintu-/[email protected]>
An: [email protected]
Gesendet: Freitag, den 25. Juni 2010, 16:53:28 Uhr
Betreff: AW: [OT] Sql nested set from dictionary

  
- Indexing -

w.optimize - nope
w.close - yes

- Searching - 

if i keep the IndexSearcher i get some "still open or not opened exception". will check later.

the following
  StandardAnalyzer analyzer = new StandardAnalyzer( Version.LUCENE_ 29);
  Directory index = new MMapDirectory( new File("lucene_ lexems")) ;
are static - lazy loaded.

well, the whole index folder is 8 megs... on the desktop it works great, but on the mobile it takes 1 second...

- i am as new as possible with Lucene, so please forgive my ignorance :) -

thank you,
Adrian




________________________________
Von: Ryan Heath <ryan.Q.heath@ gmail.com>
An: win_tech_off_ topic@yahoogroup s.com
Gesendet: Freitag, den 25. Juni 2010, 16:36:32 Uhr
Betreff: Re: [OT] Sql nested set from dictionary

  
define "slow-ish" ...

We have results ~10ms for an index that is 2 GB in size.

I think you miss a w.Optimize() and w.Close() after the documents are added?
We also keep the IndexSearcher around, its threadsafe.

HTH
// Ryan


On Fri, Jun 25, 2010 at 4:23 PM, Adrian Vintu <adrianvintu@ yahoo.com> wrote:

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>  >
>
>>
> 
>>      
> 
>Hi Ryan,
>
>After much testing and playing around with different solutions, I decided to choose Lucene :)
>
>It works nice, but I would really like to get some extra performance - it's still slow-ish on the phones.
>
>Maybe you can give me some tips.
>
>Here's what I have for now.
>
>1. ~130.000 words to index
>
>2. indexing is as follows
>  StandardAnalyzer analyzer = new StandardAnalyzer( Version.LUCENE_ 29);
>  Directory index = new MMapDirectory( new File("lucene_ lexems")) ;
>>  IndexWriter w = new IndexWriter( index, analyzer, true, IndexWriter. MaxFieldLength. UNLIMITED) ;
>
>
>  for 130.000 times do
>        Document doc = new Document();
>        doc.add(new Field("title" , value,
> Field.Store. YES, Field.Index. ANALYZED) );
>        w.addDocument( doc);
>  end for
>
>3. getting the results
>  String querystr = searched_word + "*"; 
>  Query q = new QueryParser( Version.LUCENE_ 29, "title", analyzer).parse( querystr) ;
>>  int hitsPerPage = 110;
>  IndexSearcher searcher = new IndexSearcher( index, true);
>  TopDocs topDocs = searcher.search( q, hitsPerPage) ;
>  ScoreDoc[] hits = topDocs.scoreDocs;
>
>I would have tried Snowball but I cannot find the sources and I also don't think it's much help...
>
>Any help would be greatly appreciated :)
>
>Thank you,
>Adrian
>
>
>
>
>
________________________________
Von: Ryan Heath <ryan.Q.heath@ gmail.com>
>
>An: win_tech_off_ topic@yahoogroup s.com
>
>Gesendet: Mittwoch, den 16. Juni 2010, 17:23:01 Uhr
>Betreff: Re: [OT] Sql nested set from dictionary
>
>
>  >
>
> 
>>      
> 
>Hi,
>
>
>Maybe a Lucene index is more appropriate  for your problem?
>
>
>We have recently moved from sqlserver fultext search to a lucene index and we will never look back ;)
>
>
>http://lucene. apache.org/ lucene.net/
>
>
>// Ryan
>
>
>On Wed, Jun 16, 2010 at 2:53 PM, Adrian Vintu <adrianvintu@ yahoo.com> wrote:
>
>>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>  >>
>>
>>>>
>> 
>>>>      
>> 
>>Hi all,
>>
>>I have a read-only dictionary of ~2 million words.
>>
>>They are now stored in an Sqlite database like this
>>
>>Table WORD
>>ID WORD     POINTER_TO_DATA
>>
>>1  bla      207
>>2  next     135
>>3  example  1234
>>4  other    3457
>>...
>>
>>Using this non-tree way of storing the words kills my system when doing a SELECT LIKE 'word%' - think mobile.
>>
>>I would need a script(SQL, Java, C#, Python,
>> etc) to create a tree out of the structure, or even better, a nested set like here http://dev.mysql. com/tech- resources/ articles/ hierarchical- data.html
>>
>>Any help would be greatly appreciated.
>>
>>(I have googled around, but I am not even sure what the keywords are...)
>>
>>Thank you,
>>Best regards,
>>Adrian Vintu
>>
>>
>