Re: silvafind

>> ----------------
>> search using these words: and, or, not
>>   Error Type: ParseError
>>   Error Value: Token 'ATOM' required, u'not' found
>> ----------------
>> search using these characters: +, *, %, !, $, £
>>   Error Type: ParseError
>>   Error Value: Query contains only common words: u'+'
>
> I changed this so ParseErrors are catched and the string
> "Search query contains only common or reserved words." is displayed.

verified. works as expected.

>> ----------------
>> i have renamed the silva root. then i couldn't find anything  
>> anymore. i corrected the "below path" parameter in silvafind and  
>> tried to reindex/update the zcatalog without success. how can one  
>> repopulate the catalogue?
>
> Still investigating..

fixed with the rebuild catalog button in service extensions.

>> ----------------
>> when creating a new silva root, silvalayout should be installed  
>> automatically. when silvalayout is not installed i get the  
>> following error:
>>   Error Type: AttributeError
>>   Error Value: @@set_content_type_and_nocache
>
> Done, it should be installed automaticly. The error was also very  
> confusing.

this is fixed with a completely new silva root. but strange is that  
the other silva root (same zope instance) now has always the above  
error. i can install/uninstall SilvaLayout, refresh all/restart zope  
and the error remains (traceback below). maybe it was  
SilvaOutoftheBox which i had previously installed (Error Type:  
KeyError, Error Value: u'Out of the Box'). but i removed it as well.  
so to get the old silva root working again i had to remove  
SilvaLayout from the Products folder.

>> ----------------
>> i uploaded a text file as silva file and received the following  
>> error:
>>   Error Type: UnicodeDecodeError
>>   Error Value: 'ascii' codec can't decode byte 0xc3 in position 4:  
>> ordinal not in range(128)
>> this is fized by using "return unicode(data, 'utf8')" instead of  
>> "return data" in converters.py.
>
> Oops, I shouldn't have missed that one.

verified. works as expected.

>> ----------------
>> when using "*" or "?" in a search the found word in the text  
>> snippet is not marked with the css tags. if i'm correct, in this  
>> case, the text snippet starts always from the start of the  
>> document and not from where the search string was found.
>
> Fixed this, wildcards are now taken into account when hiliting text  
> snippets.

verified. works as expected.

>> ----------------
>> when searching in pdfs the found word in the text snippet is not  
>> marked with the css tags. in this case, the text snippet starts  
>> always from the start of the document as well. why is the text all  
>> lower case?
>
> All text from pdfs is lower case and has no punctuation, because  
> this is how it is stored in the catalog, and we do not want to  
> store the fulltext again on the object. We also don't want to run  
> the conversion code for every file searchresult.
> The hiliting of text snippets should work though (it does for me),  
> and it does not always start at the beginning of the document.

now, it works here as well.

>> ----------------
>> the formatting of publication date and creation time, modification  
>> time and publication time is not identical.
>
> Yes the formatting of publication date is different. I renamed this  
> to 'date' since it's much smarter then the other metadata fields  
> (It chooses the most appropriate date, because files don't have  
> publication dates)

still different formatting: "10 mar 2007 20:46" <->  "10/03/2007    
20:46"

SilvaNews has a nice configuration option for the date formatting:  
the manager can choose the locale and between short, medium, long and  
full format (see service_news). maybe you can borrow the code from  
there.

>> ----------------
>> there should be an option that forces silvafind to respect virtual  
>> host roots and not show results from a different site.
>
> Still investigating..

i meant nested domains. what happens when silvafind reaches  
publications in lower levels which are declared as site root?

>> ----------------
>> there should be an option for hiding protected content that forces  
>> silvafind to respect virtual host roots and not show results from  
>> a different site.
>> ----------------
>
> I don't think the catalog holds any data from other sites?
> Needs further investigation..

sorry, i should have deleted this text because my questions about  
security contains this.

>> uploading file assets does not work when filesystem storage is  
>> enabled (ExtFile 1.5.2 or 1.5.4):
>>   Error Type: AttributeError
>>   Error Value: data
>> traceback
>
> Fixed this

verified. works as expected.

>> ----------------
>> it should be configurable if the thumbnails of silva image is  
>> shown or not in the search result.
>
> I don't understand. All the searchresult fields can be turned on or  
> off (just uncheck thumbnail in the edit form).

i really thought that i did this. but it works as expected. maybe you  
could add some notes "Display the thumbnails of the silva images."

>> ----------------
>> i would like a field where i could search in one go in content,  
>> title, short title and object ID and i would name this field "full  
>> text". i would rename the current "full text" to "content". this  
>> would then be analog to mysql's fulltext search meaning.
>> http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
>
> The fulltext index includes the title, so you can search for titles  
> with fulltext. Searching on id could be interesting. Maybe we  
> should make a different SearchCriterion as you suggest.

including searching for an id is a must. i can upload files without  
title/short title. how can those be found? and somebody might like to  
search for *.doc or *.pdf

hmmm, i think that the titles of images and files are not searched at  
all. i cannot find them even when i have entered some text in their  
title fields and even after rebuilding the catalog...

BTW, is searching case-sensitive? when i search for *.pdf or *.doc i  
get the expected result. but when i search for capitalized *.PDF and  
*.DOC i get a nasty zope error:
   Error Type: error
   Error Value: nothing to repeat

>> there should be an option to make the search more google-like.  
>> e.g. automatically adding an "*" to every search string.
>
> I don't think this is what Google does. Search for your lastname  
> and leave the last characters out. It's just that people write so  
> many spelling mistakes, that it always finds something ;)
> I'm not sure this would help getting better searchresults.

google uses stemming technology. i.e. when i search for "auto" it  
will also display results with "autos". or their example "diet",  
"dietary":
<http://www.google.com/support/bin/static.py? 
page=searchguides.html&ctx=basics>
in german you often have a base word with different endings (grammar  
stuff). normally, you use just the shortest common part of this word  
without the ending for searching. and thanks to the stemming  
technology you normally find what you are looking for. automatically  
adding an "*" would emulate this behavior. at least a short hint  
about "*, ?" should be shown in the public view.

>> what about security? who can use silvafind? what about protected  
>> content? are the different zope an public silva roles respected,  
>> especially fulltext results?
>
> This should be investigated further. Content that is not viewable  
> should not end up in the search. If it does end up in the search,  
> the login box would appear since anonymous users would not have  
> enough priviledges to view this. So I don't think there are  
> security issues, just some annoyances.

see andy's comments.

>> i have read about the new feature "Automatic fulltext cataloging  
>> of pdf files" and i thought, why only pdfs? so i modified  
>> converters.py a little bit and now we also have fulltext  
>> cataloging of word documents. i use antiword which can convert the  
>> files from word to plain text. i have used this program very  
>> successful last year for a conference site (previewing abstracts  
>> in the browser without the need for downloading them). the new  
>> version of converters.py is at the end of this email.
>
> Interesing. I have no experience with Antiword but I had some code  
> using wv that converted word docs. Antiword seems a bit cleaner  
> though.
> The thing with word files is that if you upload a corrupt file,  
> your whole server can go down. For example try the word.doc file  
> from the archetypes tests (Products/Archetypes/tests/input/word.doc).

antiword handles this quite nicely and fast by displaying "I'm afraid  
the text stream of this file is too small to handle." no hanging at all.

a new bug has appeared since yesterday: when SilvaLayout is not  
installed selecting "skin" in the search options result in this error:

   Error Type: AttributeError
   Error Value: 'module' object has no attribute 'SilvaLayout'

regards,
marc



Traceback (innermost last):
   Module ZPublisher.Publish, line 114, in publish
   Module ZPublisher.mapply, line 88, in mapply
   Module ZPublisher.Publish, line 40, in call_object
   Module Products.SilvaViews.ViewRegistry, line 156, in index_html
   Module Shared.DC.Scripts.Bindings, line 311, in __call__
   Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec
   Module Products.FileSystemSite.FSPageTemplate, line 195, in _exec
   Module Products.FileSystemSite.FSPageTemplate, line 134, in pt_render
   Module Products.PageTemplates.PageTemplate, line 104, in pt_render
    - <FSPageTemplate at /silva11/service_views/Silva/edit/Container/ 
tab_edit used for /silva11/service_views/Silva/edit/Container/ 
Publication>
   Module TAL.TALInterpreter, line 206, in __call__
   Module TAL.TALInterpreter, line 250, in interpret
   Module TAL.TALInterpreter, line 424, in do_optTag_tal
   Module TAL.TALInterpreter, line 406, in no_tag
   Module TAL.TALInterpreter, line 250, in interpret
   Module TAL.TALInterpreter, line 711, in do_useMacro
   Module TAL.TALInterpreter, line 250, in interpret
   Module Products.Silva.monkey, line 19, in  
silva_do_insertStructure_tal
   Module Products.PageTemplates.TALES, line 221, in evaluate
    - URL: file:Silva/views/macro_index.pt
    - Line 39, Column 2
    - Expression: standard:'request/model/ 
@@set_content_type_and_nocache'
    - Names:
       {'container': <DirectoryViewSurrogate at /silva11/ 
service_views/Silva/edit/Container>,
        'context': <DirectoryViewSurrogate at /silva11/service_views/ 
Silva/edit/Container/Publication>,
        'default': <Products.PageTemplates.TALES.Default instance at  
0x1753b70>,
        'here': <DirectoryViewSurrogate at /silva11/service_views/ 
Silva/edit/Container/Publication>,
        'loop': <Products.PageTemplates.TALES.SafeMapping object at  
0x44c1378>,
        'modules':  
<Products.PageTemplates.ZRPythonExpr._SecureModuleImporter instance  
at 0x201fe40>,
        'nothing': None,
        'options': {'args': ()},
        'repeat': <Products.PageTemplates.TALES.SafeMapping object at  
0x44c1378>,
        'request': <HTTPRequest, URL=http://localhost:8080/silva11/ 
edit/index_html>,
        'root': <Application at >,
        'template': <FSPageTemplate at /silva11/service_views/Silva/ 
edit/Container/tab_edit used for /silva11/service_views/Silva/edit/ 
Container/Publication>,
        'traverse_subpath': [],
        'user': admin}
   Module Products.PageTemplates.Expressions, line 185, in __call__
   Module Products.PageTemplates.Expressions, line 173, in _eval
   Module Products.PageTemplates.Expressions, line 127, in _eval
    - __traceback_info__: request
   Module Products.PageTemplates.Expressions, line 286, in  
restrictedTraverse
    - __traceback_info__: {'path': ['model',  
'@@set_content_type_and_nocache'], 'TraversalRequestNameStack': []}
   Module Products.Five.traversable, line 100, in __bobo_traverse__
AttributeError: @@set_content_type_and_nocache

_______________________________________________
silva-dev mailing list
[email protected]
https://infrae.com/mailman/listinfo/silva-dev