Re: [gui-dev] LimeWire 4.2.3 international font display patch

OK but you still ignore the fact that the source of these denormalized names 
is purely the internal HFS+ storage, when you parse its directory entries. 
If we only normalized strings coming from directory entries, we no longer 
have any denormalized strings anywhere, including in the Response's we send!
So when storing a NFC filename on HFS+, HFS+ would denormalize it internally 
and we would not need to change anything in our code which uses NFC forms to 
access these files individually.

The only case where we will see these denormalized names is when we will add 
again the shared folder to our library, because we will use File.list() to 
enumerate them from the HFS+ filesystem; unfortunately at this time, HFS+ 
will not return to us the NFC forms we used to create the file, or the NFC 
forms we want to share to the network and through which the file remains 
accessible. So the solution is just to normalize immediately the 
denormalized entries coming from File.list().

OK we will not be able to display the denormalized strings coming from older 
servents, but at least we will stop sending denormalized strings to the 
network in our responses... which are inconvenient, including for non MacOS 
user: notably for Windows 95/98 or Linux users trying to download from a Mac 
and get deceptive Query Replies with many square boxes after the base letter 
for each decomposed accent!

Trying to normalize these responses may help, but it may slow down the 
processing of incoming results (that's why we should not do that during the 
routing of responses, but only by the final recipient that initiated the 
query, when the response will be displayed in the search result list). But 
even this will not be necessary, if all servents on the Gnet stop sending 
denormalized results (the way Limewire does on MacOS).

----- Original Message ----- 
From: "Sam Berlin" <[email protected]>
To: <[email protected]>
Sent: Saturday, November 27, 2004 11:09 PM
Subject: RE: [gui-dev] LimeWire 4.2.3 international font display patch


> Hi Philippe,
>
> There is no problem with query routing or matching.  All those routines
> correctly use the normalized form of the filename.  The problem is in the
> actual 'Response' object -- it is constructed with a non-normalized name.
> If the name is sent from an OSX system to another OSX system, then it is
> displayed fine (according to your email).  However, if that response is 
> sent
> to a non OSX system, then it is displayed with the additional characters
> (again, according to your email).  We can correct this for the future by
> having the name normalized in the Response object.  Unfortunately, it is 
> not
> possible for us to correct all the existing LimeWire's out there sending
> non-normalized responses.  In order to display those correctly, we'd have 
> to
> normalize the filename of the search result prior to displaying it.
>
> For reference points in the code, see the various places in FileManager 
> that
> a new Response is constructed and then in the Response constructor how it 
> is
> not normalized (and is sent over the wire un-normalized).  However,
> insertion into the QueryRouteTable does use the normalized string
> (HashFunction.keywords normalizes it, and is used by QueryRouteTable.add,
> which is used by FileManager.buildQRT).  The internal Trie in FileManager
> also uses the normalized strings (done by FileManager.extractKeywords, 
> which
> is used by FileManager.addFile).  Queries sent out are also normalized 
> (done
> by the various constructors if QueryRequest) and XML queries are also
> normalized (done by InputPanel.getInput).
>
> Because the problems are not in routing or matching, this problem is 
> purely
> a display issue (unless the file can't be created on disk because of the
> improper characters).
>
> heavy -- If you've managed to read this far, can you test whether or not
> these results can be downloaded okay?
>
> Thanks,
> Sam
>
>> -----Original Message-----
>> From: [email protected] [mailto:gui-dev-
>> [email protected]] On Behalf Of Philippe Verdy
>> Sent: Saturday, November 27, 2004 4:37 PM
>> To: [email protected]
>> Subject: Re: [gui-dev] LimeWire 4.2.3 international font display patch
>>
>> Not necessarily: you can normalize filenames read from filesystems so 
>> that
>> they will be shared easily with Windows or Unic users, and this won't
>> prevent other mac users to get access to these files, or to read these
>> filenames .
>>
>> On the opposite, it is the fact that we keep the denormalized filenames
>> read
>> from Mac HFS+ filesystems that causes interoperability problems with 
>> other
>> users (this causes no problem only for Mac users, that won't see the
>> difference if they now receive normalized filenames from Windows/Linux or
>> from corrected Mac versions...)
>>
>> So there's no need to normalize queries or results in Gnutella, only when
>> reading files from local shared directories...
>>
>> Normalizing incoming results would only help reading these results on non
>> MacOS systems.
>>
>> And of course we should not normalize queries during query matching, but
>> instead assume that all files accessible on Gnutella are normalized
>> preferably in NFC form, as well as all sent queries and results... The
>> only
>> case where we will need normalization is when those strings are FIRST
>> generated:
>> - on user input within search forms, this is the keyboard driver that
>> normaly generates precomposed forms, but if not sure, we could normalize
>> the
>> user input string before processing.
>> - when reading or writing files, we can directly use filenames in NFC 
>> form
>> on all filesystem types, including HFS+!
>> - when reading directories the entries should already be in NFC form,
>> except
>> on MacOS HFS+ where it will be extremely useful to reconvert these
>> denormalized strings back to NFC before processing and inclusion in our
>> shared library list.
>>
>> So no need to modify the query routing/matching algorithms whose
>> performance
>> are already critical...
>>
>> ----- Original Message -----
>> From: "Sam Berlin" <[email protected]>
>> To: <[email protected]>
>> Sent: Saturday, November 27, 2004 10:16 PM
>> Subject: RE: [gui-dev] LimeWire 4.2.3 international font display patch
>>
>>
>> > Hi Philippe,
>> >
>> > Performing the normalization will indeed fix things for the future.
>> > Unfortunately, there's a ton of existing clients out there that aren't
>> > sending correctly formatted results.  To display those properly,
>> > normalization would indeed need to be done on every incoming result.
>>
>>
>> _______________________________________________
>> gui-dev mailing list
>> [email protected]
>> http://www.limewire.org/mailman/listinfo/gui-dev
>
> _______________________________________________
> gui-dev mailing list
> [email protected]
> http://www.limewire.org/mailman/listinfo/gui-dev
> 

_______________________________________________
gui-dev mailing list
[email protected]
http://www.limewire.org/mailman/listinfo/gui-dev