Binarycloud and UTF

Hi

I have looked at UTF and PHP and I'd propose that binarycloud should
have the following Unicode policy:

A.) All PHP and Smarty template files are UTF-8 without BOM (Quick 
overview available at [1]). This ensures that we can have all languages 
in strings etc.


B.) We use iconv and the iconv functions (from the manual):

   - iconv_strlen  Returns the character count of string
   - iconv_strpos  Finds position of first occurrence of a needle within 
a haystack
   - iconv_strrpos Finds the last occurrence of a needle within the 
specified range of haystack
   - iconv_substr  Cut out part of a string

Though mbstrings provides more multi byte safe functions it is not very 
common and if binarycloud depends on it we set the entry level pretty 
high (own server).

Let's have a look at the functions provided by mbstrings:

  - mb_ereg_*  Ereg seems not to be multi byte safe. PREG is not UTF 
safe seemingly, too. However, there seems to be a "/u" flag for unicode 
support. We need to investigaget there [2].
  - mb_parse  We need no global parsing from GET/POST/COOKIE, I guess
  - mb_split  Splitting by regex can be done with preg and "/u" flag
  - mb_strcut There is iconv_substr that does the same
  - mb_strimwidth The same can be done with iconv_substr and iconv_strlen
  - mb_strlen, mb_strpos, mb_strrpos, mb_substr There are equivalent 
methods with iconv
  - mb_strtolower, mb_strtoupper There is no equivalent, but I do not 
know if we need this for base code. Base configuration names, for 
example, are in English always and that fits into 7 bytes.
  - mb_strwidth No equivalent - required in base code?
  - mb_substitute_character, mb_substr_count No direct equivalent - 
required in base code?

I think we should evaluate all this and then look on binarycloud's base 
code (vortex packages can use mbstring IMO or we provide two versions) 
and get sure that unicode works. In many cases we will not need unicode 
safeness since the string that is operated are event ids and those come 
from configuration (See next point).


C.) Most configuration values should be English and use 7bit only. This 
means we do not need multi byte safe methods in many places.


D.) We use PEAR/I18Nv2 to transcode from UTF-8 to the charset requested 
by the user (or set in code). See [3] for the documentation of this 
feature of I18Nv2


If all base code follows this policy, it should not be too hard to 
create UTF-8 pages with binarycloud.


Regards,

Manuel

[1] http://www.cl.cam.ac.uk/~mgk25/unicode.html
[2] http://www.pcre.org/pcre.txt
[3] http://tinyurl.com/6egk3 (PEAR documentation)

-- 
GPG key: http://www.ggnore.net/~mholtgrewe/personal/files/public_key.asc

_______________________________________________
dev mailing list
dev-PnctHDZWAvB/Cz2I37pSEPZ4XP/[email protected]
http://lists.binarycloud.com/mailman/listinfo/dev