[GeneralDiscussion] unicode progress

The patches I've just pushed to -unstable include: "a bunch more unicode-related fixes. We can now upgradeAll the content of zwiki.org then serve it without obvious problems (complete cataloging requires unicode-aware indexes)"

This is a good milestone. If you are able, please test the dev.zwiki.org server and if you're brave, test the unstable branch on your own wikis. It does require a manual /upgradeAll to get things working. This will convert everything in your wiki from utf-8 to unicode.

The catalog situation is not bad - both Andreas' TextIndexNG3 and Dieter's ManagableIndex are unicode aware. The latter provides a keyword index, which we need for the parents field, but I haven't got it working yet. When I do, Zwiki-unstable/unicode/2 should be usable in production.. 

Getting the upgrade process working - even manually - was not easy at all, because of the enmeshment of updating pages and updating the outline cache. Poor layering/encapsulation. I have been reduced to groping - testing and fixing the issues as I find them - though I do have a reasonable picture of the end goal in my head, eg the boundary line between encoded and unicode text. The unicode conversion is having the expected effect of clarifying architectural problems. Some time soon, new developer docs will emerge and help clarify things further. I'm debating whether to start these in a fresh high-signal Zwiki 2 wiki, or co-exist on zwiki.org (with cleanups). 


--
forwarded from http://zwiki.org/GeneralDiscussion#[email protected]