Re: Stripping out Unicode combining characters (diacritics) -
[email protected] ("Brad Baxter") Wed, 7 May 2008 15:20:06 -0400
| Newsgroups | perl.perl4lib,perl.i18n |
|---|---|
| Message-ID | <[email protected]> |
------=_Part_11335_5522559.1210188006334 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: base64 Content-Disposition: inline SnVzdCB0byB0aHJvdyB0aGlzIG91dCB0aGVyZTogeW91IG1heSBiZSBpbnRlcmVzdGVkIGluIFRl eHQ6OlVuaWRlY29kZQooaHR0cDovL3NlYXJjaC5jcGFuLm9yZy9+c2J1cmtlL1RleHQtVW5pZGVj b2RlLTAuMDQvKSBpZiB5b3VyIHVsdGltYXRlCmdvYWwgaXMgdG8gdHJ5IHRvIHJlcHJlc2VudCBh IHVuaWNvZGUgY2hhcmFjdGVyIHdpdGggaXRzIGNsb3Nlc3QgYXNjaWkKKG9yIHBlcmhhcHMgSSBz aG91bGQgc2F5LCAicm9tYW5pemVkIikgZXF1aXZhbGVudC4KCi0tIEJyYWQKCk9uIFdlZCwgTWF5 IDcsIDIwMDggYXQgOTo1MSBBTSwgRG9yYW4sIE1pY2hhZWwgRCA8ZG9yYW5AdXRhLmVkdT4gd3Jv dGU6Cgo+IEkgcmVjZWl2ZWQgYSBudW1iZXIgb2YgaGVscGZ1bCBzdWdnZXN0aW9ucyBhbmQgc29s dXRpb25zLiAgVGhlIGFwcHJvYWNoIEkKPiBkZWNpZGVkIHRvIGFkb3B0IGluIG15IGxhcmdlciBz Y3JpcHQgaXMgdG8gJ2RlY29kZScgYWxsIHRoZSBpbmNvbWluZyBmb3JtCj4gaW5wdXQgYXMgVVRG LTggYXMgd2VsbCBhcyB0aGUgaW5wdXQgZnJvbSB0aGUgZGF0YWJhc2UgdGhhdCBJJ2xsIGJlIG1h dGNoaW5nCj4gdGhlIGZvcm0gaW5wdXQgYWdhaW5zdC4gIFRoaXMgc2VlbXMgdG8gYWxsb3cgdGhl ICdccHtNfScgc3ludGF4IHRvIHdvcmsgYXMKPiBleHBlY3RlZCBpbiBhIFBlcmwgcmVnZXhwLiAg SW4gbXkgdGVzdC5jZ2kgc2NyaXB0IGZvciBmb3JtIGlucHV0IGl0IHdvdWxkCj4gbGlrZSBsaWtl IHRoaXM6Cj4KPiAjIS91c3IvbG9jYWwvYmluL3BlcmwKPiB1c2Ugc3RyaWN0Owo+IHVzZSBDR0k7 Cj4gdXNlIEVuY29kZTsKPiBteSAkcXVlcnkgPSBDR0k6Om5ldygpOwo+IG15ICRzZWFyY2hfdGVy bSA9IGRlY29kZSgnVVRGLTgnLCRxdWVyeS0+cGFyYW0oJ3RleHQnKSk7Cj4gbXkgJHNhbnNfZGlh Y3JpdGljcyAgPSAkc2VhcmNoX3Rlcm07Cj4gJHNhbnNfZGlhY3JpdGljcyA9fiBzL1xwTSovL2c7 Cj4gcHJpbnQgcXEoQ29udGVudC10eXBlOiB0ZXh0L3BsYWluOyBjaGFyc2V0PXV0Zi04Cj4KPiBz ZWFyY2hfdGVybSAgICAgaXMgJHNlYXJjaF90ZXJtCj4gc2Fuc19kaWFjcml0aWNzIGlzICRzYW5z X2RpYWNyaXRpY3MKPiApOwo+IGV4aXQoMCk7Cj4KPiBJJ20gc2xvd2x5IGZpZ3VyaW5nIG91dCBo b3cgdG8gd29yayB3aXRoIFVuaWNvZGUgaW4gbXkgd2ViIHNjcmlwdHMsIGJ1dAo+IHN0aWxsIGhh dmUgYSBsb3QgdG8gbGVhcm4uICBUaGFua3MgZm9yIGFsbCB0aGUgaGVscC4gOi0pCj4KPiAtLSBN aWNoYWVsCj4KPiAjIE1pY2hhZWwgRG9yYW4sIFN5c3RlbXMgTGlicmFyaWFuCj4gIyBVbml2ZXJz aXR5IG9mIFRleGFzIGF0IEFybGluZ3Rvbgo+ICMgODE3LTI3Mi01MzI2IG9mZmljZQo+ICMgODE3 LTY4OC0xOTI2IG1vYmlsZQo+ICMgZG9yYW5AdXRhLmVkdQo+ICMgaHR0cDovL3JvY2t5LnV0YS5l ZHUvZG9yYW4vCj4KPgo+ID4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0KPiA+IEZyb206IERv cmFuLCBNaWNoYWVsIEQgW21haWx0bzpkb3JhbkB1dGEuZWR1XQo+ID4gU2VudDogTW9uZGF5LCBN YXkgMDUsIDIwMDggNzoyNyBQTQo+ID4gVG86IHBlcmwtaTE4bkBwZXJsLm9yZwo+ID4gQ2M6IFBl cmw0bGliCj4gPiBTdWJqZWN0OiBTdHJpcHBpbmcgb3V0IFVuaWNvZGUgY29tYmluaW5nIGNoYXJh Y3RlcnMgKGRpYWNyaXRpY3MpCj4gPgo+ID4gSSdtIHRyeWluZyB0byBzdHJpcCBvdXQgY29tYmlu aW5nIGRpYWNyaXRpY3MgZnJvbSBzb21lIGZvcm0KPiA+IGlucHV0IHVzaW5nIHRoaXMgY29kZToK PiA+Cj4gPiA8aGVhZD4KPiA+ICAgICA8TUVUQSBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiIGNv bnRlbnQ9InRleHQvaHRtbDsKPiA+IGNoYXJzZXQ9VVRGLTgiPiA8L2hlYWQ+IDxib2R5Pgo+ID4g ICA8Zm9ybSBhY3Rpb249InRlc3QuY2dpIiBhY2NlcHQtY2hhcnNldD0iVVRGLTgiIG1ldGhvZD0i Z2V0Ij4KPiA+ICAgICA8aW5wdXQgdHlwZT0idGV4dCIgbmFtZT0idGV4dCIgdmFsdWU9IiIgc2l6 ZT0iMTAiPgo+ID4gICAgIDxpbnB1dCB0eXBlPSJzdWJtaXQiIHZhbHVlPSJzdWJtaXQiPgo+ID4g ICA8L2Zvcm0+Cj4gPiA8L2JvZHk+Cj4gPiA8L2h0bWw+Cj4gPgo+ID4gIyEvdXNyL2xvY2FsL2Jp bi9wZXJsCj4gPiB1c2UgQ0dJOwo+ID4gJHF1ZXJ5ID0gQ0dJOjpuZXcoKTsKPiA+ICRzZWFyY2hf dGVybSA9ICRxdWVyeS0+cGFyYW0oJ3RleHQnKTsKPiA+ICRzYW5zX2RpYWNyaXRpY3MgID0gJHNl YXJjaF90ZXJtOwo+ID4gJHNhbnNfZGlhY3JpdGljcyAgPX4gcy9ccHtNfSovL2c7Cj4gPiAjJHNh bnNfZGlhY3JpdGljcyAgPX4gcy9vLy9nOwo+ID4gcHJpbnQgcXEoQ29udGVudC10eXBlOiB0ZXh0 L3BsYWluOyBjaGFyc2V0PXV0Zi04Cj4gPgo+ID4gJHNhbnNfZGlhY3JpdGljcwo+ID4gKTsKPiA+ IGV4aXQoMCk7Cj4gPgo+ID4KPiA+IEluIHRoZSBmb3JtLCBJJ20gaW5wdXR0aW5nIHRoZSBzdHJp bmcgIkJhcnRvzIFrIiB3aXRoIHRoZQo+ID4gYWNjZW50ZWQgY2hhcmFjdGVyIGJlaW5nIGEgYmFz ZSBjaGFyYWN0ZXIgKHNtYWxsIExhdGluIGxldHRlcgo+ID4gIm8iKSBmb2xsb3dlZCBieSBhIGNv bWJpbmluZyBhY3V0ZSBhY2NlbnQuICBIb3dldmVyLCB3aGVuIEkKPiA+IHByaW50ICh0byB0aGUg d2ViKSAkc2Fuc19kaWFjcml0aWNzLCBJIGdldCBteSBpbnB1dCB3aXRoIG5vCj4gPiBjaGFuZ2Ug LS0gdGhlIGNvbWJpbmluZyBkaWFjcml0aWMgaXMgc3RpbGwgdGhlcmUuICBJIGtub3cKPiA+IHRo YXQgbXkgaW5wdXQgaXMgbm90IGEgcHJlY29tcG9zZWQgYWNjZW50ZWQgY2hhcmFjdGVyLAo+ID4g YmVjYXVzZSBJIGNhbiBzdHJpcCBvdXQgdGhlIGJhc2UgIm8iIGFuZCB0aGUgY29tYmluaW5nIGFj Y2VudAo+ID4gZWl0aGVyIHN0YW5kcyBhbG9uZSBvciBqdW1wcyB0byBhbm90aGVyIGNoYXJhY3Rl ciBbMl0uCj4gPgo+ID4gVGhlICJccHtNfSIgaXMgYSBVbmljb2RlIGNsYXNzIG5hbWUgZm9yIHRo ZSBjaGFyYWN0ZXIgY2xhc3MKPiA+IG9mIFVuaWNvZGUgJ21hcmtzJywgZm9yIGV4YW1wbGUgYWNj ZW50IG1hcmtzIFsxXS4gIEkndmUgdHJpZWQKPiA+IHRoZXNlIHZhcmlhdGlvbnMgKGFuZCBtYW55 IG90aGVycykgYW5kIG5vbmUgc2VlbSB0byBiZSBkb2luZwo+ID4gd2hhdCBJIHdhbnQ6Cj4gPgo+ ID4gICAgICAgICRzYW5zX2RpYWNyaXRpY3MgPX4gcyNbXHB7TWFya31dKiMjZzsKPiA+ICAgICAg ICAkc2Fuc19kaWFjcml0aWNzID1+IHRyI1tccHtJbkNvbWJpbmluZ0RpYWNyaXRpY2FsTWFya3N9 XSMjOwo+ID4gICAgICAgICRzYW5zX2RpYWNyaXRpY3MgPX4gdHIjW1xwe019XSMjOwo+ID4gICAg ICAgICRzYW5zX2RpYWNyaXRpY3MgPX4gcy9ccHtNfSovL2c7Cj4gPiAgICAgICAgJHNhbnNfZGlh Y3JpdGljcyA9fiBzI1tccHtNfV0jI2c7Cj4gPiAgICAgICAgJHNhbnNfZGlhY3JpdGljcyA9fiBz I1x4ezAzMDF9IyNnOwo+ID4gICAgICAgICRzYW5zX2RpYWNyaXRpY3MgPX4gcyNceHswMDZGfVx4 ezAzMDF9IyNnOwo+ID4gICAgICAgICRzYW5zX2RpYWNyaXRpY3MgPX4gcyNbXHh7MDMwMH0tXHh7 MDM2Rn1dKiMjZzsKPiA+Cj4gPiBJJ20gcHVsbGluZyBteSBoYWlyIG91dCBvbiB0aGlzLi4uIHNv IGFueSBoZWxwIHdvdWxkIGJlCj4gPiBhcHByZWNpYXRlZC4gIElmIHRoZXJlJ3MgYW55IG90aGVy IGluZm8gSSBjYW4gcHJvdmlkZSwgbGV0IG1lIGtub3cuCj4gPgo+ID4gTXkgUGVybCB2ZXJzaW9u IGlzIDUuOC44IGFuZCB0aGUgc2NyaXB0IGlzIHJ1bm5pbmcgb24gYQo+ID4gc2VydmVyIHJ1bm5p bmcgU29sYXJpcyA5Lgo+ID4KPiA+IC0tIE1pY2hhZWwKPiA+Cj4gPiBbMV0gcGVyIGh0dHA6Ly9w ZXJsZG9jLnBlcmwub3JnL3BlcmxyZXR1dC5odG1sIGFuZCBvdGhlciBkb2N1bWVudGF0aW9uCj4g Pgo+ID4gWzJdIHVzaW5nICRzYW5zX2RpYWNyaXRpY3MgID1+IHMvby8vZzsKPiA+Cj4gPiAjIE1p Y2hhZWwgRG9yYW4sIFN5c3RlbXMgTGlicmFyaWFuCj4gPiAjIFVuaXZlcnNpdHkgb2YgVGV4YXMg YXQgQXJsaW5ndG9uCj4gPiAjIDgxNy0yNzItNTMyNiBvZmZpY2UKPiA+ICMgODE3LTY4OC0xOTI2 IG1vYmlsZQo+ID4gIyBkb3JhbkB1dGEuZWR1Cj4gPiAjIGh0dHA6Ly9yb2NreS51dGEuZWR1L2Rv cmFuLwo+ID4KPgo= ------=_Part_11335_5522559.1210188006334--