Converting byte position to character position
"Per Hedbor () @ Pike (-) importm?te f?r mailinglistan" <[email protected]>
| Newsgroups | gmane.comp.lang.pike.user |
|---|---|
| Message-ID | <[email protected]> |
> And it works, but ow! what a mess. I could alternatively go through
> the string manually, counting up by byte range (<127 = add one, <0x800
> = add two, etc), but I suspect that'll work out even messier. Is there
> a cleaner way to do this?
Well. With UTF-8, no, not really. Unless you cache it somewhere.
This version is almost exactly as fast for short strings (like your
example) but significantly faster for long strings:
int utf8byte_to_char( string txt, int pos )
{
int m = String.range(txt)[1];
if( m < 128 )
return pos;
int c;
if( m < 0x800 )
{
for( ; pos; c++,pos-- )
if( txt[c] > 0x80 ) pos--;
}
else if( m < 0x10000 )
{
for( ; pos; c++,pos-- ) {
if( txt[c] > 0x800 ) pos--;
if( txt[c] > 0x80 ) pos--;
}
}
else for( ; pos; c++,pos-- )
{
switch( txt[c] )
{
case 0x4000000..: pos--;
case 0x200000..0x3ffffff: pos--;
case 0x10000..0x1ffff: pos--;
case 0x800..0xffff: pos--;
case 0x80..0x7ff: pos--;
}
}
return c;
}
Of course, your version can also be optimized significantly, at least
when the desired position is rather small and the string is long:
int utf8byte_to_char2( string txt int pos )
{
return sizeof(utf8_to_string(string_to_utf8(txt[..pos-1])[..pos-1]));
}
This is the fastest one I have tested yet, but please avoid looking
for byte 10000394 in a 2Mchar long unicode string... :)
--
Per Hedbor