Re: Changing the charset

Am 20.05.26 um 00:45 schrieb Jim DeLaHunt via samba:
> On 2026-05-19 01:23, Stefan G. Weichinger via samba wrote:
> 
>>
>> I recently had an issue where a user couldn't correctly store an email 
>> out of his Outlook on an windows 11 PC to samba.
>>
>> Something around illegal characters (sry, I don't have the message at 
>> hand right now).
>>
>> The samba is 4.22.8 on Debian-13 and it has been historically upgraded 
>> over years ...
>>
>> The former admin configured:
>>
>>     dos charset = CP850
>>     unix charset = iso8859-15
>>
>> And I kept it that way because I didn't want to break things (and it 
>> wasn't a problem so far ...).
>>
>> Is changing this to UTF-8 problematic?
>> Does that break filenames or the displayed names? 
> 
> With all due respect to the expert and unendingly helpful Rowland Penny, 
> changing a charset from iso8859-15 to utf-8 might very well invalidate 
> some stored filenames and configuration strings.
> 
> ISO8859-15[1] is a character set for Western European languages written 
> using the Latin script. Every character occupies one byte.
> UTF-8[2] is a way of storing Unicode character values in 1, 2, 3, or 4- 
> byte sequences.
> 
> It is correct that every character in ISO8859-15 can be represented in 
> UTF-8. It is also correct that every character in ISO8859-15 which has a 
> byte value of 127 or less is represented in UTF-8 by a single byte with 
> the same byte value.
> 
> However, those characters in ISO8859-15 which have a byte value of 128 
> or greater are represented by two-byte sequences in UTF-8. So, text data 
> — for instance a file name — which was stored as bytes with values of 
> 128 or greater, encoded in ISO8859-15, will not be legible if read back 
> as UTF-8.
> 
> I do not know how Samba uses those admin settings of "dos charset" and 
> "unix charset". I do not know how it affects the interpretation of byte 
> values in configuration strings or filenames. Thus I can't advise you 
> what else you need to convert when you change these settings.  For what 
> it's worth, I have a suspicion that those settings may affect how the 
> byte sequences of filenames are interpreted, and if your filenames 
> include accented Latin characters and such, those filenames will need to 
> be converted from ISO8859-15 to UTF-8 form separately from Samba. Arnaud 
> Florent's suggestion of `convmv` to convert filenames from ISO8859-15 
> encoding to UTF-8 encoding seems like it would be helpful.

Thanks to all three of you who replied.

I also don't know what the charset setting in samba actually does. Is it 
used for "displaying" the names, does it change the way files are 
stored, etc etc (might be documented, sure, haven't yet found the time 
for that).

That's why I am very cautious here, that's a few TB of files on that 
server, and (as usual) business critical.

So far the one user failing to store his email with special chars hasn't 
complained. They don't do that often, the usual stuff works perfectly.

I also googled some setting in Outlook, how it stores exported mails. 
Suggested that to the local PC admin, he didn't test ...

I don't have a feeling how much work/time it needs to use convmv on 
these shares and if I could somehow to it step by step (or share by 
share) to proceed slowly.

So the urgency seems low and everything points to "don't touch that 
right now".

Thanks for the feedback anyway! I always like to learn.

p.s.: the stuff comes from a time with (early) Samba-3 ... I took over 
administration when their former employee left. Migrated to newer 
hardware, sure, but always kept the changes as small as possible.



-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba