Re: UTF-8 File Names
James Adcock <[email protected]> Sun, 5 Jan 2020 20:39:04 +0000
| Newsgroups | gmane.culture.literature.e-books.gutenberg.volunteers |
|---|---|
| Message-ID | <BY5PR11MB45002F58DCCA086C527E6878AE3D0@BY5PR11MB4500.namprd11.prod.outlook.com> |
--===============8377428297981406452== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_BY5PR11MB45002F58DCCA086C527E6878AE3D0BY5PR11MB4500namp_" --_000_BY5PR11MB45002F58DCCA086C527E6878AE3D0BY5PR11MB4500namp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Thanks Al -- this would be great information to continue to be shared in a = permanent appropriate location, such as in the general information section = located at upload.pglaf.org<https://upload.pglaf.org/> ________________________________ From: gutvol-d <gutvol-d-bounces-4LCm/o+qPMZ96Xl/[email protected]> on behalf of Al Haines <a= [email protected]> Sent: Sunday, January 5, 2020 11:51 AM To: [email protected] <[email protected]>; 'Project Gutenberg Volunteer Dis= cussion' <gutvol-d-4LCm/o+qPMZ96Xl/[email protected]>; Joseph E. Loewenstein, M.D. <loewenste= [email protected]>; 'Al Haines' <[email protected]>; 'Chuck Greif' <cbgrf@yahoo.= com>; 'David Widger' <[email protected]> Subject: Re: [gutvol-d] UTF-8 File Names UTF8 text files should have "-utf8" in their file names, e.g. "myfile-utf8.txt". This forces PG's posting software to treat text files as UTF8, rather than its defaulting to Latin1/ASCII. (This has been a de facto standard for some years now.) It's not necessary to do this for HTML files. If necessary, the posting software will prompt for the correct character set, but it's an easy prompt to skip through, or give the wrong answer to. The presence of "-utf8" in the filename obviates the need for the prompt. Latin1/ASCII text files don't trigger the prompt at all. If PG's upload check reports a UTF8 text file without the "-utf8", then I rename the file inside the zip file, before unzipping it. Conversely, if a text file arrives with "-lat1", "-ltn1", "-iso", "-asc", or some such Latin1/ASCII indicator, I rename the file to remove it, since addhd handles Latin1/ASCII files correctly. Also, to avoid spurious files generated by the posting software, I make sure the base name of the zip file is the same as the base names of the text and HTML files, e.g. myfile.zip contains myfile.txt (or myfile-utf8.txt) and myfile.htm. Re HTML files: PG's extension for them is ".htm". When an uploaded zip file contains an HTML file with the extension ".html", I rename it, as mentioned above. If this isn't done, the posting software copies the file to a new file with the extension ".htm", e.g. "myfile.html" is copied to "myfile.htm", leaving a spurious ".html" file. (BTW, the posting software never, ever, modifies the submitted files--it copies them to new files with the required name, then works with the new files.) While I'm at it, more on file names... As mentioned above, I rename text and HTML files, and sometimes zip files, so that their base names are the same (except for the "-utf8" part of text files). There's no need for text/HTML files to have some versioning component. For example, I recently handled a zip file named diam-pg.zip, which contained diam-b.html and diam-cc-utf8.txt. I renamed the files to remove the "-b" and "-cc" parts, and the zip file to remove the "-pg" part, of their respective names. I've also seen names like "myfile-text.txt" and "myfile-html.html", and "myfile-8.txt" and "myfile-h.html". There's no need for such double-indicators of a file's type. There have been any number of similar variations, some probably related to the PPer's versioning system as they work through the PPing process; some related the PPing software. In one of the DP posts are these fragments: > All of PG uploads are currently UTF-8, even if the file is Latin-1. No idea where this idea comes from, but it's wrong. See below. > Or even do away with the appending altogether. The direct upload panel lets > the White Washers know what encoding to expect. So they know to run the > program to convert the Latin-1 characters to UTF-8. Also wrong. The WWers don't see PG's upload screen (except for their own projects). They also never convert Latin1 to UTF8--that's done to posted Latin1/ASCII files by PG's behind-the-scenes software, which the WWers have no control over. I think that's sufficient food for thought/discussion for now... Al --_000_BY5PR11MB45002F58DCCA086C527E6878AE3D0BY5PR11MB4500namp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-= 1"> <style type=3D"text/css" style=3D"display:none;"> P {margin-top:0;margin-bo= ttom:0;} </style> </head> <body dir=3D"ltr"> <div style=3D"color: rgb(0, 0, 0); font-family: Calibri,Helvetica,sans-seri= f; font-size: 12pt;"> <span>Thanks Al -- this would be great information to continue to be shared= in a permanent appropriate location, such as in the general information se= ction located at <a href=3D"https://upload.pglaf.org/" fg_scanned=3D"1">upload.pglaf.org</a>= </span></div> <div> <div id=3D"appendonsend"></div> <div style=3D"color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif; f= ont-size:12pt"> <br> </div> <hr tabindex=3D"-1" style=3D"display:inline-block; width:98%"> <div id=3D"divRplyFwdMsg" dir=3D"ltr"><font color=3D"#000000" face=3D"Calib= ri, sans-serif" style=3D"font-size:11pt"><b>From:</b> gutvol-d <gutvol-d= -bounces-4LCm/o+qPMZ96Xl/[email protected]> on behalf of Al Haines <[email protected]>= ;<br> <b>Sent:</b> Sunday, January 5, 2020 11:51 AM<br> <b>To:</b> [email protected] <[email protected]>; 'Project Gutenberg = Volunteer Discussion' <gutvol-d-4LCm/o+qPMZ96Xl/[email protected]>; Joseph E. Loewenste= in, M.D. <[email protected]>; 'Al Haines' <[email protected]&g= t;; 'Chuck Greif' <cbgrf-/[email protected]>; 'David Widger' <cdwidger@gmai= l.com><br> <b>Subject:</b> Re: [gutvol-d] UTF-8 File Names</font> <div> </div> </div> <div class=3D"BodyFragment"><font size=3D"2"><span style=3D"font-size:11pt"= > <div class=3D"PlainText">UTF8 text files should have "-utf8" in t= heir file names, e.g.<br> "myfile-utf8.txt". This forces PG's posting software to tre= at text<br> files as UTF8, rather than its defaulting to Latin1/ASCII. (This has<= br> been a de facto standard for some years now.) It's not necessary to d= o<br> this for HTML files.<br> <br> If necessary, the posting software will prompt for the correct character<br= > set, but it's an easy prompt to skip through, or give the wrong answer<br> to. The presence of "-utf8" in the filename obviates the ne= ed for the<br> prompt. Latin1/ASCII text files don't trigger the prompt at all.<br> <br> If PG's upload check reports a UTF8 text file without the "-utf8"= , then<br> I rename the file inside the zip file, before unzipping it.<br> <br> Conversely, if a text file arrives with "-lat1", "-ltn1"= ;, "-iso",<br> "-asc", or some such Latin1/ASCII indicator, I rename the file to= remove<br> it, since addhd handles Latin1/ASCII files correctly.<br> <br> Also, to avoid spurious files generated by the posting software, I make<br> sure the base name of the zip file is the same as the base names of the<br> text and HTML files, e.g. myfile.zip contains myfile.txt (or<br> myfile-utf8.txt) and myfile.htm.<br> <br> Re HTML files: PG's extension for them is ".htm". When an u= ploaded zip<br> file contains an HTML file with the extension ".html", I rename i= t, as<br> mentioned above. If this isn't done, the posting software copies the<= br> file to a new file with the extension ".htm", e.g. "myfile.h= tml" is<br> copied to "myfile.htm", leaving a spurious ".html" file= . (BTW, the<br> posting software never, ever, modifies the submitted files--it copies<br> them to new files with the required name, then works with the new<br> files.)<br> <br> While I'm at it, more on file names...<br> <br> As mentioned above, I rename text and HTML files, and sometimes zip<br> files, so that their base names are the same (except for the "-utf8&qu= ot;<br> part of text files). There's no need for text/HTML files to have some= <br> versioning component. For example, I recently handled a zip file name= d<br> diam-pg.zip, which contained diam-b.html and diam-cc-utf8.txt. I<br> renamed the files to remove the "-b" and "-cc" parts, a= nd the zip file<br> to remove the "-pg" part, of their respective names.<br> <br> I've also seen names like "myfile-text.txt" and "myfile-html= .html", and<br> "myfile-8.txt" and "myfile-h.html". There's no ne= ed for such<br> double-indicators of a file's type.<br> <br> There have been any number of similar variations, some probably related<br> to the PPer's versioning system as they work through the PPing process;<br> some related the PPing software.<br> <br> In one of the DP posts are these fragments:<br> <br> > All of PG uploads are currently UTF-8, even if the file is Latin-1. <b= r> <br> No idea where this idea comes from, but it's wrong. See below.<br> <br> > Or even do away with the appending altogether. The direct upload panel= <br> lets<br> > the White Washers know what encoding to expect. So they know to run<br= > the <br> > program to convert the Latin-1 characters to UTF-8.<br> <br> Also wrong. The WWers don't see PG's upload screen (except for their<= br> own projects). They also never convert Latin1 to UTF8--that's done to= <br> posted Latin1/ASCII files by PG's behind-the-scenes software, which the<br> WWers have no control over.<br> <br> I think that's sufficient food for thought/discussion for now...<br> <br> <br> Al<br> <br> </div> </span></font></div> </div> </body> </html> --_000_BY5PR11MB45002F58DCCA086C527E6878AE3D0BY5PR11MB4500namp_-- --===============8377428297981406452== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZ3V0dm9sLWQg bWFpbGluZyBsaXN0Cmd1dHZvbC1kQGxpc3RzLnBnbGFmLm9yZwpodHRwczovL2xpc3RzLnBnbGFm Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2d1dHZvbC1kClVuc3Vic2NyaWJlOiBodHRwczovL2xpc3Rz LnBnbGFmLm9yZy9tYWlsbWFuL29wdGlvbnMvZ3V0dm9sLWQ= --===============8377428297981406452==--