Re: UnicodeDecodeError: 'gbk' codec can't decode byte 0xa3 in position 3242: illegal multibyte sequence

Daniel Moody <[email protected]> Thu, 28 Apr 2022 15:30:45 -0500
Newsgroups	gmane.comp.programming.tools.scons.user
Message-ID	<CANws3JTkQeUi-eduAnXDLn9WB4pYT=8s7u+dsO6SQyVZu0QO-w@mail.gmail.com>
Hello Liruncong,

I can not reproduce it most likely because I need to change my locale,
However I think I may have a fix. I have pushed a small update which I hope
solves the issue for you. Can you please try it with the latest changes
from my branch?

On Thu, Apr 28, 2022 at 10:30 AM liruncong2018 <[email protected]> wrote:

> Hi,
> The following git test project can reproduce the coding problem：
> https://github.com/liruncong/NinJaTest.git
> Although I can reproduce it using this, I don't know if you can reproduce
> it after downloading it? Maybe the local language will be different.
> log:
> <<<
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
> path=D:\MYBIN\scons-local\;D:\MYBIN\Anaconda3\;D:\MYBIN\env\tools\gnu_gcc\arm_gcc\mingw\bin;%path%
>
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
> RTT_EXEC_PATH=D:\MYBIN\env\tools\gnu_gcc\arm_gcc\mingw\bin
>
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
> RTT_ROOT=D:\work\gitee\rt-thread_ninja
>
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>scons
> -j%NUMBER_OF_PROCESSORS% --verbose --experimental=ninja
>
> scons: Reading SConscript files ...
> Newlib version:2.4.0
> scons: done reading SConscript files.
> scons: Building targets ...
> Generating: build.ninja...
> Generated new build.ninja!
> Executing: run_ninja_env.bat
>
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set SystemDrive=C:
>
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set SystemRoot=C:\Windows
>
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
> TEMP=C:\Users\ARCHYI~1\AppData\Local\Temp
>
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
> TMP=C:\Users\ARCHYI~1\AppData\Local\Temp
>
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
> USERPROFILE=C:\Users\archyizimi
>
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
> COMSPEC=C:\Windows\system32\cmd.exe
>
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
> PATH=D:\MYBIN\env\tools\gnu_gcc\arm_gcc\mingw\bin;D:\MYBIN\scons-local\;D:\MYBIN;D:\MYBIN\Anaconda3\;C:\Program
> Files\Microsoft Visual
> Studio\2022\Enterprise\VC\Tools\MSVC\14.31.31103\bin\HostX86\x86;C:\Program
> Files\Microsoft Visual
> Studio\2022\Enterprise\Common7\IDE\VC\VCPackages;C:\Program Files\Microsoft
> Visual
> Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\Program
> Files\Microsoft Visual
> Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team
> Explorer;C:\Program Files\Microsoft Visual
> Studio\2022\Enterprise\MSBuild\Current\bin\Roslyn;C:\Program
> Files\Microsoft Visual Studio\2022\Enterprise\Team Tools\Performance
> Tools;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8
> Tools\;C:\Program Files (x86)\Windows
> Kits\10\bin\10.0.19041.0\\x86;C:\Program Files (x86)\Windows
> Kits\10\bin\\x86;C:\Program Files\Microsoft Visual
> Studio\2022\Enterprise\\MSBuild\Current\Bin\amd64;C:\Windows\Microsoft.NET\Framework\v4.0.30319;C:\Program
> Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\;C:\Program
> Files\Microsoft Visual
> Studio\2022\Enterprise\Common7\Tools\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program
> Files\Microsoft SQL Server\150\Tools\Binn\;C:\Program Files\IDM Computer
> Solutions\UltraEdit;C:\Program Files\IDM Computer
> Solutions\UltraCompare;C:\Program Files\Git\cmd;C:\Program
> Files\dotnet\;C:\Program
> Files\TortoiseGit\bin;C:\Users\archyizimi\AppData\Local\Microsoft\WindowsApps;C:\Users\archyizimi\.dotnet\tools;C:\Program
> Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\Llvm\bin;C:\Program
> Files\Microsoft Visual
> Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;C:\Program
> Files\Microsoft Visual
> Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja;C:\Program
> Files\Microsoft Visual
> Studio\2022\Enterprise\Common7\IDE\VC\Linux\bin\ConnectionManagerExe;c:\MinGW\bin
>
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
> PATHEXT=.COM;.EXE;.BAT;.CMD
>
> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>D:\MYBIN\Anaconda3\lib\site-packages\ninja\data\bin\ninja.exe
> -f D:\work\gitee\rt-thread_ninja\bsp\beaglebone\build.ninja
>
> [1/56] Starting scons daemon...[2K
> [2/56] Defer to SCons to build build\__link_objects.txt[2K
> [3/56] Defer to SCons to build build\__cpp_path.txt[2K
> [4/56] Defer to SCons to build build\__define_options.txt[2Kscons: ***
> [build.ninja] UnicodeDecodeError : 'gbk' codec can't decode byte 0xaf in
> position 218: illegal multibyte sequence
>
> [5/56] Compiling build\drivers\gpio.oscons: building terminated because of
> errors.
> Traceback (most recent call last):
>   File "D:\MYBIN\scons-local\scons-local-4.3.1\SCons\Action.py", line
> 1295, in execute
>     result = self.execfunction(target=target, source=rsources, env=env)
>   File
> "D:\MYBIN\scons-local\scons-local-4.3.1\SCons\Tool\ninja\__init__.py", line
> 112, in ninja_builder
>     for output in execute_ninja():
>   File
> "D:\MYBIN\scons-local\scons-local-4.3.1\SCons\Tool\ninja\__init__.py", line
> 104, in execute_ninja
>     for stdout_line in iter(proc.stdout.readline, ""):
> UnicodeDecodeError: 'gbk' codec can't decode byte 0xaf in position 218:
> illegal multibyte sequence
> >>>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "SCons users mailing list" <[email protected]>;
> *发送时间:* 2022年4月27日(星期三) 凌晨2:34
> *收件人:* "SCons users mailing list"<[email protected]>;
> *主题:* Re: [Scons-users] UnicodeDecodeError: 'gbk' codec can't decode byte
> 0xa3 in position 3242: illegal multibyte sequence
>
> On windows you should use the run_ninja_env.bat to ensure that your
> environment is setup as the ninja file itself is unable to correctly setup
> windows cmdshells.
>
> On Tue, Apr 26, 2022 at 1:33 PM Daniel Moody <[email protected]> wrote:
>
>> As a work around you can run ninja directly. This particular issue is
>> scons reading and reprinting the ninja subprocess stdout. I'll see if I can
>> reproduce and get a fix.
>>
>> On Tue, Apr 26, 2022 at 1:11 PM Mats Wichmann <[email protected]> wrote:
>>
>>> On 4/26/22 06:10, liruncong2018 via Scons-users wrote:
>>> >         Hi,
>>> >
>>> >         The following log appears when compiling with ninja, is there a
>>> >         problem with the encoding process?(ninja\__init__.py)
>>> >
>>> >         ···
>>> >
>>> >                 [1/2518] Starting scons daemon... [2K
>>> >
>>> >                 [2/2518] Defer to SCons to build build\__cpp_path.txt
>>> [2K
>>> >
>>> >                 [3/2518] Defer to SCons to build
>>> >                 build\__exe_cpp_path.txt [2K
>>> >
>>> >                 [4/2518] Defer to SCons to build
>>> >                 build\__link_objects.txt [2K
>>> >
>>> >                 [5/2518] Defer to SCons to build
>>> >                 build\__define_options.txt [2Kscons: *** [build.ninja]
>>> >                 UnicodeDecodeError : 'gbk' codec can't decode byte 0xa3
>>> >                 in position 3299: illegal multibyte sequence
>>> >
>>> >                 Traceback (most recent call last):
>>> >
>>> >                   File
>>> >
>>>  "D:\MYBIN\scons-local\scons-local-4.3.1\SCons\Action.py", line
>>> >                 1295, in execute
>>> >
>>> >                     result = self.execfunction(target=target,
>>> >                 source=rsources, env=env)
>>> >
>>> >                   File
>>> >
>>>  "D:\MYBIN\scons-local\scons-local-4.3.1\SCons\Tool\ninja\__init__.py",
>>> >                 line 112, in ninja_builder
>>> >
>>> >                     for output in execute_ninja():
>>> >
>>> >                   File
>>> >
>>>  "D:\MYBIN\scons-local\scons-local-4.3.1\SCons\Tool\ninja\__init__.py",
>>> >                 line 104, in execute_ninja
>>> >
>>> >                     for stdout_line in iter(proc.stdout.readline, ""):
>>> >
>>> >                 UnicodeDecodeError: 'gbk' codec can't decode byte 0xa3
>>> >                 in position 3299: illegal multibyte sequence
>>> >
>>>
>>> that error usually happens when the system codec isn't what we US-based
>>> folk expect, and we haven't specified an encoding for a file/stream when
>>> reading in text mode so the system codec gets used.
>>>
>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> [email protected]
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>

_______________________________________________
Scons-users mailing list
[email protected]
https://pairlist4.pair.net/mailman/listinfo/scons-users