Re: Can't have access to a corrupted VARCHAR2.
Anthony Tuininga <[email protected]> Mon, 19 Mar 2018 15:44:27 -0600
| Newsgroups | gmane.comp.python.db.cx-oracle |
|---|---|
| Message-ID | <CAE1XR-517gVUw3AMjx487qE5t=fky_QdJitT4tOSaPsZC-EarQ@mail.gmail.com> |
--===============8182745403462265882==
Content-Type: multipart/alternative; boundary="94eb2c059220c0ad7f0567cadcff"
--94eb2c059220c0ad7f0567cadcff
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Hi,
As Walter mentioned, you can set the environment variable NLS_LANG or you
can set the encoding parameter when you create the connection. The latter
is usually preferable and easier to use. For example:
cx_Oracle.connect("user/pw@dsn", encoding =3D "WINDOWS-1252", nencoding =3D
"UTF-8")
This will ensure that all data is encoded using the actual database
character set. If you use something different you do run the risk of
getting encoding errors, and currently there is no way in cx_Oracle to
specify the "errors" parameter to the decode() that takes place internally.
That could be an enhancement request. Feel free to log such a request on
the issues page: https://github.com/oracle/python-cx_Oracle/issues. It
might also be useful to include information on the character sets in use in
your database by posting the results of this query:
select * from nls_database_parameters where parameter in
('NLS_CHARACTERSET', 'NLS_NCHAR_CHARACTERSET');
Anthony
On Fri, Mar 16, 2018 at 7:58 AM, St=C3=A9phane JULIEN <stephane-W5t6FLjSki/[email protected]> =
wrote:
> Hello,
>
> I have a WINDOWS-1252 (cp1252) encoded Oracle database. A 6.2.1 cx_Oracle
> and a Python 3.6.4 distribution on Wondows.
>
> In some VARCHAR2 record of a table, I have corrupted data. When I try to
> select such records, I have :
>
> 'charmap' codec can't decode byte 0x90 in position 36: character maps to
> <undefi
> ned>
> (<class 'UnicodeDecodeError'>, UnicodeDecodeError('charmap', b'ELLE
> REVIENDRA VE
> RS NOUS - BIEN PLAC*\x90*\xc9S - ELLE ATTEND SON CLIENT', 36, 37,
> 'character maps
> to <undefined>'), <traceback object at 0x0000000003C6B5C8>)
>
> The \x90 character is not allowed in cp1252 encoding. Where cx_Oracle
> tries to convert to unicode, there is an exception. I suppose that a stri=
ct
> decode is used.
>
> I can't find a way to prevent or detect the crash. I can't event find a
> way to detect the key of the corrupted record.
>
> Is there a way to do a *decode(..., 'IGNORE')* in my Python Program or to
> receive the binary data like in Python 2 ?
>
> I thank you for your help.
>
> Best wishes.
> St=C3=A9phane
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> cx-oracle-users mailing list
> cx-oracle-users-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
>
>
--94eb2c059220c0ad7f0567cadcff
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">Hi,<div><br></div><div>As Walter mentioned, you can set th=
e environment variable NLS_LANG or you can set the encoding parameter when =
you create the connection. The latter is usually preferable and easier to u=
se. For example:</div><div><br></div><div>cx_Oracle.connect("user/pw@d=
sn", encoding =3D "WINDOWS-1252", nencoding =3D "UTF-8&=
quot;)</div><div><br></div><div>This will ensure that all data is encoded u=
sing the actual database character set. If you use something different you =
do run the risk of getting encoding errors, and currently there is no way i=
n cx_Oracle to specify the "errors" parameter to the decode() tha=
t takes place internally. That could be an enhancement request. Feel free t=
o log such a request on the issues page:=C2=A0<a href=3D"https://github.com=
/oracle/python-cx_Oracle/issues">https://github.com/oracle/python-cx_Oracle=
/issues</a>. It might also be useful to include information on the characte=
r sets in use in your database by posting the results of this query:</div><=
div><br></div><div>select * from nls_database_parameters where parameter in=
('NLS_CHARACTERSET', 'NLS_NCHAR_CHARACTERSET');</div><div>=
<br></div><div>Anthony</div></div><div class=3D"gmail_extra"><br><div class=
=3D"gmail_quote">On Fri, Mar 16, 2018 at 7:58 AM, St=C3=A9phane JULIEN <spa=
n dir=3D"ltr"><<a href=3D"mailto:stephane-W5t6FLjSki/[email protected]" target=3D"_blank">s=
tephane-W5t6FLjSki/[email protected]</a>></span> wrote:<br><blockquote class=3D"gmail_quote=
" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><=
div style=3D"font-size:10pt;font-family:Verdana,Geneva,sans-serif">
<p>Hello,</p>
<p>I have a WINDOWS-1252 (cp1252) encoded Oracle database. A 6.2.1 cx_Oracl=
e and a Python 3.6.4 distribution on Wondows.</p>
<p>In some VARCHAR2 record of a table, I have corrupted data. When I try to=
select such records, I have :</p>
<p>'charmap' codec can't decode byte 0x90 in position 36: chara=
cter maps to <undefi<br>ned><br>(<class 'UnicodeDecodeError=
9;>, UnicodeDecodeError('charmap', b'ELLE REVIENDRA VE<br>RS=
NOUS - BIEN PLAC<strong>\x90</strong>\xc9S - ELLE ATTEND SON CLIENT', =
36, 37, 'character maps<br>to <undefined>'), <traceback ob=
ject at 0x0000000003C6B5C8>)</p>
<p>The \x90 character is not allowed in cp1252 encoding. Where cx_Oracle tr=
ies to convert to unicode, there is an exception. I suppose that a strict d=
ecode is used.</p>
<p>I can't find a way to prevent or detect the crash. I can't event=
find a way to detect the key of the corrupted record.</p>
<p>Is there a way to do a <strong>decode(..., 'IGNORE')</strong> in=
my Python Program or to receive the binary data like in Python 2 ?</p>
<p>I thank you for your help.</p>
<p>Best wishes.</p><span class=3D"HOEnZb"><font color=3D"#888888">
<div>
<div class=3D"m_63699978946222938pre" style=3D"margin:0;padding:0;font-fami=
ly:monospace">St=C3=A9phane<br><br></div>
</div>
</font></span></div>
<br>------------------------------<wbr>------------------------------<wbr>-=
-----------------<br>
Check out the vibrant tech community on one of the world's most<br>
engaging tech sites, Slashdot.org! <a href=3D"http://sdm.link/slashdot" rel=
=3D"noreferrer" target=3D"_blank">http://sdm.link/slashdot</a><br>_________=
_____________________<wbr>_________________<br>
cx-oracle-users mailing list<br>
<a href=3D"mailto:cx-oracle-users-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org">cx-oracle-users@li=
sts.<wbr>sourceforge.net</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/cx-oracle-users" re=
l=3D"noreferrer" target=3D"_blank">https://lists.sourceforge.net/<wbr>lists=
/listinfo/cx-oracle-users</a><br>
<br></blockquote></div><br></div>
--94eb2c059220c0ad7f0567cadcff--
--===============8182745403462265882==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--===============8182745403462265882==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
cx-oracle-users mailing list
cx-oracle-users-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/cx-oracle-users
--===============8182745403462265882==--