Re: Python 3 snmp set operations fail with octet strings with values >= 0x80
Bill Fenner <[email protected]> Fri, 3 Feb 2023 10:54:19 -0500
| Newsgroups | gmane.network.net-snmp.user |
|---|---|
| Message-ID | <CAF4SogazJGK-njY96ROQWYQoRQ=26WFJqg0UFoz9WterO+Ke-g@mail.gmail.com> |
--===============4217302200166899301== Content-Type: multipart/alternative; boundary="0000000000006f118405f3cdb20b" --0000000000006f118405f3cdb20b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The API has to deal with both DisplayString (underlying type OCTETSTR) and actual octet strings. It makes sense for a DisplayString to be utf-8 encoded, but, it doesn't make sense for actual octet strings. Python 3 does have two different types that can very usefully be used for this: a string could be treated as being utf-8 encoded, assuming it's a DisplayString, and a bytes value could be treated as a series of bytes. Unfortunately, get() would have to return bytes only, as consulting the MIB is the only way to tell whether a given incoming OCTETSTR is a DisplayString or not. I don't have cycles to work on this, unfortunately. Bill On Thu, Feb 2, 2023 at 4:10 PM Steve Hollywood <[email protected]= z> wrote: > Net-SNMP v5.9.3 with python 3.6.3. > > > > I=E2=80=99m not sure if this has only become an issue since the move of s= trings > from bytes to Unicode in Python 3 but I cannot set an OID value to an oct= et > string containing octets >=3D128. > > I am trying to set a MAC address in an OID. > > > > macAddress =3D '\x02\0\0\0\x80\xff' > > varbind =3D netsnmp.Varbind(oidSvcMacNameAddr, iid, macAddress, > 'OCTETSTR') > > > > The python bindings convert everything to strings in the Varbind object. > > > > class Varbind(object): > > def __init__(self, tag=3DNone, iid=3DNone, val=3DNone, type_arg=3DNon= e): > > self.tag =3D STR(tag) > > self.iid =3D STR(iid) > > self.val =3D STR(val) > > self.type =3D STR(type_arg) > > > > The result on the wire of the performing an SNMP set on using the values > above is: > > > > variable-bindings: 2 items > > > TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.4= 9 > (1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49): > 02000000c280c3bf > > Object Name: > 1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49 > (TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.= 49) > > Value (OctetString): 02000000c280c3bf > > > TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.4= 9 > (1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49): > 020000007f55 > > Object Name: > 1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49 > (TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.= 49) > > Value (OctetString): 020000007f55 > > > > As a result an error is returned because the octet string is 8 bytes > rather than 6. > > Note: Setting values <0x80 works without issue. The second variable > binding is using a macAddress =3D =E2=80=98\x02\0\0\0\x7f\x55=E2=80=99 is= correct and when > used without the 1st binding sets the value. > > > > Within the library bindings, the val attribute is read using the function > py_netsnmp_attr_string, which calls PyUnicode_AsUTF8AndSize function > which returns a UTF8 encoded char buffer. The value sent on the wire is t= he > UTF-8 encoded string. > > > > >>> macAddress =3D '\x02\0\0\0\x80\xff' > > >>> macAddress > > '\x02\x00\x00\x00\x80=C3=BF' > > >>> macAddress.encode('utf8') > > b'\x02\x00\x00\x00\xc2\x80\xc3\xbf' > > > > UTF8 encoding is probably a poor choice for encoding octet strings. I > believe that if the string was encoded as latin-1 (which is the same as > iso-8859-1) within the bindings the problem wouldn=E2=80=99t occur. > > > > >>> macAddress.encode('latin-1') > > b'\x02\x00\x00\x00\x80\xff' > > > > Does anyone have any idea of how to cleverly work around this or failing > that patch the bindings? > > > The content of this email (including any attachments) is intended for the > addressee only, is confidential and may be legally privileged. If you=E2= =80=99ve > received this email in error, you shouldn=E2=80=99t read it - please cont= act me > immediately, destroy it, and do not copy or use any of the content of thi= s > email . No confidentiality or privilege is waived or lost by any > mis-transmission or error. This communication does not designate an > information system for the purposes of Part 4 of the Contract and > Commercial Law Act 2017. Although we have taken reasonable precautions to > ensure no viruses are present in this email, we cannot accept > responsibility for any loss or damage arising from the use of this email = or > its attachments. > _______________________________________________ > Net-snmp-users mailing list > [email protected] > Please see the following page to unsubscribe or change other options: > https://lists.sourceforge.net/lists/listinfo/net-snmp-users > --0000000000006f118405f3cdb20b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">The API has to deal with both DisplayString (underlying ty= pe OCTETSTR) and actual octet strings.=C2=A0 It makes sense for a DisplaySt= ring to be utf-8 encoded, but, it doesn't make sense for actual octet s= trings.<div><br></div><div>Python 3 does have two different types that can = very usefully be used for this: a string could be treated as being utf-8 en= coded, assuming it's a DisplayString, and a bytes value could be treate= d as a series of bytes.</div><div><br></div><div>Unfortunately, get() would= have to return bytes only, as consulting the MIB is the only way to tell w= hether a given incoming OCTETSTR is a DisplayString or not.</div><div><br><= /div><div>I don't have cycles to work on this, unfortunately.</div><div= ><br></div><div>=C2=A0 Bill</div><div><br></div></div><br><div class=3D"gma= il_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Thu, Feb 2, 2023 at 4:10= PM Steve Hollywood <<a href=3D"mailto:[email protected]">Ste= [email protected]</a>> wrote:<br></div><blockquote class=3D"gmai= l_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,20= 4,204);padding-left:1ex"><div class=3D"msg-4573603972108979472"> <div lang=3D"EN-NZ" style=3D"overflow-wrap: break-word;"> <div class=3D"m_-9042484721725263771WordSection1"> <p class=3D"MsoNormal">Net-SNMP v5.9.3 with python 3.6.3.<u></u><u></u></p> <p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p> <p class=3D"MsoNormal">I=E2=80=99m not sure if this has only become an issu= e since the move of strings from bytes to Unicode in Python 3 but I cannot = set an OID value to an octet string containing octets >=3D128.<u></u><u>= </u></p> <p class=3D"MsoNormal">I am trying to set a MAC address in an OID.<u></u><u= ></u></p> <p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p> <p class=3D"MsoNormal">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 macAddres= s =3D '\x02\0\0\0\x80\xff'<u></u><u></u></p> <p class=3D"MsoNormal">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 varbind = =3D netsnmp.Varbind(oidSvcMacNameAddr, iid, macAddress, 'OCTETSTR')= <u></u><u></u></p> <p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p> <p class=3D"MsoNormal">The python bindings convert everything to strings in= the Varbind object. <u></u><u></u></p> <p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p> <p class=3D"MsoNormal"><span style=3D"font-size:10pt;font-family:"Cour= ier New"">class Varbind(object):<u></u><u></u></span></p> <p class=3D"m_-9042484721725263771Code">=C2=A0=C2=A0=C2=A0 def __init__(sel= f, tag=3DNone, iid=3DNone, val=3DNone, type_arg=3DNone):<span style=3D"font= -size:10pt"><u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:10pt;font-family:"Cour= ier New"">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 self.tag =3D STR(= tag)<u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:10pt;font-family:"Cour= ier New"">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 self.iid =3D STR(= iid)<u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:10pt;font-family:"Cour= ier New"">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 self.val =3D STR(= val)<u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:10pt;font-family:"Cour= ier New"">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 self.type =3D STR= (type_arg)<u></u><u></u></span></p> <p class=3D"MsoNormal"><u></u>=C2=A0<u></u></p> <p class=3D"MsoNormal"><span>The result on the wire of the performing an SN= MP set on using the values above is:<u></u><u></u></span></p> <p class=3D"MsoNormal"><span><u></u>=C2=A0<u></u></span></p> <p class=3D"MsoNormal"><span>variable-bindings: 2 items<u></u><u></u></span= ></p> <p class=3D"MsoNormal"><span>=C2=A0=C2=A0=C2=A0 TIMETRA-GLOBAL-MIB::tmnxSRO= bjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49 (1.3.6.1.4.1.6527.3.1.2.4.2= .59.1.4.10.83.84.69.45.80.79.76.84.48.49): 02000000c280c3bf<u></u><u></u></= span></p> <p class=3D"MsoNormal"><span>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Obj= ect Name: 1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.4= 9 (TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.= 49)<u></u><u></u></span></p> <p class=3D"MsoNormal"><span>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Val= ue (OctetString): 02000000c280c3bf<u></u><u></u></span></p> <p class=3D"MsoNormal"><span>=C2=A0=C2=A0=C2=A0 TIMETRA-GLOBAL-MIB::tmnxSRO= bjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49 (1.3.6.1.4.1.6527.3.1.2.4.2= .59.1.4.10.83.84.69.45.80.79.76.84.48.49): 020000007f55<u></u><u></u></span= ></p> <p class=3D"MsoNormal"><span>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Obj= ect Name: 1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.4= 9 (TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.= 49)<u></u><u></u></span></p> <p class=3D"MsoNormal"><span>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Val= ue (OctetString): 020000007f55<u></u><u></u></span></p> <p class=3D"MsoNormal"><span><u></u>=C2=A0<u></u></span></p> <p class=3D"MsoNormal"><span>As a result an error is returned because the o= ctet string is 8 bytes rather than 6. <u></u><u></u></span></p> <p class=3D"MsoNormal"><span>Note: Setting values <0x80 works without is= sue. The second variable binding is using a macAddress =3D =E2=80=98\x02\0\= 0\0\x7f\x55=E2=80=99 is correct and when used without the 1<sup>st</sup> bi= nding sets the value.<u></u><u></u></span></p> <p class=3D"MsoNormal"><span><u></u>=C2=A0<u></u></span></p> <p class=3D"MsoNormal"><span>Within the library bindings, the val attribute= is read using the function </span>py_netsnmp_attr_string<span>, which calls PyUnicode_AsUTF8AndSize fu= nction which returns a UTF8 encoded char buffer. The value sent on the wire= is the UTF-8 encoded string.<u></u><u></u></span></p> <p class=3D"MsoNormal"><span><u></u>=C2=A0<u></u></span></p> <p class=3D"MsoNormal"><span>>>> macAddress =3D '\x02\0\0\0\x8= 0\xff'<u></u><u></u></span></p> <p class=3D"MsoNormal"><span>>>> macAddress<u></u><u></u></span></= p> <p class=3D"MsoNormal"><span>'\x02\x00\x00\x00\x80=C3=BF'<u></u><u>= </u></span></p> <p class=3D"MsoNormal"><span>>>> macAddress.encode('utf8')= <u></u><u></u></span></p> <p class=3D"MsoNormal"><span>b'\x02\x00\x00\x00\xc2\x80\xc3\xbf'<u>= </u><u></u></span></p> <p class=3D"MsoNormal"><span><u></u>=C2=A0<u></u></span></p> <p class=3D"MsoNormal"><span>UTF8 encoding is probably a poor choice for en= coding octet strings. I believe that if the string was encoded as latin-1 (= which is the same as iso-8859-1) within the bindings the problem wouldn=E2= =80=99t occur.<u></u><u></u></span></p> <p class=3D"MsoNormal"><span><u></u>=C2=A0<u></u></span></p> <p class=3D"MsoNormal"><span>>>> macAddress.encode('latin-1= 9;)<u></u><u></u></span></p> <p class=3D"MsoNormal"><span>b'\x02\x00\x00\x00\x80\xff'<u></u><u><= /u></span></p> <p class=3D"MsoNormal"><span><u></u>=C2=A0<u></u></span></p> <p class=3D"MsoNormal"><span>Does anyone have any idea of how to cleverly w= ork around this or failing that patch the bindings?<u></u><u></u></span></p= > <p class=3D"MsoNormal"><span><u></u>=C2=A0<u></u></span></p> </div> The content of this email (including any attachments) is intended for the a= ddressee only, is confidential and may be legally privileged. If you=E2=80= =99ve received this email in error, you shouldn=E2=80=99t read it - please = contact me immediately, destroy it, and do not copy or use any of the content of this email . No confidentiality or privilege = is waived or lost by any mis-transmission or error. This communication does= not designate an information system for the purposes of Part 4 of the Cont= ract and Commercial Law Act 2017. Although we have taken reasonable precautions to ensure no viruses are pre= sent in this email, we cannot accept responsibility for any loss or damage = arising from the use of this email or its attachments. </div> _______________________________________________<br> Net-snmp-users mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">N= [email protected]</a><br> Please see the following page to unsubscribe or change other options:<br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/net-snmp-users" rel= =3D"noreferrer" target=3D"_blank">https://lists.sourceforge.net/lists/listi= nfo/net-snmp-users</a><br> </div></blockquote></div> --0000000000006f118405f3cdb20b-- --===============4217302200166899301== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============4217302200166899301== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Net-snmp-users mailing list [email protected] Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users --===============4217302200166899301==--