Re: "Rehabilitating Pickle" a talk at PyCon UK 2018

--0000000000001c5ea80580381c77
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I use the ZODB as file format for a application that is used by end users.
So telling them "don't open files from untrusted sources" seems a bit weird
:)

I just looked at the ZEO code and think i found the relevant code sections
where unpickling is restricted. Could you please give me a hint where i
should look in the ZODB code and what i have to wrap exactly to use a
restricted Unpickler?

Best regards,
J=C3=BCrgen

Am Do., 24. Jan. 2019 um 15:04 Uhr schrieb Jim Fulton <[email protected]>:

>
>
> On Wed, Jan 23, 2019 at 3:00 PM 'Juergen Herrmann' via zodb <
> [email protected]> wrote:
>
>> What would be necessary to limit pickling in ZODB to read only
>> specific/whitelisted classes?
>>
>
> This could be done with a storage wrapper.
>
> There is a pickle hook for getting global objects that can be used to
> limit what pickle is willing to call. For example, ZEO does this at the
> protocol level avoid accepting all but a limited number of globals in
> pickles in it's wire protocol. Pickle is really way more powerful than ZE=
O
> needs and I wouldn't use it today and ZEO recently got support for msgpac=
k.
>
>
>> I chose ZODB as the file storage format for my speaker crossover program
>> and wrote a very thin wrapper around ZODB which puts a FileStorage .fs i=
nto
>> the tmp dir, opens it and copies it back when saving. Works great but no=
w
>> that I hear that pickles are inherently unsafe I wonder how I can mitiga=
te
>> this.
>>
>
> They aren't inherently unsafe.  By default, you shouldn't accept pickles
> from untrusted sources. Are you doing that?  If not, you have nothing to
> worry about.
>
> Jim
>
>
>
>
>
>>
>> Best regards,
>> J=C3=BCrgen
>>
>> Am Mi., 16. Jan. 2019 um 11:33 Uhr schrieb Christopher Lozinski <
>> [email protected]>:
>>
>>> I found this talk quite interesting.
>>>
>>> 9 minutes ago
>>> <https://twitter.com/PythonLinks/status/1085482554000580608>
>>>
>>> Pickle is a compact serialization protocol for Python objects. Great fo=
r
>>> communication between #distributed
>>> <https://twitter.com/hashtag/distributed?src=3Dhash> #Python
>>> <https://twitter.com/hashtag/Python?src=3Dhash> programs, but it is not
>>> safe. What can be done?
>>>
>>> https://pythonlinks.info/rehabilitating-pickle
>>>
>>>
>>> Warm Regards
>>> Chris
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "zodb" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> You received this message because you are subscribed to the Google Group=
s
>> "zodb" group.
>> To unsubscribe from this group and stop receiving emails from it, send a=
n
>> email to [email protected].
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> --
> Jim Fulton
> http://jimfulton.info
>

--=20
You received this message because you are subscribed to the Google Groups "=
zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to [email protected].
For more options, visit https://groups.google.com/d/optout.

--0000000000001c5ea80580381c77
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I use the ZODB as file format for a application that is us=
ed by end users. So telling them &quot;don&#39;t open files from untrusted =
sources&quot; seems a bit weird :)<div><br></div><div>I just looked at the =
ZEO code and think i found the relevant code sections where unpickling is r=
estricted. Could you please give me a hint where i should look in the ZODB =
code and what i have to wrap exactly to use a restricted Unpickler?</div><d=
iv><br></div><div>Best regards,</div><div>J=C3=BCrgen</div></div><br><div c=
lass=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">Am Do., 24. Jan.=
 2019 um 15:04=C2=A0Uhr schrieb Jim Fulton &lt;<a href=3D"mailto:jim@jimful=
ton.info">[email protected]</a>&gt;:<br></div><blockquote class=3D"gmail_q=
uote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,2=
04);padding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div =
class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail-m_-731955051368408545=
6gmail_attr">On Wed, Jan 23, 2019 at 3:00 PM &#39;Juergen Herrmann&#39; via=
 zodb &lt;<a href=3D"mailto:[email protected]" target=3D"_blank">zodb@g=
ooglegroups.com</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" s=
tyle=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pad=
ding-left:1ex"><div dir=3D"ltr">What would be necessary to limit pickling i=
n ZODB to read only specific/whitelisted classes?</div></blockquote><div><b=
r></div><div>This could be done with a storage wrapper.=C2=A0</div><div><br=
></div><div>There is a pickle hook for getting global objects that can be u=
sed to limit what pickle is willing to call. For example, ZEO does this at =
the protocol level avoid accepting all but a limited number of globals in p=
ickles in it&#39;s wire protocol. Pickle is really way more powerful than Z=
EO needs and I wouldn&#39;t use it today and ZEO recently got support for m=
sgpack.</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"ma=
rgin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:=
1ex"><div dir=3D"ltr"><div>I chose ZODB as the file storage format for my s=
peaker crossover program and wrote a very thin wrapper around ZODB which pu=
ts a FileStorage .fs into the tmp dir, opens it and copies it back when sav=
ing. Works great but now that I hear that pickles are inherently unsafe I w=
onder how I can mitigate this.<br></div></div></blockquote><div><br></div><=
div>They aren&#39;t inherently unsafe.=C2=A0 By default, you shouldn&#39;t =
accept pickles from untrusted sources. Are you doing that?=C2=A0 If not, yo=
u have nothing=C2=A0to worry about.</div><div></div><div><br></div><div>Jim=
</div><div><br></div><div><br></div><div><br></div><div>=C2=A0</div><blockq=
uote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1p=
x solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div></div><div=
><br></div><div>Best regards,</div><div>J=C3=BCrgen</div></div><br><div cla=
ss=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail-m_-7319550513684085456gm=
ail-m_-7916493716034677636gmail_attr">Am Mi., 16. Jan. 2019 um 11:33=C2=A0U=
hr schrieb Christopher Lozinski &lt;<a href=3D"mailto:lozinski@specialtyjob=
markets.com" target=3D"_blank">[email protected]</a>&gt;:<br=
></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;=
border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div>I found =
this talk quite interesting. =C2=A0</div><div><br></div><div><div class=3D"=
gmail-m_-7319550513684085456gmail-m_-7916493716034677636gmail-m_-1581236281=
649401542stream-item-header" style=3D"display:flex;color:rgb(20,23,26);font=
-family:&quot;Helvetica Neue&quot;,Helvetica,Arial,sans-serif;font-size:14p=
x;font-variant-ligatures:normal;background-color:rgb(255,255,255)"><small c=
lass=3D"gmail-m_-7319550513684085456gmail-m_-7916493716034677636gmail-m_-15=
81236281649401542time" style=3D"font-size:14px;color:rgb(101,119,134);margi=
n-right:5px;white-space:nowrap"><span class=3D"gmail-m_-7319550513684085456=
gmail-m_-7916493716034677636gmail-m_-1581236281649401542u-hiddenVisually" s=
tyle=3D"background-color:transparent;color:rgb(101,119,134);text-decoration=
:none;border:0px;font-size:1px;height:1px;overflow:hidden;padding:0px;width=
:1px"><a href=3D"https://twitter.com/PythonLinks/status/1085482554000580608=
" class=3D"gmail-m_-7319550513684085456gmail-m_-7916493716034677636gmail-m_=
-1581236281649401542js-nav gmail-m_-7319550513684085456gmail-m_-79164937160=
34677636gmail-m_-1581236281649401542tweet-timestamp gmail-m_-73195505136840=
85456gmail-m_-7916493716034677636gmail-m_-1581236281649401542js-permalink g=
mail-m_-7319550513684085456gmail-m_-7916493716034677636gmail-m_-15812362816=
49401542js-tooltip" title=3D"2:23 AM - 16 Jan 2019" style=3D"background-col=
or:transparent;color:rgb(101,119,134);text-decoration:none" target=3D"_blan=
k">9 minutes ago</a></span></small></div><div class=3D"gmail-m_-73195505136=
84085456gmail-m_-7916493716034677636gmail-m_-1581236281649401542js-tweet-te=
xt-container" style=3D"color:rgb(20,23,26);font-family:&quot;Helvetica Neue=
&quot;,Helvetica,Arial,sans-serif;font-size:14px;font-variant-ligatures:nor=
mal;background-color:rgb(255,255,255)"><p class=3D"gmail-m_-731955051368408=
5456gmail-m_-7916493716034677636gmail-m_-1581236281649401542TweetTextSize g=
mail-m_-7319550513684085456gmail-m_-7916493716034677636gmail-m_-15812362816=
49401542tweet-text gmail-m_-7319550513684085456gmail-m_-7916493716034677636=
gmail-m_-1581236281649401542js-tweet-text" lang=3D"en" style=3D"margin:0px =
0px 8px;line-height:20px;white-space:pre-wrap">Pickle is a compact serializ=
ation protocol for Python objects. Great for communication between <a href=
=3D"https://twitter.com/hashtag/distributed?src=3Dhash" class=3D"gmail-m_-7=
319550513684085456gmail-m_-7916493716034677636gmail-m_-1581236281649401542t=
witter-hashtag gmail-m_-7319550513684085456gmail-m_-7916493716034677636gmai=
l-m_-1581236281649401542js-nav gmail-m_-7319550513684085456gmail-m_-7916493=
716034677636gmail-m_-1581236281649401542pretty-link" dir=3D"ltr" style=3D"b=
ackground:transparent;color:rgb(27,149,224);text-decoration:none" target=3D=
"_blank">#distributed</a> <a href=3D"https://twitter.com/hashtag/Python?src=
=3Dhash" class=3D"gmail-m_-7319550513684085456gmail-m_-7916493716034677636g=
mail-m_-1581236281649401542twitter-hashtag gmail-m_-7319550513684085456gmai=
l-m_-7916493716034677636gmail-m_-1581236281649401542js-nav gmail-m_-7319550=
513684085456gmail-m_-7916493716034677636gmail-m_-1581236281649401542pretty-=
link" dir=3D"ltr" style=3D"background:transparent;color:rgb(27,149,224);tex=
t-decoration:none" target=3D"_blank">#Python</a> programs, but it is not sa=
fe.=20
What can be done?</p></div></div><div><br></div><div><a href=3D"https://pyt=
honlinks.info/rehabilitating-pickle" target=3D"_blank">https://pythonlinks.=
info/rehabilitating-pickle</a></div><div><br></div><div><br></div><div>Warm=
 Regards</div><div>Chris</div></div>

<p></p>

-- <br>
You received this message because you are subscribed to the Google Groups &=
quot;zodb&quot; group.<br>
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:[email protected]" target=3D"_bla=
nk">[email protected]</a>.<br>
For more options, visit <a href=3D"https://groups.google.com/d/optout" targ=
et=3D"_blank">https://groups.google.com/d/optout</a>.<br>
</blockquote></div>

<p></p>

-- <br>
You received this message because you are subscribed to the Google Groups &=
quot;zodb&quot; group.<br>
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:[email protected]" target=3D"_bla=
nk">[email protected]</a>.<br>
For more options, visit <a href=3D"https://groups.google.com/d/optout" targ=
et=3D"_blank">https://groups.google.com/d/optout</a>.<br>
</blockquote></div><br clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr"=
 class=3D"gmail-m_-7319550513684085456gmail_signature">Jim Fulton<br><a hre=
f=3D"http://jimfulton.info" target=3D"_blank">http://jimfulton.info</a><br>=
</div></div>
</blockquote></div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;zodb&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:[email protected]">zodb+unsubscri=
[email protected]</a>.<br />
For more options, visit <a href=3D"https://groups.google.com/d/optout">http=
s://groups.google.com/d/optout</a>.<br />

--0000000000001c5ea80580381c77--