Re: notarization works! (was Re: not that)
Glyph <[email protected]> Mon, 29 Oct 2018 10:22:05 -0700
| Newsgroups | gmane.comp.python.pyobjc.devel,gmane.comp.python.apple |
|---|---|
| Message-ID | <1540833725.1740834.1558553816.5E07E4B9@webmail.messagingengine.com> |
This is a multi-part message in MIME format. --===============2708377343312227577== Content-Transfer-Encoding: 7bit Content-Type: multipart/alternative; boundary="_----------=_154083372517408340" This is a multi-part message in MIME format. --_----------=_154083372517408340 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" On Sun, Oct 28, 2018, at 11:58 PM, Ronald Oussoren wrote: >=20 >=20 >> On 29 Oct 2018, at 00:56, Glyph <[email protected]> wrote: >>=20 >>=20 >>=20 >>> On Oct 28, 2018, at 2:57 PM, Glyph <[email protected]> wrote:>>>= =20 >>>> I wonder what the =E2=80=9Chardened runtime=E2=80=9D option actually d= oes and >>>> enforces. In 3.7 the line in ctypes/__init__.py that causes the >>>> exception is a call that creates a dummy C function, and likely >>>> triggers the first allocation for storing a libffi closure which >>>> could be something the hardened runtime doesn=E2=80=99t like (being >>>> writeable + executable memory).>>>=20 >>> Interesting. Perhaps what I want is simply >>> https://developer.apple.com/documentation/security/com_apple_security_c= s_allow-unsigned-executable-memory >>> then? Any chance you know how to jam that into a `codesign` command >>> line somehow? :-)>>>=20 >>=20 >> Thank you so much for this tip, Ronald! This was much easier than I >> anticipated, and things are working now!>=20 > Great. >=20 >>=20 >> The relevant entitlements file is literally just: >>=20 >>> <?xml version=3D"1.0" encoding=3D"UTF-8"?> >>> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" " >>> http://www.apple.com/DTDs/PropertyList-1.0.dtd">>>> <plist version=3D"1= .0"> >>> <dict> >>> <key>com.apple.security.cs.allow-unsigned-executable-memory</key> >>> <true/> >>> </dict> >>> </plist> >>=20 >> I dropped that in a file, added `--entitlements=3D$THAT_FILE.plist` to >> my codesign invocations, removed all my workarounds for ctypes et. >> al. (except for the hard-coded 'import _cffi_backend' still necessary >> to convince modulegraph to include enough code for SSL to work), and >> then tried launching my app.>=20 > Which package needs _cffi_backend? I can add a recipe for that to > py2app to do this automagically. This may sound obvious, but: cffi :-). In my case, pyOpenSSL -> cryptography -> cffi. >=20 >> Success! Then I tried notarizing it: also success! Time permitting, >> I'll be updating my blog post at >> https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html >> with this information, and possibly publishing the now unfortunately >> somewhat complex tooling I use to do signing now.>>=20 >> So I don't know if I'm the first to *do* this, but looking at the >> archives for these lists I seem to be the first to *report* it: you >> can successfully codesign and notarize apps created with py2app and >> python 3.6!>>=20 >> It seems to me that whatever "MAP_JIT" is (an mmap flag, I'm >> guessing?) libffi needs to be using it for the memory it places >> synthetic closures into, so that this entitlement won't be necessary >> with some future version of Python. But it looks like Apple is not >> pushing particularly hard to deprecate this one right now, thank >> goodness :-).>=20 > MAP_JIT is a mmap flag that=E2=80=99s apparently introduced in 10.14. The > slides at https://developer.apple.com/videos/play/wwdc2018/702/ > mention this flag and the hardened runtime.>=20 > I guess we should add this flag to the code in > Modules/_ctypes/malloc_closure.c CPython) and in the similar code in > PyObjC. The annoying bit is that the flag is new in 10.14, and > CPython installers are created on 10.9 which means those won=E2=80=99t in= clude > the new flag for a long time.>=20 > I=E2=80=99ll have to check if using MAP_JIT is ok when deploying on older > macOS versions, or if the code should do a runtime version check. I can't shed any light on this, but I suspect the cffi folks will also have to figure this out, and may already have some sense of how this works.=C2=A0 I filed an issue with them here: https://bitbucket.org/cffi/cffi/issues/391/cffi-doesnt-work-inside-a-macos-= app-bundle --_----------=_154083372517408340 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="utf-8" <!DOCTYPE html> <html> <head> <title></title> <style type=3D"text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style> </head> <body><div style=3D"font-family:menlo, consolas, monospace;"><br></div> <div><br></div> <div><br></div> <div>On Sun, Oct 28, 2018, at 11:58 PM, Ronald Oussoren wrote:<br></div> <blockquote type=3D"cite"><div style=3D"font-family:menlo, consolas, monosp= ace;"><br></div> <div><div style=3D"font-family:menlo, consolas, monospace;"><br></div> <blockquote type=3D"cite"><div>On 29 Oct 2018, at 00:56, Glyph <<a href= =3D"mailto:[email protected]">[email protected]</a>> wrote:<= br></div> <div style=3D"font-family:menlo, consolas, monospace;"><br></div> <div><div style=3D"overflow-wrap:break-word;"><div style=3D"font-family:men= lo, consolas, monospace;"><br></div> <div><div style=3D"font-family:menlo, consolas, monospace;"><br></div> <blockquote type=3D"cite"><div>On Oct 28, 2018, at 2:57 PM, Glyph <<a hr= ef=3D"mailto:[email protected]">[email protected]</a>> wrote= :<br></div> <div style=3D"font-family:menlo, consolas, monospace;"><br></div> <div><div><blockquote type=3D"cite" style=3D"font-family:Menlo-Regular;font= -size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;le= tter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;wh= ite-space:normal;word-spacing:0px;text-size-adjust:auto;-webkit-text-stroke= -width:0px;text-decoration-line:none;text-decoration-style:initial;text-dec= oration-color:initial;"><div dir=3D"ltr"><div>I wonder what the =E2=80=9Cha= rdened runtime=E2=80=9D option actually does and enforces. In 3.7 th= e line in ctypes/__init__.py that causes the exception is a call that creat= es a dummy C function, and likely triggers the first allocation for storing= a libffi closure which could be something the hardened runtime doesn=E2=80= =99t like (being writeable + executable memory). <br></div> </div> </blockquote><div style=3D"font-family:Menlo-Regular;font-size:12px;font-st= yle:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:norma= l;text-align:start;text-indent:0px;text-transform:none;white-space:normal;w= ord-spacing:0px;-webkit-text-stroke-width:0px;text-decoration-line:none;tex= t-decoration-style:initial;text-decoration-color:initial;"><br></div> <div style=3D"font-family:Menlo-Regular;font-size:12px;font-style:normal;fo= nt-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:= start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0= px;-webkit-text-stroke-width:0px;text-decoration-line:none;text-decoration-= style:initial;text-decoration-color:initial;">Interesting. Perhaps what I w= ant is simply <a href=3D"https://developer.apple.com/documentation/sec= urity/com_apple_security_cs_allow-unsigned-executable-memory">https://devel= oper.apple.com/documentation/security/com_apple_security_cs_allow-unsigned-= executable-memory</a> then? Any chance you know how to jam that = into a `codesign` command line somehow? :-)<br></div> </div> <div style=3D"font-family:menlo, consolas, monospace;"><br></div> </div> </blockquote></div> <div style=3D"font-family:menlo, consolas, monospace;"><br></div> <div>Thank you so much for this tip, Ronald! This was much easier tha= n I anticipated, and things are working now!<br></div> </div> </div> </blockquote><div><br></div> <div style=3D"font-family:menlo, consolas, monospace;">Great.<br></div> </div> <div><div style=3D"font-family:menlo, consolas, monospace;"><br></div> <blockquote type=3D"cite"><div><div style=3D"overflow-wrap:break-word;"><di= v><br></div> <div>The relevant entitlements file is literally just:<br></div> <div><br></div> <blockquote style=3D"margin-top:0px;margin-right:0px;margin-bottom:0px;marg= in-left:40px;border-top-width:initial;border-right-width:initial;border-bot= tom-width:initial;border-left-width:initial;border-top-style:none;border-ri= ght-style:none;border-bottom-style:none;border-left-style:none;border-top-c= olor:initial;border-right-color:initial;border-bottom-color:initial;border-= left-color:initial;border-image-source:initial;border-image-slice:initial;b= order-image-width:initial;border-image-outset:initial;border-image-repeat:i= nitial;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0p= x;"><div><div><?xml version=3D"1.0" encoding=3D"UTF-8"?><br></div> <div><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "<a href=3D"ht= tp://www.apple.com/DTDs/PropertyList-1.0.dtd">http://www.apple.com/DTDs/Pro= pertyList-1.0.dtd</a>"><br></div> <div><plist version=3D"1.0"><br></div> <div><dict><br></div> <div><span style=3D"white-space:pre;"></span><key>com.apple.security.= cs.allow-unsigned-executable-memory</key><br></div> <div><span style=3D"white-space:pre;"></span><true/><br></div> <div></dict><br></div> <div></plist><br></div> </div> </blockquote><div style=3D"font-family:menlo, consolas, monospace;"><br></d= iv> <div>I dropped that in a file, added `--entitlements=3D$THAT_FILE.plist` to= my codesign invocations, removed all my workarounds for ctypes et. al. (ex= cept for the hard-coded 'import _cffi_backend' still necessary to convince = modulegraph to include enough code for SSL to work), and then tried launchi= ng my app. <br></div> </div> </div> </blockquote><div><br></div> <div style=3D"font-family:menlo, consolas, monospace;">Which package needs = _cffi_backend? I can add a recipe for that to py2app to do this automagical= ly.<br></div> </div> </blockquote><div style=3D"font-family:menlo, consolas, monospace;"><br></d= iv> <div style=3D"font-family:menlo, consolas, monospace;">This may sound obvio= us, but: cffi :-). In my case, pyOpenSSL -> cryptography -> cff= i.</div> <div style=3D"font-family:menlo, consolas, monospace;"><br></div> <blockquote type=3D"cite"><div><div style=3D"font-family:menlo, consolas, m= onospace;"><br></div> <blockquote type=3D"cite"><div><div style=3D"overflow-wrap:break-word;"><di= v>Success! Then I tried notarizing it: also success! Time permi= tting, I'll be updating my blog post at <a href=3D"https://glyph.twist= edmatrix.com/2018/01/shipping-pygame-mac-app.html">https://glyph.twistedmat= rix.com/2018/01/shipping-pygame-mac-app.html</a> with this information= , and possibly publishing the now unfortunately somewhat complex tooling I = use to do signing now.<br></div> <div><br></div> <div>So I don't know if I'm the first to <i>do</i> this, but looking a= t the archives for these lists I seem to be the first to <i>report</i> = ;it: you can successfully codesign and notarize apps created with py2app an= d python 3.6!<br></div> <div><br></div> <div>It seems to me that whatever "MAP_JIT" is (an mmap flag, I'm guessing?= ) libffi needs to be using it for the memory it places synthetic closures i= nto, so that this entitlement won't be necessary with some future version o= f Python. But it looks like Apple is not pushing particularly hard to= deprecate this one right now, thank goodness :-).<br></div> </div> </div> </blockquote><div><br></div> <div style=3D"font-family:menlo, consolas, monospace;">MAP_JIT is a mmap fl= ag that=E2=80=99s apparently introduced in 10.14. The slides at <a hre= f=3D"https://developer.apple.com/videos/play/wwdc2018/702/">https://develop= er.apple.com/videos/play/wwdc2018/702/</a> mention this flag and the h= ardened runtime. <br></div> </div> <div><br></div> <div>I guess we should add this flag to the code in <span class=3D"fon= t" style=3D"font-family:Menlo"><span class=3D"size" style=3D"font-size:11px= ">Modules/_ctypes/malloc_closure.c</span></span><span class=3D"font" style= =3D"font-family:Menlo"><span class=3D"size" style=3D"font-size:11px"> = C</span></span><span class=3D"font" style=3D"font-family:Menlo"><span class= =3D"size" style=3D"font-size:11px">Python) and in the similar code in = PyObjC. The annoying bit is that the flag is new in 10.14, and CPytho= n installers are created on 10.9 which means those won=E2=80=99t include th= e new flag for a long time. </span></span><br></div> <div><span class=3D"font" style=3D"font-family:Menlo"><span class=3D"size" = style=3D"font-size:11px"></span></span><br></div> <div><span class=3D"font" style=3D"font-family:Menlo"><span class=3D"size" = style=3D"font-size:11px">I=E2=80=99ll have to check if using MAP_JIT is ok = when deploying on older macOS versions, or if the code should do a runtime = version check. </span></span><br></div> </blockquote><div style=3D"font-family:menlo, consolas, monospace;"><br></d= iv> <div style=3D"font-family:menlo, consolas, monospace;">I can't shed any lig= ht on this, but I suspect the cffi folks will also have to figure this out,= and may already have some sense of how this works. I filed an issue = with them here: <a href=3D"https://bitbucket.org/cffi/cffi/issues/391/= cffi-doesnt-work-inside-a-macos-app-bundle">https://bitbucket.org/cffi/cffi= /issues/391/cffi-doesnt-work-inside-a-macos-app-bundle</a><br></div> </body> </html> --_----------=_154083372517408340-- --===============2708377343312227577== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============2708377343312227577== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Pyobjc-dev mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/pyobjc-dev --===============2708377343312227577==--