Re: notarization works! (was Re: not that)
Ronald Oussoren via Pyobjc-dev <[email protected]> Mon, 29 Oct 2018 07:58:06 +0100
| Newsgroups | gmane.comp.python.pyobjc.devel,gmane.comp.python.apple |
|---|---|
| Message-ID | <[email protected]> |
--===============1049061394510867115== Content-type: multipart/alternative; boundary="Apple-Mail=_89404A7D-436A-4321-884B-607463C710BA" --Apple-Mail=_89404A7D-436A-4321-884B-607463C710BA Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 29 Oct 2018, at 00:56, Glyph <[email protected]> wrote: >=20 >=20 >=20 >> On Oct 28, 2018, at 2:57 PM, Glyph <[email protected] = <mailto:[email protected]>> wrote: >>=20 >>> I wonder what the =E2=80=9Chardened runtime=E2=80=9D option actually = does and enforces. In 3.7 the line in ctypes/__init__.py that causes = the exception is a call that creates a dummy C function, and likely = triggers the first allocation for storing a libffi closure which could = be something the hardened runtime doesn=E2=80=99t like (being writeable = + executable memory).=20 >>=20 >> Interesting. Perhaps what I want is simply = https://developer.apple.com/documentation/security/com_apple_security_cs_a= llow-unsigned-executable-memory = <https://developer.apple.com/documentation/security/com_apple_security_cs_= allow-unsigned-executable-memory> then? Any chance you know how to jam = that into a `codesign` command line somehow? :-) >>=20 >=20 > Thank you so much for this tip, Ronald! This was much easier than I = anticipated, and things are working now! Great. >=20 > The relevant entitlements file is literally just: >=20 > <?xml version=3D"1.0" encoding=3D"UTF-8"?> > <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" = "http://www.apple.com/DTDs/PropertyList-1.0.dtd = <http://www.apple.com/DTDs/PropertyList-1.0.dtd>"> > <plist version=3D"1.0"> > <dict> > = <key>com.apple.security.cs.allow-unsigned-executable-memory</key> > <true/> > </dict> > </plist> >=20 > I dropped that in a file, added `--entitlements=3D$THAT_FILE.plist` to = my codesign invocations, removed all my workarounds for ctypes et. al. = (except for the hard-coded 'import _cffi_backend' still necessary to = convince modulegraph to include enough code for SSL to work), and then = tried launching my app. =20 Which package needs _cffi_backend? I can add a recipe for that to py2app = to do this automagically. > Success! Then I tried notarizing it: also success! Time permitting, = I'll be updating my blog post at = https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html = <https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.html> = with this information, and possibly publishing the now unfortunately = somewhat complex tooling I use to do signing now. >=20 > So I don't know if I'm the first to do this, but looking at the = archives for these lists I seem to be the first to report it: you can = successfully codesign and notarize apps created with py2app and python = 3.6! >=20 > It seems to me that whatever "MAP_JIT" is (an mmap flag, I'm = guessing?) libffi needs to be using it for the memory it places = synthetic closures into, so that this entitlement won't be necessary = with some future version of Python. But it looks like Apple is not = pushing particularly hard to deprecate this one right now, thank = goodness :-). MAP_JIT is a mmap flag that=E2=80=99s apparently introduced in 10.14. = The slides at https://developer.apple.com/videos/play/wwdc2018/702/ = <https://developer.apple.com/videos/play/wwdc2018/702/> mention this = flag and the hardened runtime. =20 I guess we should add this flag to the code in = Modules/_ctypes/malloc_closure.c CPython) and in the similar code in = PyObjC. The annoying bit is that the flag is new in 10.14, and CPython = installers are created on 10.9 which means those won=E2=80=99t include = the new flag for a long time.=20 I=E2=80=99ll have to check if using MAP_JIT is ok when deploying on = older macOS versions, or if the code should do a runtime version check.=20= Ronald >=20 > -glyph > _______________________________________________ > Pyobjc-dev mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/pyobjc-dev --Apple-Mail=_89404A7D-436A-4321-884B-607463C710BA Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br = class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D"">On 29 Oct 2018, at 00:56, Glyph <<a = href=3D"mailto:[email protected]" = class=3D"">[email protected]</a>> wrote:</div><br = class=3D"Apple-interchange-newline"><div class=3D""><meta = http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dutf-8" = class=3D""><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: = space; line-break: after-white-space;" class=3D""><br class=3D""><div = class=3D""><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D"">On Oct 28, 2018, at 2:57 PM, Glyph <<a = href=3D"mailto:[email protected]" = class=3D"">[email protected]</a>> wrote:</div><br = class=3D"Apple-interchange-newline"><div class=3D""><div dir=3D"auto" = class=3D"Singleton"><blockquote type=3D"cite" style=3D"font-family: = Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: = normal; font-weight: normal; letter-spacing: normal; orphans: auto; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><div = dir=3D"ltr" class=3D""><div class=3D"">I wonder what the =E2=80=9Chardened= runtime=E2=80=9D option actually does and enforces. In 3.7 the = line in ctypes/__init__.py that causes the exception is a call that = creates a dummy C function, and likely triggers the first allocation for = storing a libffi closure which could be something the hardened runtime = doesn=E2=80=99t like (being writeable + executable = memory). </div></div></blockquote><div style=3D"caret-color: rgb(0, = 0, 0); font-family: Menlo-Regular; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><br class=3D""></div><div = style=3D"caret-color: rgb(0, 0, 0); font-family: Menlo-Regular; = font-size: 12px; font-style: normal; font-variant-caps: normal; = font-weight: normal; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;" class=3D"">Interesting. Perhaps what I want is simply <a = href=3D"https://developer.apple.com/documentation/security/com_apple_secur= ity_cs_allow-unsigned-executable-memory" = class=3D"">https://developer.apple.com/documentation/security/com_apple_se= curity_cs_allow-unsigned-executable-memory</a> then? Any = chance you know how to jam that into a `codesign` command line somehow? = :-)</div></div><br = class=3D"Apple-interchange-newline"></div></blockquote></div><br = class=3D""><div class=3D"">Thank you so much for this tip, Ronald! = This was much easier than I anticipated, and things are working = now!</div></div></div></blockquote><div><br = class=3D""></div>Great.</div><div><br class=3D""><blockquote type=3D"cite"= class=3D""><div class=3D""><div style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = class=3D""><br class=3D""></div><div class=3D"">The relevant = entitlements file is literally just:</div><div class=3D""><br = class=3D""></div><blockquote style=3D"margin: 0 0 0 40px; border: none; = padding: 0px;" class=3D""><div class=3D""><div class=3D""><?xml = version=3D"1.0" encoding=3D"UTF-8"?></div><div class=3D""><!DOCTYPE = plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "<a = href=3D"http://www.apple.com/DTDs/PropertyList-1.0.dtd" = class=3D"">http://www.apple.com/DTDs/PropertyList-1.0.dtd</a>"></div><d= iv class=3D""><plist version=3D"1.0"></div><div = class=3D""><dict></div><div class=3D""><span = class=3D"Apple-tab-span" style=3D"white-space:pre"> = </span><key>com.apple.security.cs.allow-unsigned-executable-memory&l= t;/key></div><div class=3D""><span class=3D"Apple-tab-span" = style=3D"white-space:pre"> </span><true/></div><div = class=3D""></dict></div><div = class=3D""></plist></div></div></blockquote><br class=3D""><div = class=3D"">I dropped that in a file, added = `--entitlements=3D$THAT_FILE.plist` to my codesign invocations, removed = all my workarounds for ctypes et. al. (except for the hard-coded 'import = _cffi_backend' still necessary to convince modulegraph to include enough = code for SSL to work), and then tried launching my app. = </div></div></div></blockquote><div><br class=3D""></div>Which = package needs _cffi_backend? I can add a recipe for that to py2app to do = this automagically.<br class=3D""><div><br class=3D""></div><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; line-break: = after-white-space;" class=3D""><div class=3D"">Success! Then I = tried notarizing it: also success! Time permitting, I'll be = updating my blog post at <a = href=3D"https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app.ht= ml" = class=3D"">https://glyph.twistedmatrix.com/2018/01/shipping-pygame-mac-app= .html</a> with this information, and possibly publishing the now = unfortunately somewhat complex tooling I use to do signing = now.</div><div class=3D""><br class=3D""></div><div class=3D"">So I = don't know if I'm the first to <i class=3D"">do</i> this, but = looking at the archives for these lists I seem to be the first to <i = class=3D"">report</i> it: you can successfully codesign and = notarize apps created with py2app and python 3.6!</div><div class=3D""><br= class=3D""></div><div class=3D"">It seems to me that whatever "MAP_JIT" = is (an mmap flag, I'm guessing?) libffi needs to be using it for the = memory it places synthetic closures into, so that this entitlement won't = be necessary with some future version of Python. But it looks like = Apple is not pushing particularly hard to deprecate this one right now, = thank goodness :-).</div></div></div></blockquote><div><br = class=3D""></div>MAP_JIT is a mmap flag that=E2=80=99s apparently = introduced in 10.14. The slides at <a = href=3D"https://developer.apple.com/videos/play/wwdc2018/702/" = class=3D"">https://developer.apple.com/videos/play/wwdc2018/702/</a> = mention this flag and the hardened runtime. </div><div><br = class=3D""></div><div>I guess we should add this flag to the code = in <span style=3D"font-family: Menlo; font-size: 11px;" = class=3D"">Modules/_ctypes/malloc_closure.c</span><span = style=3D"font-family: Menlo; font-size: 11px;" = class=3D""> C</span><font face=3D"Menlo" class=3D""><span = style=3D"font-size: 11px;" class=3D"">Python) and in the similar = code in PyObjC. The annoying bit is that the flag is new in 10.14, = and CPython installers are created on 10.9 which means those won=E2=80=99t= include the new flag for a long = time. </span></font></div><div><font face=3D"Menlo" class=3D""><span = style=3D"font-size: 11px;" class=3D""><br = class=3D""></span></font></div><div><font face=3D"Menlo" class=3D""><span = style=3D"font-size: 11px;" class=3D"">I=E2=80=99ll have to check if = using MAP_JIT is ok when deploying on older macOS versions, or if the = code should do a runtime version = check. </span></font></div><div><font face=3D"Menlo" class=3D""><span= style=3D"font-size: 11px;" class=3D""><br = class=3D""></span></font></div><div><font face=3D"Menlo" class=3D""><span = style=3D"font-size: 11px;" class=3D"">Ronald</span></font></div><div><font= face=3D"Menlo" class=3D""><span style=3D"font-size: 11px;" class=3D""><br= class=3D""></span></font></div><div><font face=3D"Menlo" class=3D""><span= style=3D"font-size: 11px;" class=3D""><br = class=3D""></span></font></div><div><blockquote type=3D"cite" = class=3D""><div class=3D""><div style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = class=3D""><br class=3D""></div><div = class=3D"">-glyph</div></div>_____________________________________________= __<br class=3D"">Pyobjc-dev mailing list<br class=3D""><a = href=3D"mailto:[email protected]" = class=3D"">[email protected]</a><br = class=3D"">https://lists.sourceforge.net/lists/listinfo/pyobjc-dev<br = class=3D""></div></blockquote></div><br class=3D""></body></html>= --Apple-Mail=_89404A7D-436A-4321-884B-607463C710BA-- --===============1049061394510867115== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============1049061394510867115== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Pyobjc-dev mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/pyobjc-dev --===============1049061394510867115==--