Re: py2app apps codesigned with "--options runtime", i.e. any notorizable app, fails to launch with ctypes error (was Re: ctypes MemoryError when attempting to launch notarized app)
Ronald Oussoren via Pyobjc-dev <[email protected]> Sun, 28 Oct 2018 21:55:31 +0100
| Newsgroups | gmane.comp.python.pyobjc.devel,gmane.comp.python.apple |
|---|---|
| Message-ID | <[email protected]> |
--===============3622263285723262327== Content-type: multipart/alternative; boundary="Apple-Mail=_AE1B37ED-8C05-44C2-ABDB-19FB112007E5" --Apple-Mail=_AE1B37ED-8C05-44C2-ABDB-19FB112007E5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 28 Oct 2018, at 19:20, Glyph <[email protected]> wrote: >=20 >=20 >=20 >>>=20 >>> Curiously, this is the same traceback that comes from = https://forum.kodi.tv/showthread.php?tid=3D329171 = <https://forum.kodi.tv/showthread.php?tid=3D329171>, which suggests it's = something fundamental to strict shared-library sandboxing that ctypes = trips over when trying to initialize itself. >>>=20 >>> Does anyone have experience with this, or ideas about what to do? >>=20 >> I=E2=80=99m afraid not. I currently get away with not signing apps at = all, although properly supporting signing is on my way too long wish = list for py2app. =20 >=20 > The ability to distribute unsigned apps is not-so-slowly going away; = even the ability to distribute non-notarized apps has a very limited = shelf-life at this point. So this ought to be an alarming development = for everyone - having Python apps effectively banned from macOS = distribution is a big potential problem :-\. Yup. That=E2=80=99s something that worries me as well, and not just for = Python apps. Not being able to run your own code without paying Apple = for the privilege is not something I look forward to. I=E2=80=99m still = hoping that the option to run unsigned code doesn=E2=80=99t go away. >=20 > The good news here is that aside from having to write a little for = loop in shell (shown below) getting the app codesigned previously was = easy, and my app *did* pass notarization, so nothing that py2app is = doing is breaking things on apple's end. It's just a matter of a ctypes = bug. >=20 > As I see it, there's 2 problems here: >=20 > py2app's __boot__.py should fail more gracefully if initializing = ctypes doesn't work, since not everybody needs ctypes. Shall I file = this on the tracker? Yes please. I=E2=80=99d prefer a solution that doesn=E2=80=99t involve = ignoring errors, but that=E2=80=99s probably the easiest fix for now. = What=E2=80=99s the exception you=E2=80=99re getting? Tweaking the code = in py2app/bootstrap/ctypes_setup.py to ignore that exception would be = trivial.=20 > ctypes itself should address whatever eldritch hideousness is causing = this; in addition to the windows security layer stuff I found, = grsecurity TPE causes the same traceback: = https://bugs.python.org/issue28429 <https://bugs.python.org/issue28429> >> With some luck there=E2=80=99s some entitlement or code signing = option that causes this problem. What is the output of "codesign = --display --verbose=3D4=E2=80=9D for the application? Both with and = without notarisation?=20 >=20 > Sorry, my original message was not clear. App notarization itself is = not the problem, it's the "stricter requirements" that I ambiguously = referenced. The requirement in question is the '--options runtime' flag = passed to 'codesign'. So you can just codesign an app (even with an = ad-hoc identity, you technically could do this without even having a = valid cert, although the way one generates one of those escapes me) with = the 'runtime' option, you can reproduce this. >=20 > So if I sign my app like this: >=20 > #!/bin/bash > find "${NAME}.app" -iname '*.so' -or -iname '*.dylib' | > while read libfile; do > codesign --sign "${IDENTITY}" \ > --deep "${libfile}" \ > --force \ > --options runtime; > done; >=20 > codesign --sign "${IDENTITY}" \ > --deep "${NAME}.app" \ > --force \ > --options runtime; >=20 > and then run it as "./${NAME}.app/Contents/MacOS/${NAME}". I = immediately get the traceback given above. Great. That should make it easier for me to reproduce the issue.=20 Ronald --Apple-Mail=_AE1B37ED-8C05-44C2-ABDB-19FB112007E5 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br = class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D"">On 28 Oct 2018, at 19:20, Glyph <<a = href=3D"mailto:[email protected]" = class=3D"">[email protected]</a>> wrote:</div><br = class=3D"Apple-interchange-newline"><div class=3D""><meta = http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dutf-8" = class=3D""><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: = space; line-break: after-white-space;" class=3D""><div class=3D""><br = class=3D""><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D""><blockquote type=3D"cite" style=3D"font-family: = Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: = normal; font-weight: normal; letter-spacing: normal; orphans: auto; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><br = class=3D"">Curiously, this is the same traceback that comes from<span = class=3D"Apple-converted-space"> </span><a = href=3D"https://forum.kodi.tv/showthread.php?tid=3D329171" = class=3D"">https://forum.kodi.tv/showthread.php?tid=3D329171</a>, which = suggests it's something fundamental to strict shared-library sandboxing = that ctypes trips over when trying to initialize itself.<br class=3D""><br= class=3D"">Does anyone have experience with this, or ideas about what = to do?<br class=3D""></blockquote><br style=3D"caret-color: rgb(0, 0, = 0); font-family: Menlo-Regular; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, = 0); font-family: Menlo-Regular; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;" = class=3D"">I=E2=80=99m afraid not. I currently get away with not signing = apps at all, although properly supporting signing is on my way too long = wish list for py2app. </span><br style=3D"caret-color: rgb(0, 0, = 0); font-family: Menlo-Regular; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""></div></blockquote><div class=3D""><br = class=3D""></div><div class=3D"">The ability to distribute unsigned apps = is not-so-slowly going away; even the ability to distribute = non-notarized apps has a very limited shelf-life at this point. So = this ought to be an alarming development for everyone - having Python = apps effectively banned from macOS distribution is a big potential = problem :-\.</div></div></div></div></blockquote><div><br = class=3D""></div>Yup. That=E2=80=99s something that worries me as well, = and not just for Python apps. Not being able to run your own code = without paying Apple for the privilege is not something I look forward = to. I=E2=80=99m still hoping that the option to run unsigned code = doesn=E2=80=99t go away.</div><div><br class=3D""></div><div><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; line-break: = after-white-space;" class=3D""><div class=3D""><div class=3D""><br = class=3D""></div><div class=3D"">The good news here is that aside from = having to write a little for loop in shell (shown below) getting the app = codesigned previously was easy, and my app *did* pass notarization, so = nothing that py2app is doing is breaking things on apple's end. = It's just a matter of a ctypes bug.</div><div class=3D""><br = class=3D""></div><div class=3D"">As I see it, there's 2 problems = here:</div><div class=3D""><br class=3D""></div><div class=3D""><ol = class=3D"MailOutline"><li class=3D"">py2app's __boot__.py should fail = more gracefully if initializing ctypes doesn't work, since not everybody = needs ctypes. Shall I file this on the = tracker?</li></ol></div></div></div></div></blockquote><div><br = class=3D""></div>Yes please. I=E2=80=99d prefer a solution that = doesn=E2=80=99t involve ignoring errors, but that=E2=80=99s probably the = easiest fix for now. What=E2=80=99s the exception you=E2=80=99re = getting? Tweaking the code in py2app/bootstrap/ctypes_setup.py to ignore = that exception would be trivial. </div><div><br = class=3D""></div><div><br class=3D""><blockquote type=3D"cite" = class=3D""><div class=3D""><div style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = class=3D""><div class=3D""><ol class=3D"MailOutline" start=3D"2"><li = class=3D"">ctypes itself should address whatever eldritch hideousness is = causing this; in addition to the windows security layer stuff I found, = grsecurity TPE causes the same traceback: <a = href=3D"https://bugs.python.org/issue28429" = class=3D"">https://bugs.python.org/issue28429</a></li></ol></div><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Menlo-Regular; = font-size: 12px; font-style: normal; font-variant-caps: normal; = font-weight: normal; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none; float: none; display: inline !important;" class=3D"">With some = luck there=E2=80=99s some entitlement or code signing option that causes = this problem. What is the output of "codesign --display = --verbose=3D4=E2=80=9D for the application? Both with and without = notarisation?<span class=3D"Apple-converted-space"> </span></span><br= style=3D"caret-color: rgb(0, 0, 0); font-family: Menlo-Regular; = font-size: 12px; font-style: normal; font-variant-caps: normal; = font-weight: normal; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;" class=3D""></div></blockquote></div><br class=3D""><div = class=3D"">Sorry, my original message was not clear. App = notarization itself is not the problem, it's the "stricter requirements" = that I ambiguously referenced. The requirement in question is the = '--options runtime' flag passed to 'codesign'. So you can just = codesign an app (even with an ad-hoc identity, you technically could do = this without even having a valid cert, although the way one generates = one of those escapes me) with the 'runtime' option, you can reproduce = this.</div><div class=3D""><br class=3D""></div><div class=3D"">So if I = sign my app like this:</div><div class=3D""><br = class=3D""></div><blockquote style=3D"margin: 0 0 0 40px; border: none; = padding: 0px;" class=3D""><div class=3D"">#!/bin/bash</div><div = class=3D"">find "${NAME}.app" -iname '*.so' -or -iname '*.dylib' = |</div><div class=3D""> while read libfile; do</div><div = class=3D""> codesign --sign = "${IDENTITY}" \</div><div class=3D""> = --deep "${libfile}" \</div><div = class=3D""> = --force \</div><div class=3D""> = --options runtime;</div><div = class=3D""> done;</div><div class=3D""><br = class=3D""></div><div class=3D"">codesign --sign "${IDENTITY}" = \</div><div class=3D""> --deep = "${NAME}.app" \</div><div class=3D""> = --force \</div><div class=3D""> = --options runtime;</div></blockquote><div class=3D""><br = class=3D""></div><div class=3D"">and then run it as = "./${NAME}.app/Contents/MacOS/${NAME}". I immediately get the = traceback given above.</div></div></div></blockquote><div><br = class=3D""></div>Great. That should make it easier for me to reproduce = the issue. </div><div><br class=3D""></div><div>Ronald</div><div><br = class=3D""></div></body></html>= --Apple-Mail=_AE1B37ED-8C05-44C2-ABDB-19FB112007E5-- --===============3622263285723262327== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============3622263285723262327== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Pyobjc-dev mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/pyobjc-dev --===============3622263285723262327==--