Re: Re: Have existing python script file that I want to execute from within a cherrypy web site/page. How can this be done 100% native to cherrypy?

2018-05-06 0:48 GMT+02:00 Jibun no Kage <[email protected]>:

> Not to debate, from my perspective, you are talking about virtualization,
> about application virtualization.  Per the link (
> https://virtualenv.pypa.io/en/stable/) you sent me, "In all these cases,
> virtualenv can help you. It creates an environment that has its own
> installation directories, that doesn’t share libraries with other
> virtualenv environments (and optionally doesn’t access the globally
> installed libraries either)."  That is 'application' virtualization from my
> perspective.  :)
>

Well, my assumption might be wrong, but I have a feeling that you might not
realize how virtualenv works. While it contains word "virtual" it's still
not a typical virtualization, nor isolation. Yes, it creates a separate
"env" folder with binaries linked against some other (system) installation
of python and site-packages, which won't collide with that system install
of python. So you add that thing to $PATH or invoke directly.
My point is that when running app and subprocesses you may unintentionally
switch versions when being not careful (for example, when you reset/erase
env vars available for subshell). And since I don't know anything about how
you run web app and scripts it is hard for me to point where to look closer.
So logging executable path and env vars would help you figure out more
details about this.


>
> As for code examples, I can't publish the explicit code, if I could, I
> would have.  I realize this limits your assistance, just the same, thanks
> for the insight, and suggestions, you have helped, I appreciate it.
>

You don't have to expose any of your super secret business logic or so. The
rule of thumb is to post minimal amount of similar code, which might
contain dummy "hello world", but still show how exactly to spawn
subprocesses, which from your words looks like:

*cp_app.py:*
import os
import cherrypy

class RootWebApp:
    @cherrypy.expose
    def index(self):
        exec_result =
os.subprocess.run(['/path/to/specific/py/env/bin/python',
'some_script.py'])  # might be different and in fact has lots of arguments
to play with.
        return exec_result.stdout

    @classmethod
    def run(cls):
        cherrypy.quickstart(cls(), '', {})


def main():
*    RootWebApp.run()*


*__name__ == '__main__' and main()*


*some_script.py:def main():*
    print('Hello world!')


*__name__ == '__main__' and main()*

As you can see, your example doesn't have to include any logic specific to
your project, but to your env and the way your components interact with
each other. This would help with troubleshooting.


> Since your domain is UA... you are in Ukraine?  I spent most of my summers
> as a kid in Germany, even got to visit Hungry, Poland and Yugoslavia
> (when it still existed), even East Germany once, but I never got to the
> Ukraine.  Maybe when I retire and have time, I can get back to Eastern
> Europe again.
>

I used to live in Ukraine for my whole life, but 3 months ago I moved to
Brno, Czech Republic for Red Hat :)


>
>
> Schorschi
>
>
> On 05/05/2018 02:28, Sviatoslav Sydorenko wrote:
>
> I'm not talking about virtualization at all. I only mentioned
> https://virtualenv.pypa.io/en/stable/
> There's also Pipenv which exposes a bit more full feature set
> https://packaging.python.org/tutorials/managing-dependencies/
>
> Besides this, I cannot help you with your problem, because this would
> require you posting a real code example with a reproducible issue.
>
> 2018-05-05 2:57 GMT+02:00 Jibun no Kage <[email protected]>:
>
>> I am under the impression you did not see my other note, showing an an
>> example of the odd scenario I found, where the non-CherryPy environment
>> software 'stack' accepts an enum based on a simple class definition, but
>> the CherryPy inclusive environment software 'stack' does not.  Believe
>> CherryPy is exposing something odd, rather than creating something odd, as
>> I noted before.
>>
>> Yes, virtualization is a best practice, and where and when we can use it,
>> it is, but not all micro-controller environments can be emulated, for
>> example BCM processors are very hard to emulate under Qemu.  I have design
>> and supported virtual labs and large scale production environments for over
>> 16 years, just about every variant of VMware, KVM, and even Hyper-V (and
>> Microsoft Virtual Server before Hyper-V).  Moreover, running applications
>> in slices, in such as things like docker (even LXC) is a best practice when
>> possible, application virtualization versus OS virtualization.  But
>> micro-controllers are often so specialized and narrow in resource scope,
>> that is not an option.  The interesting scenario is when ARM (as now ATOM)
>> based systems continue to scale up, from 32bit to 64bit, greater than 1GB
>> RAM, etc., more CPU cores, etc.  Then, realistic partitioning of
>> micro-controllers will be possible.  Looking forward to it.
>>
>> Side note, much of VMware's ESXi server solution, under the hood/behind
>> the scenes, leverages quite a bit of python in its control (console)
>> environment.  My first experience with python came from its internal use in
>> ESXi.
>>
>> Schorschi
>>
>>
>> On 05/04/2018 15:16, Sviatoslav Sydorenko wrote:
>>
>> First of all, it's not CherryPy picking interpreter, but vice versa - you
>> run CherryPy app under a certain interpreter (better explicitly chosen).
>> Also, I advise you to check sys.executable to learn which binary exactly
>> you use.
>> Finally, it is best practice to use a virtualenv to isolate environments
>> for different apps you run.
>>
>> --Sviatoslav.
>>
>> Sent from my phone, please pardon any typos.
>>
>> On Fri, May 4, 2018, 21:45 Jibun no Kage <[email protected]> wrote:
>>
>>> Ah... Yes, I know how shebang works.  But, I had not worried about it,
>>> since the expectation was that everything would be completely python 2.  I
>>> went snooping... all the 'existing' child scripts, are 'banged' explicitly
>>> to Python2.  However, one of my test scripts, I was using/learning CherryPy
>>> was not explicitly qualified.  So I added a bit of code to the CherryPy
>>> script and left it NOT banged on purpose to see what CherryPy was
>>> defaulting to:
>>>
>>> >>> Sys Get Version 2
>>> >>> Platform Get Version 2.7.13
>>>
>>> So CherryPy defaults to system default, and it is version 2, the child
>>> scripts are directed to version 2 via shebang.  All the modules used by the
>>> child scripts were developed in a version 2 context.  Moreover, I modified
>>> one of the child scripts to report what version bash was defaulting to when
>>> invoked by bash, a sanity check to validate, and it also reported version 2.
>>>
>>> Child script direct invocation...
>>> [2018-05-04 19:27:13 UTC]  >>> Platform Get Version 2.7.13
>>> [2018-05-04 19:27:13 UTC]  >>> Sys Get Version 2
>>>
>>> Child script indirect invocation via the os shell/bash trick...
>>> [2018-05-04 19:29:43 UTC]  >>> Platform Get Version 2.7.13
>>> [2018-05-04 19:29:43 UTC]  >>> Sys Get Version 2
>>>
>>> So the OS default is 2, the scripts with os module direction are version
>>> 2,  the CherryPy defaults to 2, the subprocess defaults to 2.
>>>
>>> So there is still something different between module load into CherryPy
>>> and invoking it, and module load into child script and then invoking it,
>>> the child script invoking always works, the module load to CherryPy then
>>> invoking, never works, the modules that exhibit the issue.  I will find it,
>>> in due course, because like we agreed, really want to import the modules
>>> and just use them directly, and avoid the os shell/bash trick.  At this
>>> point it has to be something really odd, tripping it.  Likely, it is some
>>> funky code in the module that is not very pythontic as it should be, since
>>> we do a lot of micro-controller/hardware driving python coding.
>>>
>>> Not to confuse things, but I already found one issue and resolved it, in
>>> our development lab.  It had to do with different enum modules clashing, I
>>> discovered that 'enum34' was conflicting with 'enum' module.  On one
>>> specific system, the modules installed was incorrect, got to love when
>>> (many) developers touch stuff... oh... wait, I am one of the developers.
>>> Ouch.
>>>
>>> And, thanks for the comments and insight, appreciate it.
>>>
>>> Schorschi
>>>
>>>
>>>
>>> On 05/03/2018 22:45, Sviatoslav Sydorenko wrote:
>>>
>>> 1) You run CherryPy correctly, but it doesn't matter, because you
>>> obviously run the scripts themselves in a way that they might choose
>>> different Python env or even version. Do you know about shebang, for
>>> example? Anyway, you didn't provide sufficient details to know what happens
>>> exactly.
>>> 2) Running scripts is not CherryPy specific, it's just Python. There's
>>> no connection of your problem with CherryPy, this is not something which
>>> has to be documented.
>>> 3) My guess is that you just don't know how os.subprocess or bash
>>> executable lookup works. However I cannot be sure, because you did not show
>>> any reproducible examples of code you think you have problems with.
>>>
>>> --Sviatoslav.
>>>
>>> Sent from my phone, please pardon any typos.
>>>
>>> On Fri, May 4, 2018, 03:28 Jibun no Kage <[email protected]> wrote:
>>>
>>>> That could be, but how would anyone know?  When I run python <script
>>>> name>.py (which is a CherryPy) script, I am executing the default python
>>>> for the environment, which happens to be 2.7.x.  The CherryPy examples to
>>>> not qualify doing anything different.  For example...
>>>>
>>>> import cherrypy
>>>>
>>>> class HelloWorld(object):
>>>>
>>>>     @cherrypy.expose
>>>>
>>>>     def index(self):
>>>>         return "Hello world!"
>>>>
>>>> if __name__ == '__main__':
>>>>     cherrypy.quickstart(HelloWorld())
>>>>
>>>> Run as "python <script name>.py"  This is taken straight from the
>>>> online CherryPy official documentation.  So, I should be 100% python
>>>> version consistent, regardless of the invocation.  Of it was otherwise,
>>>> then CherryPy documentation should be explicit on this point, no?  None of
>>>> the modules I am calling other than CherryPy as imported python 3
>>>> specific.  Well, less the pending backports, when I get to that point.
>>>>
>>>> Schorschi
>>>>
>>>> On 05/02/2018 14:53, Sviatoslav Sydorenko wrote:
>>>>
>>>> It *is supported*, but you are doing it wrong. So it looks to me that
>>>> you are spawning python3 interpreter from within python2 process.
>>>>
>>>> 2018-05-02 23:36 GMT+02:00 Jibun no Kage <[email protected]>:
>>>>
>>>>> Thanks for the suggestions.  Unfortunately, at this time I can't
>>>>> address Python 3.  Python 2 only environment at the moment.  Odd, CherryPy
>>>>> documentation states that Python 2.7 is supported, but not surprised
>>>>> something tripped up.  Long term, as we revise the classes, migrate off of
>>>>> Python 2, expect to import and use them directly, as you suggest.
>>>>>
>>>>> Schorschi
>>>>>
>>>>>
>>>>> On 04/28/2018 23:21, Sviatoslav Sydorenko wrote:
>>>>>
>>>>> 1) don't use os module to spawn subprocesses, just have a function in
>>>>> those modules, import it and call it then.
>>>>> 2) you probably wrote scripts in a manner/syntax, which is
>>>>> incompatible with Python 3 and run CherryPy under Python 3. Or maybe just a
>>>>> way you run scripts is different when you spawn subprocesses. Anyway, I'd
>>>>> recommend you to not use subprocesses and do normal direct imports.
>>>>> To mitigate cross python compatibility problems I strongly recommend
>>>>> you writing scripts using syntax, which works in both or completely switch
>>>>> to Python 3.
>>>>> Look for a package distribution called "six", it may provide you some
>>>>> helpers for cross compat.
>>>>>
>>>>> --Sviatoslav.
>>>>>
>>>>> Sent from my phone, please pardon any typos.
>>>>>
>>>>>
>>>>> On Sun, Apr 29, 2018, 06:12 <[email protected]> wrote:
>>>>>
>>>>>> Yeah, I got a variant of it working.  Not sure it is pretty, but
>>>>>> works.  We have a lot of existing python scripts we want to continue to use
>>>>>> from console (Linux).  We just wanted to wrapper these with some simple web
>>>>>> pages, to make them a bit easier to use.  So, I just imported the os
>>>>>> module, can execute the script.  This is all internal stuff, nothing that
>>>>>> has to be production hardened or such.  This lets the guys writing the
>>>>>> python scripts for console do their thing, and lets me provide another
>>>>>> avenue for execution.
>>>>>>
>>>>>> I plan to swing back around on this one, and do more work, but using
>>>>>> os module for now works.  I did notice one odd thing, a couple of our
>>>>>> python classes/functions that work great from console, break when I import
>>>>>> them to a CherryPy script, and then execute as you note.  One error is that
>>>>>> bit shifting with the '<<' operator gripes about how it is used.  Only get
>>>>>> that error, when the given function/routine is imported to CherryPy
>>>>>> environment.  From console (python 2), no error at all.
>>>>>>
>>>>>>
>>>>>> On Saturday, April 28, 2018 at 1:04:56 PM UTC-7, [email protected]
>>>>>> wrote:
>>>>>>>
>>>>>>> Have existing python script file that I want to execute from within
>>>>>>> a cherrypy web site/page.  How can this be done 100% native to cherrypy?
>>>>>>> Many examples reference java or CGI mods, that is not acceptable nor should
>>>>>>> it be IMHO.  There has to be a 100% cherrypi method to do this, no?
>>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "cherrypy-users" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to cherrypy-users+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/[email protected]
>>>>>> To post to this group, send email to cherrypy-users-/[email protected].
>>>>>> Visit this group at https://groups.google.com/group/cherrypy-users.
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "cherrypy-users" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to cherrypy-users+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/[email protected]
>>>>> To post to this group, send email to cherrypy-users-/JYPxA39Uh5TLH3MbocFF+G/[email protected]
>>>>> Visit this group at https://groups.google.com/group/cherrypy-users.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Kind regards,
>>>>
>>>> Sviatoslav Sydorenko
>>>> email: [email protected]
>>>> cell #: +380978963757
>>>>
>>>>
>>>>
>>>
>>
>
>
> --
> Kind regards,
>
> Sviatoslav Sydorenko
> email: [email protected]
> cell #: +380978963757
> --
> You received this message because you are subscribed to the Google Groups
> "cherrypy-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cherrypy-users+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/[email protected]
> To post to this group, send email to cherrypy-users-/JYPxA39Uh5TLH3MbocFF+G/[email protected]
> Visit this group at https://groups.google.com/group/cherrypy-users.
> For more options, visit https://groups.google.com/d/optout.
>
>
>


-- 
Kind regards,

Sviatoslav Sydorenko
email: [email protected]
cell #: +380978963757

-- 
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cherrypy-users+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/[email protected]
To post to this group, send email to cherrypy-users-/JYPxA39Uh5TLH3MbocFF+G/[email protected]
Visit this group at https://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/d/optout.