Re: get duplicate frames when doing multiprocessing

On 4/4/21 9:39 AM, Yodash wrote:

> Hello
> i'm trying to speed up my data load from pandas dataframe using 
> multiprocessing

I can't tell what you were trying to achieve here.  Your code loads the 
same file in three different processes but then does nothing with it.  
Each process will load the entire file.

Multiprocessing is really only useful in very limited circumstances.  
Remember that, when you use multiprocessing, you are literally running 
different processes with different memory spaces.  They are not 
connected in any way.  You can feed data in and copy data back out at 
the end.  And, of course, you can only manipulate the UI in your main 
process.  None of the others have any UI.

IF you are trying to load a single dataframe by splitting the load 
across three processes, that's impossible.


> _my problem is that when i do multiprocessing from wx.frame , I 
> surprisingly get duplicates of my frame_ -_as the number of processes 
> i'm trying to use_  (_no matter what method i 
> use_...Pool/Pool.apply_sync or Process(target=f_action,)
>
> it doesn't happen all the time
>
> for example..this code will duplicate the frame 3 times:

The code you showed will certainly do that, because you're loading the 
same file 3 times.

-- 

Tim Roberts, [email protected]
Providenza & Boekelheide, Inc.

-- 
You received this message because you are subscribed to the Google Groups "wxPython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/wxpython-users/fe5c4830-7213-070d-8f62-349228eaaacd%40probo.com.