Re: get duplicate frames when doing multiprocessing

For I/O bound tasks such as reading from disk, use threads.

On Mon, Apr 5, 2021, 7:35 AM Yodash <[email protected]> wrote:

> first, thanks for the quick replay :)..
> my target is to load data frames in parallel in order to save time
> (trigger for the action is button click on frame)
> you are correct, i loaded same df 3 times but its just an example (on my
> production app they are not the same)
>
> the issue is that whenever I try to perform parallelism using
> multiprocessing..my frame is duplicated as number of child processes I
> declare
>
> it will be a pity not to be able use parallelism from wx...
> Sure I'm missing something
>
> thanks,Yodash
> On Sunday, April 4, 2021 at 10:08:43 PM UTC+3 Tim Roberts wrote:
>
>> On 4/4/21 9:39 AM, Yodash wrote:
>>
>> Hello
>> i'm trying to speed up my data load from pandas dataframe using
>> multiprocessing
>>
>> I can't tell what you were trying to achieve here.  Your code loads the
>> same file in three different processes but then does nothing with it.  Each
>> process will load the entire file.
>>
>> Multiprocessing is really only useful in very limited circumstances.
>> Remember that, when you use multiprocessing, you are literally running
>> different processes with different memory spaces.  They are not connected
>> in any way.  You can feed data in and copy data back out at the end.  And,
>> of course, you can only manipulate the UI in your main process.  None of
>> the others have any UI.
>>
>> IF you are trying to load a single dataframe by splitting the load across
>> three processes, that's impossible.
>>
>>
>> *my problem is that when i do multiprocessing from wx.frame , I
>> surprisingly get duplicates of my frame* -* as the number of processes
>> i'm trying to use*  (*no matter what method i use*...Pool/Pool.apply_sync
>> or Process(target=f_action,)
>>
>> it doesn't happen all the time
>>
>> for example..this code will duplicate the frame 3 times:
>>
>> The code you showed will certainly do that, because you're loading the
>> same file 3 times.
>> --
>>
>> Tim Roberts, [email protected]
>> Providenza & Boekelheide, Inc.
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "wxPython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/wxpython-users/8224ee76-b447-4c2e-a39f-65bff32cef41n%40googlegroups.com
> <https://groups.google.com/d/msgid/wxpython-users/8224ee76-b447-4c2e-a39f-65bff32cef41n%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups "wxPython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/wxpython-users/CA%2B7ESbpeX-Je%3DgurbYr2FZHHUgkJo%3DUdq1mE1nYwJTy%3DNqW-wQ%40mail.gmail.com.