URGENT || Python 3.11, MySQL, SQLobject 3.9.1 not working in multiprocessing pool

Nitin Chauhan <[email protected]> Tue, 2 Apr 2024 09:37:51 -0700
Newsgroups gmane.comp.python.sqlobject
Message-ID <CAE-itpY4WqVxnKFEf2kv_dFQUGsZBDujAN7uMC7w53qiSgpBVw@mail.gmail.com>
--===============1180102969252324474==
Content-Type: multipart/alternative; boundary="000000000000cb79d406151fba84"

--000000000000cb79d406151fba84
Content-Type: text/plain; charset="UTF-8"

Hi SQLObject forum,

I have been trying to implement a multi-processed migration job that
parallelly reads mutually exclusive batches of records from a single table
based on the record's ID and sends that data to elasticsearch.

I use the cpu_count in multiprocessing module to at run time how many cores
exists, creates same number of batches and then create a multi processed
pool based on that value of cpu_count that each one of them reads mutually
exclusive records with absolutely no shared state to read the records.But
this doesn't seem to work.

Can anyone please share if SQLObject even supports multiprocessing pools?
If yes, can someone please guide me to an example blog or some code snippet
that is able to successfully handle SQlObject MySQL connection in
multiprocessing pools?

Looking forward to hearing back from you.

Best Regards,
Nitin Chauhan

--000000000000cb79d406151fba84
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi SQLObject forum,<div><br></div><div>I have been trying =
to implement a multi-processed=C2=A0migration job that parallelly reads mut=
ually exclusive batches of records from a single table based on the record&=
#39;s ID and sends that data to elasticsearch.=C2=A0</div><div><br></div><d=
iv>I use the cpu_count in multiprocessing module to at run time how many co=
res exists, creates same number of batches and then create a multi processe=
d pool based on that value of cpu_count that each one of them reads mutuall=
y exclusive records with absolutely no shared state to read the records.But=
 this doesn&#39;t seem to work.=C2=A0</div><div><br></div><div>Can anyone p=
lease share if SQLObject even supports multiprocessing pools? If yes, can s=
omeone please guide me to an example blog or some code snippet that is able=
 to successfully handle SQlObject MySQL connection in multiprocessing pools=
?</div><div><br></div><div>Looking forward to hearing back from you.</div><=
div><br></div><div>Best Regards,</div><div>Nitin Chauhan</div></div>

--000000000000cb79d406151fba84--


--===============1180102969252324474==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


--===============1180102969252324474==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
sqlobject-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss

--===============1180102969252324474==--