URGENT || Python 3.11, MySQL, SQLobject 3.9.1 not working in multiprocessing pool
Nitin Chauhan <[email protected]> Tue, 2 Apr 2024 09:37:51 -0700
| Newsgroups | gmane.comp.python.sqlobject |
|---|---|
| Message-ID | <CAE-itpY4WqVxnKFEf2kv_dFQUGsZBDujAN7uMC7w53qiSgpBVw@mail.gmail.com> |
--===============1180102969252324474== Content-Type: multipart/alternative; boundary="000000000000cb79d406151fba84" --000000000000cb79d406151fba84 Content-Type: text/plain; charset="UTF-8" Hi SQLObject forum, I have been trying to implement a multi-processed migration job that parallelly reads mutually exclusive batches of records from a single table based on the record's ID and sends that data to elasticsearch. I use the cpu_count in multiprocessing module to at run time how many cores exists, creates same number of batches and then create a multi processed pool based on that value of cpu_count that each one of them reads mutually exclusive records with absolutely no shared state to read the records.But this doesn't seem to work. Can anyone please share if SQLObject even supports multiprocessing pools? If yes, can someone please guide me to an example blog or some code snippet that is able to successfully handle SQlObject MySQL connection in multiprocessing pools? Looking forward to hearing back from you. Best Regards, Nitin Chauhan --000000000000cb79d406151fba84 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Hi SQLObject forum,<div><br></div><div>I have been trying = to implement a multi-processed=C2=A0migration job that parallelly reads mut= ually exclusive batches of records from a single table based on the record&= #39;s ID and sends that data to elasticsearch.=C2=A0</div><div><br></div><d= iv>I use the cpu_count in multiprocessing module to at run time how many co= res exists, creates same number of batches and then create a multi processe= d pool based on that value of cpu_count that each one of them reads mutuall= y exclusive records with absolutely no shared state to read the records.But= this doesn't seem to work.=C2=A0</div><div><br></div><div>Can anyone p= lease share if SQLObject even supports multiprocessing pools? If yes, can s= omeone please guide me to an example blog or some code snippet that is able= to successfully handle SQlObject MySQL connection in multiprocessing pools= ?</div><div><br></div><div>Looking forward to hearing back from you.</div><= div><br></div><div>Best Regards,</div><div>Nitin Chauhan</div></div> --000000000000cb79d406151fba84-- --===============1180102969252324474== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============1180102969252324474== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ sqlobject-discuss mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss --===============1180102969252324474==--