Re: Bug#1117718: Introducing Free-Threaded Python to Debian

I am not keen introducing this before 3.14 is the default.  This way, we 
can concentrate on the 3.14 transition first.  Then we can look at the 
free-threaded build, and also have a chance to look for regressions.

Matthias


On 11/16/25 16:16, ben0i0d wrote:
> Hello everyone,
> In my previous email (https://lists.debian.org/debian-python/2025/11/ 
> msg00002.html), I introduced my work on bringing free-threaded Python to 
> Debian. I'm now pleased to announce that the integration of nogil with 
> the GIL version has been successfully completed and has passed necessary 
> testing.
> Benefits of Introducing Free-Threaded Python:
> 1.Significant improvement in multi-threaded parallel performance
> 2.No need to migrate to multiprocessing or alternative languages - 
> currently multiprocessing is required to utilize multi-core performance, 
> but it comes with substantial overhead
> 3.Reduced maintenance burden for C/C++ extensions - currently complex 
> GIL management or custom thread pools are needed to work around the GIL
> 4.Alignment with future direction - PEP 703 has been accepted, and 
> upstream has confirmed that No-GIL will be the default in the future
> Current Progress:
> A merge request has been opened on salsa: https://salsa.debian.org/ 
> cpython-team/python3/-/merge_requests/41
> An immediately testable package repository is available at: https:// 
> salsa.debian.org/ben0i0d/python3t-repo
> The related Debian Bug report (#1117718) is also relevant to this work.
> Design Decisions:
> Based on discussions with Stefano, the current approach:
> -Maintains isolated standard libraries
> -Does NOT isolate dist-packages
> -Is not a separate new package, but rather a variant of python3
> My guiding principle remains: introducing free-threaded Python should 
> not create new problems, break existing functionality, or hinder 
> anyone's work.
> Technical Implementation Details:
> Key changes in python3.14:
> 1.Introduced ABI variant control: 'ABI_VARIANTS := gil nogil', with '-- 
> disable-gil --with-suffix=t' enabled for nogil
> (Note: '--enable-experimental-jit' cannot be used with '--disable-gil')
> 2.Build system optimization: Implemented duplication of nogil tasks via 
> '$(foreach abi,$(ABI_VARIANTS),$(eval $(call BUILD_STATIC,$(abi))))' to 
> avoid code redundancy
> 3.Test adjustments: Added 'TEST_EXCLUDES += test_tools' since 'Tools/ 
> freeze/test/Makefile' uses hardcoded 'python'
> 4.venv fix: Added 'add-abiflags-sitepackages.diff' to fix venv 
> sitepackages recognition after nogil build (this issue has been fixed 
> upstream and the patch will be removed after 3.14.1 release)
> 5.Minimal installation: The nogil version excludes documentation, idle, 
> tk, desktop, binfmt, etc., ensuring it's an extension rather than a 
> rewrite of the original python3
> 6.Code readability: Made harmless text adjustments to rules file, such 
> as consolidating scattered 'TEST_EXCLUDES +=' statements
> Build Process:
> git clone [email protected]:ben0i0d/python3t.git
> cd python3t
> git checkout python3
> uscan --download-current-version --verbose
> dpkg-source -b .
> sudo env DEB_BUILD_OPTIONS="nocheck nobench" pbuilder build ../ 
> python3.14_3.14.0-5.dsc 2>&1 | tee ../log.txt
> Test Environment:
> -OS: Debian GNU/Linux forky/sid (forky) x86_64
> -CPU: AMD Ryzen 5 9600X (12) @ 5.68 GHz
> -Memory: 30.47 GiB
> -Kernel: 6.16.12+deb14+1-amd64
> Test Results:
> 1.Basic Performance Test
> # GIL version, single-threaded
> python3.14 benchmark.py --n 512 --threads 1
> Elapsed: 1.836 s
> # GIL version, 8 threads
> python3.14 benchmark.py --n 512 --threads 8
> Elapsed: 2.026 s
> # nogil version, single-threaded
> python3.14t benchmark.py --n 512 --threads 1
> Elapsed: 2.408 s
> # nogil version, 8 threads
> python3.14t benchmark.py --n 512 --threads 8
> Elapsed: 0.674 s
> 2.NumPy Compatibility Test
> Both versions successfully create virtual environments and install numpy:
> GIL environment:
> python3.14 -m venv gil
> source gil/bin/activate
> pip install numpy
> # Runs normally, Elapsed: 0.003 s
> nogil environment:
> python3.14t -m venv nogil
> source nogil/bin/activate
> pip install numpy
> # Runs normally, Elapsed: 0.009 s
> Important Notes:
> -The GIL version build results are identical to the current master branch
> -Building only the GIL version is supported, but building only nogil is 
> not (ensuring nogil doesn't become default)
> -No backport to python3.13 is planned since nogil support there is still 
> experimental
> Regarding dist-packages:
> I understand some colleagues have concerns about not isolating dist- 
> packages. Currently, I recommend users employ the nogil version within 
> venv environments. I will fully support subsequent migration efforts to 
> help more people transition smoothly.
> The current implementation requires more testing and refinement. I'm not 
> a CPython expert, so I genuinely welcome valuable suggestions from the 
> community and commit to actively participating in improvement efforts.
> Please refer to the attachments for detailed build logs and test scripts.
> I look forward to your feedback!
> Best regards,
> *Xu Chen (ben0i0d)*