Re: [Biopython] A possibility for speeding up FASTA/FASTQ reading in BioPython

--===============5306454801347556624==
Content-Type: multipart/alternative; boundary="00000000000026294d0643df6960"

--00000000000026294d0643df6960
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Great work! Sounds fantastic!

Can you (or Claude!) create a PR?

On Tue, Nov 11, 2025, 10:00=E2=80=AFPM Jones Kelly, Terence Carleton <
[email protected]> wrote:

> Hi all
>
> I regularly process reasonably large FASTQ (hundreds of billions of
> sequencing reads) and FASTA files using BioPython. For some years I've be=
en
> meaning to implement a FASTQ/FASTA reader in a compiled language and add
> Python bindings to improve the speed. I could've done this in C but I spe=
nt
> some decades writing C and I wanted to learn something new, so I consider=
ed
> a few languages. Because Rust makes it very easy to create Python binding=
s,
> I decided to give it a try. I thought I'd get going by asking the Claude
> CLI to write me some Rust. That turned out to be a much, much better
> experience than I had anticipated. With Claude I played with several
> implementations, keeping track of timing. Claude also wrote some tests.
> To compare what I was seeing I got Claude to write a pure Python version,=
 a
> pure C version, Python bindings to the C, and to create a benchmark suite=
.
> From what I can tell, the Rust/Python (and the C/Python) FASTA reading is
> twice as fast as BioPython and FASTQ reading is four times as fast. I
> didn't write a single line of code. I just did some minimal cleaning up
> when things were already far along. I've been using the code for the last
> month or two with no problems.
>
> The repo is at https://github.com/VirologyCharite/prseq  (prseq =3D
> Python/Rust for sequences). You'll find the benchmark results on that
> page.  There are still some small things I would adjust in the API.  BTW,
> Claude also wrote the README (which should definitely be improved).
>
> I am wondering if there might be interest in incorporating this into
> BioPython. I don't know if there are any Rust dependencies in BioPython b=
ut
> I know that there are some C extensions. We could use either, as their
> speeds are comparable. If there's interest, I'd be happy to help (or to
> do it all, after some discussion and maybe with some guidance).
>
> Thanks very much for all the work on BioPython. It's really been a
> pleasure to use the code over the last dozen years or so.
>
> Terry Jones
>
>
> _______________________________________________
> Biopython mailing list  -  [email protected]
> https://mailman.open-bio.org/mailman/listinfo/biopython
>

--00000000000026294d0643df6960
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto">Great work! Sounds fantastic!=C2=A0<div dir=3D"auto"><br>=
</div><div dir=3D"auto">Can you (or Claude!) create a PR?</div></div><br><d=
iv class=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gm=
ail_attr">On Tue, Nov 11, 2025, 10:00=E2=80=AFPM Jones Kelly, Terence Carle=
ton &lt;<a href=3D"mailto:[email protected]">[email protected]=
e</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin=
:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">



<div>
<div style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font-size:12pt;c=
olor:rgb(0,0,0)">
Hi all</div>
<div dir=3D"ltr" style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font=
-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div><span style=3D"font-size:16px">I regularly process reasonably large FA=
STQ (hundreds of billions of sequencing reads) and FASTA files using BioPyt=
hon. For some years I&#39;ve been meaning to implement a FASTQ/FASTA reader=
 in a compiled language and add Python
 bindings to improve the speed. I could&#39;ve done this in C but I spent s=
ome decades writing C and I wanted to learn something new, so I considered =
a few languages. Because Rust makes it very easy to create Python bindings,=
 I decided to give it a try. I thought
 I&#39;d get going by asking the Claude CLI to write me some Rust. That tur=
ned out to be a much, much better experience than I had anticipated. With C=
laude I played with several implementations, keeping track of timing. Claud=
e
</span>also<span style=3D"font-size:16px">=C2=A0wrote some tests. To compar=
e what I was seeing I got Claude to write a pure Python version, a pure C v=
ersion, Python bindings to the C, and to create a benchmark suite. From wha=
t I can tell, the Rust/Python (and the
 C/Python) FASTA reading is twice as fast as BioPython and FASTQ reading is=
 four times as fast.
</span><span style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font-siz=
e:16px;color:rgb(0,0,0);background-color:rgb(255,255,255)">I didn&#39;t wri=
te a single line of code. I just did some minimal cleaning up when things w=
ere already far along.</span><span style=3D"font-family:Aptos,Arial,Helveti=
ca,sans-serif;font-size:12pt;color:rgb(0,0,0)">=C2=A0I&#39;ve
 been using the code for the last month or two with no problems.</span></di=
v>
<div dir=3D"ltr" style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font=
-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font-size:12pt;c=
olor:rgb(0,0,0)">
The repo is at <a href=3D"https://github.com/VirologyCharite/prseq" target=
=3D"_blank" rel=3D"noreferrer">
https://github.com/VirologyCharite/prseq</a>=C2=A0 (prseq =3D Python/Rust f=
or sequences). You&#39;ll find the benchmark results on that page.=C2=A0 Th=
ere are still some small things I would adjust in the API.=C2=A0 BTW, Claud=
e also wrote the README (which should definitely be improved).</div>
<div dir=3D"ltr" style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font=
-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div dir=3D"ltr" style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font=
-size:16px;color:rgb(0,0,0)">
<span style=3D"background-color:rgb(255,255,255)">I am wondering if there m=
ight be interest in incorporating this into BioPython. I don&#39;t know if =
there are any Rust dependencies in BioPython but I know that there are some=
 C extensions. We could use either,
 as their speeds are comparable. </span><span style=3D"font-size:12pt">If t=
here&#39;s interest, I&#39;d be happy to help (or to do it all, after some =
discussion and maybe with some guidance).</span></div>
<div dir=3D"ltr" style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font=
-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font-size:12pt;c=
olor:rgb(0,0,0)">
Thanks very much for all the work on BioPython. It&#39;s really been a plea=
sure to use the code over the last dozen years or so.</div>
<div dir=3D"ltr" style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font=
-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font-size:12pt;c=
olor:rgb(0,0,0)">
Terry Jones</div>
<div dir=3D"ltr" style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font=
-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div dir=3D"ltr" style=3D"font-family:Aptos,Arial,Helvetica,sans-serif;font=
-size:12pt;color:rgb(0,0,0)">
<br>
</div>
</div>

_______________________________________________<br>
Biopython mailing list=C2=A0 -=C2=A0 <a href=3D"mailto:Biopython@biopython.=
org" target=3D"_blank" rel=3D"noreferrer">[email protected]</a><br>
<a href=3D"https://mailman.open-bio.org/mailman/listinfo/biopython" rel=3D"=
noreferrer noreferrer" target=3D"_blank">https://mailman.open-bio.org/mailm=
an/listinfo/biopython</a><br>
</blockquote></div>

--00000000000026294d0643df6960--

--===============5306454801347556624==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Biopython mailing list  -  [email protected]
https://mailman.open-bio.org/mailman/listinfo/biopython

--===============5306454801347556624==--