Re: [Biopython] Comparing two already aligned sequences quickly to count gaps/matches/mismatches
Michiel de Hoon <[email protected]> Thu, 6 Feb 2025 12:29:10 +0000 (UTC)
| Newsgroups | gmane.comp.python.bio.general |
|---|---|
| Message-ID | <[email protected]> |
--===============6846260113852510385==
Content-Type: multipart/alternative;
boundary="----=_Part_9228321_80885842.1738844950540"
------=_Part_9228321_80885842.1738844950540
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
> This is with a FASTA input MSA (all sequences the same length with gap c=
haracters, and some pairs will have common gaps).
Then you can use the `"fasta"` parser in `Bio.Align`, which parses the alig=
ned sequences in C, so it should be fast.Then call the new `.counts`=C2=A0 =
method in Biopython 1.86dev with ignore_sequences=3DTrue to calculate the n=
umber of insertions and deletions quickly.This won't give you the number of=
gap-against-gap alignments, but those are not meaningful anyway.
-Michiel
On Thursday, February 6, 2025 at 07:51:48 PM GMT+9, Peter Cock <p.j.a.c=
[email protected]> wrote: =20
=20
This is with a FASTA input MSA (all sequences the same length with gap cha=
racters, and some pairs will have common gaps).
I have been loading this incrementally so only two sequences were in RAM at=
any point - and then switched to multiple threads (so at any point with K =
threads only 2*K sequences were loaded).
The test case is only about 100MB on disk, 100 sequences each of 1 million =
base pairs, and takes about a minute or two (multi-threaded laptop).
I don't think this will be memory constrained, so working on the entire MSA=
with a single thread is fine (if faster).
Thank you,
Peter
On Thu, Feb 6, 2025 at 10:36=E2=80=AFAM Michiel de Hoon <[email protected]=
m> wrote:
> I have lots of pairs of pre-aligned sequences (imported from an external=
MSA file),
In which format is your MSA file?
-Michiel
On Thursday, January 30, 2025 at 11:59:33 PM GMT+9, Peter Cock <p.j.a.c=
[email protected]> wrote: =20
=20
Hello all, and Michiel in particular,
I am wondering if any of the pairwise alignment code in Bio.Align (much of =
which is written in C for speed) could help with this use case?:
I have lots of pairs of pre-aligned sequences (imported from an external MS=
A file), for which I am doing something like this:
```python
def count_matches_etc(query_seq, subject_seq):
=C2=A0 =C2=A0 assert len(query_seq) =3D=3D len(subject_seq), "Should be sam=
e length"
=C2=A0 =C2=A0 matches =3D non_gap_mismatches =3D either_gapped =3D both_gap=
ped =3D 0
=C2=A0 =C2=A0 for q, s in zip(query_seq, subject_seq, strict=3DTrue):
=C2=A0 =C2=A0 =C2=A0 =C2=A0 if q =3D=3D "-" and s =3D=3D "-":
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 both_gapped +=3D 1
=C2=A0 =C2=A0 =C2=A0 =C2=A0 elif q =3D=3D "-" or s =3D=3D "-":
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 either_gapped +=3D 1
=C2=A0 =C2=A0 =C2=A0 =C2=A0 elif q =3D=3D s:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 matches +=3D 1
=C2=A0 =C2=A0 =C2=A0 =C2=A0 else:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 non_gap_mismatches +=3D 1
=C2=A0 =C2=A0 assert matches + non_gap_mismatches + either_gapped + both_ga=
pped =3D=3D len(query_seq)
=C2=A0 =C2=A0 return matches, non_gap_mismatches, either_gapped, both_gappe=
d
# Test case
assert (9, 1, 2, 1) =3D=3D count_matches_etc("ACGTAC-TAC-GT", "AGGT-CGTAC-G=
T")
```
Sticking with Python that could be optimized (e.g. I am currently using thi=
s with sequences of a million base pairs but few gaps), however I have writ=
ten this example with clarity foremost in mind.
Thank you,
Peter_______________________________________________
Biopython mailing list=C2=A0 -=C2=A0 [email protected]
https://mailman.open-bio.org/mailman/listinfo/biopython
=20
=20
------=_Part_9228321_80885842.1738844950540
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<html><head></head><body><div class=3D"ydp67658d82yahoo-style-wrap" style=
=3D"font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:10px=
;"><div></div>
<div dir=3D"ltr" data-setdir=3D"false">> <span><div>This is with=
a FASTA input MSA (all sequences the same length with gap characters, and =
some pairs will have common gaps).</div></span></div><div dir=3D"ltr" data-=
setdir=3D"false"><br></div><div dir=3D"ltr" data-setdir=3D"false">Then you =
can use the `"fasta"` parser in `Bio.Align`, which parses the aligned seque=
nces in C, so it should be fast.</div><div dir=3D"ltr" data-setdir=3D"false=
">Then call the new `.counts` method in Biopython 1.86dev with ignore=
_sequences=3DTrue to calculate the number of insertions and deletions quick=
ly.</div><div dir=3D"ltr" data-setdir=3D"false">This won't give you the num=
ber of gap-against-gap alignments, but those are not meaningful anyway.</di=
v><div dir=3D"ltr" data-setdir=3D"false"><br></div><div dir=3D"ltr" data-se=
tdir=3D"false">-Michiel<br></div><div><br></div>
=20
</div><div id=3D"yahoo_quoted_8899767467" class=3D"yahoo_quoted">
<div style=3D"font-family:'Helvetica Neue', Helvetica, Arial, s=
ans-serif;font-size:13px;color:#26282a;">
=20
<div>
On Thursday, February 6, 2025 at 07:51:48 PM GMT+9,=
Peter Cock <[email protected]> wrote:
</div>
<div><br></div>
<div><br></div>
=20
=20
<div><div id=3D"yiv7863050864"><div><div dir=3D"ltr"><div>T=
his is with a FASTA input MSA (all sequences the same length with gap chara=
cters, and some pairs will have common gaps).</div><div><br clear=3D"none">=
</div><div>I have been loading this incrementally so only two sequences wer=
e in RAM at any point - and then switched to multiple threads (so at any po=
int with K threads only 2*K sequences were loaded).</div><div><br clear=3D"=
none"></div><div>The test case is only about 100MB on disk, 100 sequences e=
ach of 1 million base pairs, and takes about a minute or two (multi-threade=
d laptop).<br clear=3D"none"></div><div><br clear=3D"none"></div><div><div>=
I don't think this will be memory constrained, so working on the entire MSA=
with a single thread is fine (if faster).<br clear=3D"none"></div><div><br=
clear=3D"none"></div><div>Thank you,</div><div><br clear=3D"none"></div><d=
iv>Peter<br clear=3D"none"></div></div></div><br clear=3D"none"><div id=3D"=
yiv7863050864yqt97629" class=3D"yiv7863050864yqt1257871432"><div class=3D"y=
iv7863050864gmail_quote yiv7863050864gmail_quote_container"><div dir=3D"ltr=
" class=3D"yiv7863050864gmail_attr">On Thu, Feb 6, 2025 at 10:36=E2=80=AFAM=
Michiel de Hoon <<a rel=3D"nofollow noopener noreferrer" shape=3D"rect"=
ymailto=3D"mailto:[email protected]" target=3D"_blank" href=3D"mailto:mj=
[email protected]">[email protected]</a>> wrote:<br clear=3D"none"></d=
iv><blockquote style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(=
204,204,204);padding-left:1ex;" class=3D"yiv7863050864gmail_quote"><div><di=
v style=3D"font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-si=
ze:10px;"><div></div>
<div dir=3D"ltr">> <span>I have lots of pairs of pre-aligned seq=
uences (imported from an external MSA file),</span></div><div dir=3D"ltr"><=
span><br clear=3D"none"></span></div><div dir=3D"ltr"><span>In which format=
is your MSA file?</span></div><div dir=3D"ltr"><span><br clear=3D"none"></=
span></div><div dir=3D"ltr"><span>-Michiel<br clear=3D"none"></span></div><=
div><br clear=3D"none"></div>
=20
</div><div id=3D"yiv7863050864m_8475683859613579565yahoo_quoted_975=
3999089">
<div style=3D"font-family:Helvetica, Arial, sans-serif;font-siz=
e:13px;color:rgb(38,40,42);">
=20
<div>
On Thursday, January 30, 2025 at 11:59:33 PM GMT+9,=
Peter Cock <<a rel=3D"nofollow noopener noreferrer" shape=3D"rect" ymai=
lto=3D"mailto:[email protected]" target=3D"_blank" href=3D"mailto:p=
[email protected]">[email protected]</a>> wrote:
</div>
<div><br clear=3D"none"></div>
<div><br clear=3D"none"></div>
=20
=20
<div><div id=3D"yiv7863050864m_8475683859613579565yiv860322=
6725"><div dir=3D"ltr">Hello all, and Michiel in particular,<br clear=3D"no=
ne"><br clear=3D"none">I am wondering if any of the pairwise alignment code=
in Bio.Align (much of which is written in C for speed) could help with thi=
s use case?:<br clear=3D"none"><br clear=3D"none">I have lots of pairs of p=
re-aligned sequences (imported from an external MSA file), for which I am d=
oing something like this:<br clear=3D"none"><br clear=3D"none">```python<br=
clear=3D"none">def count_matches_etc(query_seq, subject_seq):<br clear=3D"=
none"> assert len(query_seq) =3D=3D len(subject_seq), "Should =
be same length"<br clear=3D"none"> matches =3D non_gap_mismatc=
hes =3D either_gapped =3D both_gapped =3D 0<br clear=3D"none"> =
for q, s in zip(query_seq, subject_seq, strict=3DTrue):<br clear=3D"none">=
if q =3D=3D "-" and s =3D=3D "-":<br clear=3D"n=
one"> both_gapped +=3D 1<br clear=
=3D"none"> elif q =3D=3D "-" or s =3D=3D "-":<br=
clear=3D"none"> either_gapped +=
=3D 1<br clear=3D"none"> elif q =3D=3D s:<br cle=
ar=3D"none"> matches +=3D 1<br cle=
ar=3D"none"> else:<br clear=3D"none"> &nbs=
p; non_gap_mismatches +=3D 1<br clear=3D"none">=
assert matches + non_gap_mismatches + either_gapped + both_ga=
pped =3D=3D len(query_seq)<br clear=3D"none"> return matches, =
non_gap_mismatches, either_gapped, both_gapped<br clear=3D"none"><br clear=
=3D"none"><br clear=3D"none"># Test case<br clear=3D"none">assert (9, 1, 2,=
1) =3D=3D count_matches_etc("ACGTAC-TAC-GT", "AGGT-CGTAC-GT")<br clear=3D"=
none">```<br clear=3D"none"><br clear=3D"none">Sticking with Python that co=
uld be optimized (e.g. I am currently using this with sequences of a millio=
n base pairs but few gaps), however I have written this example with clarit=
y foremost in mind.<br clear=3D"none"><div><br clear=3D"none"></div><div>Th=
ank you,</div><div><br clear=3D"none"></div>Peter</div>
</div>_______________________________________________<br clear=3D"none">Bio=
python mailing list - <a rel=3D"nofollow noopener noreferrer" s=
hape=3D"rect" ymailto=3D"mailto:[email protected]" target=3D"_blank" =
href=3D"mailto:[email protected]">[email protected]</a><br clea=
r=3D"none"><a rel=3D"nofollow noopener noreferrer" shape=3D"rect" target=3D=
"_blank" href=3D"https://mailman.open-bio.org/mailman/listinfo/biopython">h=
ttps://mailman.open-bio.org/mailman/listinfo/biopython</a><br clear=3D"none=
"></div>
</div>
</div></div></blockquote></div></div>
</div></div></div>
</div>
</div></body></html>
------=_Part_9228321_80885842.1738844950540--
--===============6846260113852510385==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Biopython mailing list - [email protected]
https://mailman.open-bio.org/mailman/listinfo/biopython
--===============6846260113852510385==--