Re: Adding overall coverage to blast-xml
John T <[email protected]> Wed, 13 Sep 2017 11:09:54 +0100
| Newsgroups | gmane.comp.python.bio.devel |
|---|---|
| Message-ID | <CAL0Rc+zwoB95cz6EG1_gJSWhRT3JfvOwhLA6o8FXcgd2jLjf8Q@mail.gmail.com> |
--===============8990771939142299645== Content-Type: multipart/alternative; boundary="94eb2c0df6c488df6105590f5c64" --94eb2c0df6c488df6105590f5c64 Content-Type: text/plain; charset="UTF-8" Hi Bow & Peter, That seems fair enough, I'll have a look if `HSP.hit_span_all` and `HSP.query_span_all` are the same as qcov* and add my thing to the cook book if they aren't. Cheers, John On 12 September 2017 at 11:12, Wibowo Arindrarto <[email protected]> wrote: > Hi John, Peter, > > Thanks for looking into this. Indeed the `qcov` and `qcovhsp` values are > already captured when parsing `blast-tab` formats with SearchIO. They are > accessible via the `Hit.query_coverage` and `HSP.query_coverage` attributes > respectively. > > For non-`blast-tab` formats such as `blast-xml`, there may already be a > similar attribute you can use in SearchIO. They are the `HSP.hit_span_all` > and `HSP.query_span_all` attributes, which calculates how many bases are > covered by the hit and query, respectively. You can find more about them > here: http://biopython.org/DIST/docs/api/Bio.SearchIO._model. > hsp.HSP-class.html. However I have to say I don't recall if these values > are always the same as the qcov values BLAST outputs when using the > `blast-tab` format. Maybe that is worth checking first. > > That being said, if they do not end up being always equal (i.e. there are > corner cases in BLAST that those attributes do not properly capture), I > also am quite hesitant to add more derived values for specific file > formats. The main reason is I feel this would make the current object model > more brittle. We are already quite brittle in some respects, because of the > fact that the formats we are parsing do not have precise specifications. > > I am very open to having this put in our cookbook ( > http://biopython.org/wiki/Category%3ACookbook), however, since this would > be more akin to a BLAST-specific recipe that many would find useful. > > Let me know what you think. > > Cheers, > Bow > > > > > > -------- Original Message -------- > Subject: Re: [Biopython-dev] Adding overall coverage to blast-xml > Local Time: September 12, 2017 11:28 AM > UTC Time: September 12, 2017 9:28 AM > From: [email protected] > To: John T <[email protected]> > [email protected] <[email protected]>, Biopython-Dev Mailing List < > [email protected]> > > Hi John, > > At least if its qcovs (or qcovhsp) there is a clear base value > from the NCBI code for guidance and conformance testing. > > I'm still a little uncomfortable with a parser inferring to many > advanced values like this, but will defer to Bow as the > SearchIO author to comment on this might fit into his > object model (e.g. does this have parallels in the other > supported formats?). > > (I'm pretty sure this general topic has come up in the past, > but couldn't immediately find an old issue or email thread.) > > Bow? > > Peter > > On Mon, Sep 11, 2017 at 10:02 PM, John T <[email protected]> wrote: > >> It is indeed qcovs, I was stuck using XML, didn't investigate that >> before. I haven't looked into the SearchIO version yet, but I assume it >> won't be more difficult to implement there so I'm happy to commit to that. >> >> John >> >> On 11 September 2017 at 21:14, Peter Cock <[email protected]> >> wrote: >> >>> Hi John, CC Bow, >>> >>> Is this information in any of the BLAST output formats? >>> >>> I suspect you are talking about recreating one or more >>> of the coverage fields which can be requested in the >>> BLAST csv or tabular output files formats? i.e. qcovs >>> or qcovhsp perhaps? >>> >>> If so, I can see a rational for computing this for the >>> BLAST XML parsing... although given we'd like to >>> push people towards SearchIO (which supports the >>> BLAST tabular format as well), doing it there might >>> make more sense than in NCBIXML? >>> >>> Peter >>> >>> >>> On Mon, Sep 11, 2017 at 5:40 PM, John T <[email protected]> wrote: >>> > Hi all, >>> > >>> > I've written a script that calculates the overall coverage for all HSP >>> in an >>> > alignment, like that given on the BLAST homepage. Would it make sense >>> to add >>> > this to Bio.Blast.Record.Alignment (and/or the BlastIO equivalent)? >>> It's not >>> > something that's explicitly defined in the XML and so may be out of >>> scope, >>> > but it is something that's given in the website version of BLAST >>> results so >>> > people might reasonably expect it to be there. I'd add it as a >>> @property so >>> > it as only calculated when required. >>> > >>> > I've never tried contributing to an open source project before so I >>> didn't >>> > want to dive in and do something stupid. >>> > >>> > Cheers, >>> > John >>> > >>> > >>> > _______________________________________________ >>> > Biopython-dev mailing list >>> > [email protected] >>> > http://mailman.open-bio.org/mailman/listinfo/biopython-dev >>> >> > --94eb2c0df6c488df6105590f5c64 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Hi Bow & Peter,<div><br></div><div>That seems fair eno= ugh, I'll have a look if=C2=A0<span style=3D"font-size:12.8px">`HSP.hit= _span_all` and `HSP.query_span_all` are the same as qcov* and add my thing = to the cook book if they aren't.</span></div><div><span style=3D"font-s= ize:12.8px"><br></span></div><div><span style=3D"font-size:12.8px">Cheers,<= /span></div><div><span style=3D"font-size:12.8px">John</span></div></div><d= iv class=3D"gmail_extra"><br><div class=3D"gmail_quote">On 12 September 201= 7 at 11:12, Wibowo Arindrarto <span dir=3D"ltr"><<a href=3D"mailto:bow@b= ow.web.id" target=3D"_blank">[email protected]</a>></span> wrote:<br><block= quote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc= solid;padding-left:1ex"><div>Hi John, Peter,<br></div><div><br></div><div>= Thanks for looking into this. Indeed the `qcov` and `qcovhsp` values are al= ready captured when parsing `blast-tab` formats with SearchIO. They are acc= essible via the `Hit.query_coverage` and `HSP.query_coverage` attributes re= spectively.<br></div><div><br></div><div>For non-`blast-tab` formats such a= s `blast-xml`, there may already be a similar attribute you can use in Sear= chIO. They are the `HSP.hit_span_all` and `HSP.query_span_all` attributes, = which calculates how many bases are covered by the hit and query, respectiv= ely. You can find more about them here: <a href=3D"http://biopython.org/DIS= T/docs/api/Bio.SearchIO._model.hsp.HSP-class.html" target=3D"_blank">http:/= /biopython.org/DIST/<wbr>docs/api/Bio.SearchIO._model.<wbr>hsp.HSP-class.ht= ml</a>. However I have to say I don't recall if these values are always= the same as the qcov values BLAST outputs when using the `blast-tab` forma= t. Maybe that is worth checking first.<br></div><div><br></div><div>That be= ing said, if they do not end up being always equal (i.e. there are corner c= ases in BLAST that those attributes do not properly capture), I also am qui= te hesitant to add more derived values for specific file formats. The main = reason is I feel this would make the current object model more brittle. We = are already quite brittle in some respects, because of the fact that the fo= rmats we are parsing do not have precise specifications.<br></div><div><br>= </div><div>I am very open to having this put in our cookbook (<a href=3D"ht= tp://biopython.org/wiki/Category%3ACookbook" target=3D"_blank">http://biopy= thon.org/wiki/<wbr>Category%3ACookbook</a>), however, since this would be m= ore akin to a BLAST-specific recipe that many would find useful.<br></div><= div><br></div><div>Let me know what you think.<br></div><div><br></div><div= >Cheers,<br></div><div>Bow<br></div><div class=3D"HOEnZb"><div class=3D"h5"= ><div><br></div><div><br></div><div class=3D"m_-1494729809966613732protonma= il_signature_block m_-1494729809966613732protonmail_signature_block-empty">= <div class=3D"m_-1494729809966613732protonmail_signature_block-user m_-1494= 729809966613732protonmail_signature_block-empty"><br></div><div class=3D"m_= -1494729809966613732protonmail_signature_block-proton m_-149472980996661373= 2protonmail_signature_block-empty"><br></div></div><div><br></div><blockquo= te class=3D"m_-1494729809966613732protonmail_quote" type=3D"cite"><div>----= ---- Original Message --------<br></div><div>Subject: Re: [Biopython-dev] A= dding overall coverage to blast-xml<br></div><div>Local Time: September 12,= 2017 11:28 AM<br></div><div>UTC Time: September 12, 2017 9:28 AM<br></div>= <div>From: <a href=3D"mailto:[email protected]" target=3D"_blank">p= [email protected]</a><br></div><div>To: John T <<a href=3D"mailto= :[email protected]" target=3D"_blank">[email protected]</a>><br></div>= <div><a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>= <<a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>= >, Biopython-Dev Mailing List <<a href=3D"mailto:biopython-dev@mailma= n.open-bio.org" target=3D"_blank">[email protected]<wbr>bio.org</= a>><br></div><div><br></div><div dir=3D"ltr"><div>Hi John,<br></div><div= ><br></div><div><div>At least if its qcovs (or qcovhsp) there is a clear ba= se value<br></div><div>from the NCBI code for guidance and conformance test= ing.<br></div></div><div><br></div><div>I'm still a little uncomfortabl= e with a parser inferring to many<br></div><div>advanced values like this, = but will defer to Bow as the<br></div><div>SearchIO author to comment on th= is might fit into his<br></div><div>object model (e.g. does this have paral= lels in the other<br></div><div>supported formats?).<br></div><div><br></di= v><div><div>(I'm pretty sure this general topic has come up in the past= ,<br></div><div>but couldn't immediately find an old issue or email thr= ead.)<br></div></div><div><br></div><div>Bow?<br></div><div><br></div><div>= Peter<br></div></div><div class=3D"gmail_extra"><div><br></div><div class= =3D"gmail_quote"><div>On Mon, Sep 11, 2017 at 10:02 PM, John T <span dir=3D= "ltr"><<a href=3D"mailto:[email protected]" target=3D"_blank">jaytee00@= gmail.com</a>></span> wrote:<br></div><blockquote style=3D"margin:0 0 0 = .8ex;border-left:1px #ccc solid;padding-left:1ex" class=3D"gmail_quote"><di= v dir=3D"ltr"><div>It is indeed qcovs, I was stuck using XML, didn't in= vestigate that before. I haven't looked into the SearchIO version yet, = but I assume it won't be more difficult to implement there so I'm h= appy to commit to that.<span class=3D"m_-1494729809966613732HOEnZb"><span c= lass=3D"m_-1494729809966613732colour" style=3D"color:#888888"></span></span= ><br></div><span class=3D"m_-1494729809966613732HOEnZb"><span class=3D"m_-1= 494729809966613732colour" style=3D"color:#888888"><div><br></div><div>John<= br></div></span></span></div><div class=3D"m_-1494729809966613732HOEnZb"><d= iv class=3D"m_-1494729809966613732h5"><div class=3D"gmail_extra"><div><br><= /div><div class=3D"gmail_quote"><div>On 11 September 2017 at 21:14, Peter C= ock <span dir=3D"ltr"><<a href=3D"mailto:[email protected]" targ= et=3D"_blank">[email protected]</a>></span> wrote:<br></div><blo= ckquote style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:= 1ex" class=3D"gmail_quote"><div>Hi John, CC Bow,<br></div><div> <br></div><= div> Is this information in any of the BLAST output formats?<br></div><div>= <br></div><div> I suspect you are talking about recreating one or more<br>= </div><div> of the coverage fields which can be requested in the<br></div><= div> BLAST csv or tabular output files formats? i.e. qcovs<br></div><div> o= r qcovhsp perhaps?<br></div><div> <br></div><div> If so, I can see a ration= al for computing this for the<br></div><div> BLAST XML parsing... although = given we'd like to<br></div><div> push people towards SearchIO (which s= upports the<br></div><div> BLAST tabular format as well), doing it there mi= ght<br></div><div> make more sense than in NCBIXML?<br></div><div> <br></di= v><div> Peter<br></div><div> <br></div><div><div class=3D"m_-14947298099666= 13732m_9172762734742828488h5"><div><br></div><div>On Mon, Sep 11, 2017 at 5= :40 PM, John T <<a href=3D"mailto:[email protected]" target=3D"_blank">= [email protected]</a>> wrote:<br></div><div> > Hi all,<br></div><div= > ><br></div><div> > I've written a script that calculates the ov= erall coverage for all HSP in an<br></div><div> > alignment, like that g= iven on the BLAST homepage. Would it make sense to add<br></div><div> > = this to Bio.Blast.Record.Alignment (and/or the BlastIO equivalent)? It'= s not<br></div><div> > something that's explicitly defined in the XM= L and so may be out of scope,<br></div><div> > but it is something that&= #39;s given in the website version of BLAST results so<br></div><div> > = people might reasonably expect it to be there. I'd add it as a @propert= y so<br></div><div> > it as only calculated when required.<br></div><div= > ><br></div><div> > I've never tried contributing to an open sou= rce project before so I didn't<br></div><div> > want to dive in and = do something stupid.<br></div><div> ><br></div><div> > Cheers,<br></d= iv><div> > John<br></div><div> ><br></div><div> ><br></div></div><= /div><div>> ______________________________<wbr>_________________<br></di= v><div> > Biopython-dev mailing list<br></div><div> > <a href=3D"mail= to:[email protected]" target=3D"_blank">Biopython-dev@mail= man.open-bio<wbr>.org</a><br></div><div> > <a rel=3D"noreferrer" href=3D= "http://mailman.open-bio.org/mailman/listinfo/biopython-dev" target=3D"_bla= nk">http://mailman.open-bio.org/ma<wbr>ilman/listinfo/biopython-dev</a><br>= </div></blockquote></div></div></div></div></blockquote></div></div></block= quote><div><br></div></div></div></blockquote></div><br></div> --94eb2c0df6c488df6105590f5c64-- --===============8990771939142299645== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Biopython-dev mailing list [email protected] http://mailman.open-bio.org/mailman/listinfo/biopython-dev --===============8990771939142299645==--