Re: [Biopython] pd.df.to_csv(..., compression="bgzip")?
Dan Bolser <[email protected]> Wed, 13 Mar 2024 11:22:57 +0000
| Newsgroups | gmane.comp.python.bio.general |
|---|---|
| Message-ID | <CANs1yPKQ1C7tDnR6NcF1q8WaBqatmTW7EMqVbAWy_FBZGBcmEg@mail.gmail.com> |
--===============7842720590585068867== Content-Type: multipart/alternative; boundary="000000000000e49a04061388ff0b" --000000000000e49a04061388ff0b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Nice idea, I would never have thought of that. Thanks Peter! On Wed, Mar 13, 2024, 11:18 AM Peter Cock <[email protected]> wrote= : > Ah. I would give it a file handle then: > > with bgzf.open("example.txt.bgz", "w") as bgzf_handle: > my_data_frame.to_csv(bgzf_handle, ...) > > I would expect that to work according to > > https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFra= me.to_csv.html > - possibly with an explicit compression=3DNone added? > > Peter > > > On Wed, Mar 13, 2024 at 11:08=E2=80=AFAM Dan Bolser <[email protected]= .uk> > wrote: > > > > pandas.to_csv is the function that writes data. pandas.read_csv silentl= y > handles decompression as needed. > > > > > > > > On Wed, Mar 13, 2024, 10:49 AM Peter Cock <[email protected]> > wrote: > >> > >> Yes. BGZF is just a special kind of GZIP file, if all you are doing is > >> decompressing it for reading it then the standard gzip.open(...) > >> is fine. > >> > >> Peter > >> > >> > >> On Wed, Mar 13, 2024 at 10:03=E2=80=AFAM Dan Bolser <dan.bolser@outsee= .co.uk> > wrote: > >>> > >>> bgzip is a 'bio' thing, so thought I'd ask here. It's perhaps not > 'biopython', but it's bio/python. > >>> > >>> On Tue, 12 Mar 2024 at 19:11, Sean Brimer <[email protected]> wrote: > >>>> > >>>> Hi Dan, > >>>> > >>>> This feels more like a panda's issue than a biopython issue. That > said, I think you could just use gzip. I think. bgzip for samtools was > built on top of gzip so it probably decompresses in a similar way. > >>>> > >>>> On Tue, Mar 12, 2024 at 12:52=E2=80=AFPM Dan Bolser <dan.bolser@outs= ee.co.uk> > wrote: > >>>>> > >>>>> Hi, > >>>>> > >>>>> I can pass `compression=3D"gzip"` to pandas.DataFrame.to_csv, but n= ot > bgzip... how to update pandas to support bgzip? > >>>>> > >>>>> > >>>>> Thanks, > >>>>> _______________________________________________ > >>>>> Biopython mailing list - [email protected] > >>>>> https://mailman.open-bio.org/mailman/listinfo/biopython > >>> > >>> _______________________________________________ > >>> Biopython mailing list - [email protected] > >>> https://mailman.open-bio.org/mailman/listinfo/biopython > --000000000000e49a04061388ff0b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto">Nice idea, I would never have thought of that.<div dir=3D= "auto"><br></div><div dir=3D"auto">Thanks Peter!</div></div><br><div class= =3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Mar 13, 2024= , 11:18 AM Peter Cock <<a href=3D"mailto:[email protected]">p.j.= [email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_qu= ote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex= ">Ah. I would give it a file handle then:<br> <br> with bgzf.open("example.txt.bgz", "w") as bgzf_handle:<= br> =C2=A0 =C2=A0 my_data_frame.to_csv(bgzf_handle, ...)<br> <br> I would expect that to work according to<br> <a href=3D"https://pandas.pydata.org/pandas-docs/stable/reference/api/panda= s.DataFrame.to_csv.html" rel=3D"noreferrer noreferrer" target=3D"_blank">ht= tps://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.t= o_csv.html</a><br> - possibly with an explicit compression=3DNone added?<br> <br> Peter<br> <br> <br> On Wed, Mar 13, 2024 at 11:08=E2=80=AFAM Dan Bolser <<a href=3D"mailto:d= [email protected]" target=3D"_blank" rel=3D"noreferrer">dan.bolser@out= see.co.uk</a>> wrote:<br> ><br> > pandas.to_csv is the function that writes data. pandas.read_csv silent= ly handles decompression as needed.<br> ><br> ><br> ><br> > On Wed, Mar 13, 2024, 10:49 AM Peter Cock <<a href=3D"mailto:p.j.a.= [email protected]" target=3D"_blank" rel=3D"noreferrer">p.j.a.cock@google= mail.com</a>> wrote:<br> >><br> >> Yes. BGZF is just a special kind of GZIP file, if all you are doin= g is<br> >> decompressing it for reading it then the standard gzip.open(...)<b= r> >> is fine.<br> >><br> >> Peter<br> >><br> >><br> >> On Wed, Mar 13, 2024 at 10:03=E2=80=AFAM Dan Bolser <<a href=3D= "mailto:[email protected]" target=3D"_blank" rel=3D"noreferrer">dan.b= [email protected]</a>> wrote:<br> >>><br> >>> bgzip is a 'bio' thing, so thought I'd ask here. I= t's perhaps not 'biopython', but it's bio/python.<br> >>><br> >>> On Tue, 12 Mar 2024 at 19:11, Sean Brimer <<a href=3D"mailt= o:[email protected]" target=3D"_blank" rel=3D"noreferrer">[email protected]= om</a>> wrote:<br> >>>><br> >>>> Hi Dan,<br> >>>><br> >>>> This feels more like a panda's issue than a biopython = issue. That said, I think you could just use gzip. I think. bgzip for samto= ols was built on top of gzip so it probably decompresses in a similar way.<= br> >>>><br> >>>> On Tue, Mar 12, 2024 at 12:52=E2=80=AFPM Dan Bolser <<a= href=3D"mailto:[email protected]" target=3D"_blank" rel=3D"noreferre= r">[email protected]</a>> wrote:<br> >>>>><br> >>>>> Hi,<br> >>>>><br> >>>>> I can pass `compression=3D"gzip"` to pandas.= DataFrame.to_csv, but not bgzip... how to update pandas to support bgzip?<b= r> >>>>><br> >>>>><br> >>>>> Thanks,<br> >>>>> _______________________________________________<br> >>>>> Biopython mailing list=C2=A0 -=C2=A0 <a href=3D"mailto= :[email protected]" target=3D"_blank" rel=3D"noreferrer">Biopython@bi= opython.org</a><br> >>>>> <a href=3D"https://mailman.open-bio.org/mailman/listin= fo/biopython" rel=3D"noreferrer noreferrer" target=3D"_blank">https://mailm= an.open-bio.org/mailman/listinfo/biopython</a><br> >>><br> >>> _______________________________________________<br> >>> Biopython mailing list=C2=A0 -=C2=A0 <a href=3D"mailto:Biopyth= [email protected]" target=3D"_blank" rel=3D"noreferrer">Biopython@biopython.= org</a><br> >>> <a href=3D"https://mailman.open-bio.org/mailman/listinfo/biopy= thon" rel=3D"noreferrer noreferrer" target=3D"_blank">https://mailman.open-= bio.org/mailman/listinfo/biopython</a><br> </blockquote></div> --000000000000e49a04061388ff0b-- --===============7842720590585068867== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Biopython mailing list - [email protected] https://mailman.open-bio.org/mailman/listinfo/biopython --===============7842720590585068867==--