COPY from STDIN vs file with large CSVs
Wells Oliver <[email protected]> Wed, 8 Jan 2020 08:54:14 -0800
| Newsgroups | gmane.comp.db.postgresql.admin |
|---|---|
| Message-ID | <CAOC+FBVjk4W=nhn3FZUiTgroFFhaQ6bvh-NiU207PhD9nO3Qww@mail.gmail.com> |
--000000000000d560b7059ba3c02c Content-Type: text/plain; charset="UTF-8" I have a CSV that's ~30GB. Some 400m rows. Would there be a meaningful performance difference to run COPY from STDIN using: cat f.csv | psql "COPY .. FROM STDIN WITH CSV" versus just doing "COPY ... FROM 'f.csv' WITH CSV"? Thanks. It took about four hours to copy one and I felt that was a little much. -- Wells Oliver [email protected] <[email protected]> --000000000000d560b7059ba3c02c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>I have a CSV that's ~30GB. Some 400m rows. Would = there be a meaningful performance difference to run COPY from STDIN using: = cat f.csv | psql "COPY .. FROM STDIN WITH CSV" versus just doing = "COPY ... FROM 'f.csv' WITH CSV"?</div><div><br></div><di= v>Thanks. It took about four hours to copy one and I felt that was a little= much.<br></div><div><br>-- <br><div dir=3D"ltr" class=3D"gmail_signature" = data-smartmail=3D"gmail_signature"><div dir=3D"ltr"><div>Wells Oliver<br><a= href=3D"mailto:[email protected]" target=3D"_blank">wells.oliver@gmail= .com</a></div></div></div></div></div> --000000000000d560b7059ba3c02c--