Re: COPY from STDIN vs file with large CSVs

"David G. Johnston" <[email protected]> Wed, 8 Jan 2020 10:33:51 -0800
Newsgroups gmane.comp.db.postgresql.admin
Message-ID <CAKFQuwZSUCOc9KdzJj6xWtf+9xqpE_hA-scGEnYh8qwMYv3Xqw@mail.gmail.com>
--00000000000056171e059ba52313
Content-Type: multipart/alternative; boundary="00000000000056171c059ba52312"

--00000000000056171c059ba52312
Content-Type: text/plain; charset="UTF-8"

On Wed, Jan 8, 2020 at 9:31 AM Indirani Venkopa Chetty <
[email protected]> wrote:

> Hi Wells,
>
> I am trying import CSV file in to Amazon RDS PostgreSQL
>
> I am getting following error , In internal PostgreSQL it works
>
>
>

https://www.postgresql.org/docs/12/sql-copy.html

"COPY with a file name instructs the PostgreSQL server to directly read
from or write to a file." -  i.e., Even had the server accepted your
command it wasn't going to find your file.

Use either psql's "\copy" meta command or "COPY FROM STDIN" (search around
for the needed syntax to make that work).

David J.

--00000000000056171c059ba52312
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div class=3D"gmail_default" style=3D"fon=
t-family:arial,helvetica,sans-serif"><span style=3D"font-family:Arial,Helve=
tica,sans-serif">On Wed, Jan 8, 2020 at 9:31 AM Indirani Venkopa Chetty &lt=
;<a href=3D"mailto:[email protected]">[email protected]</a>=
&gt; wrote:</span><br></div></div><div class=3D"gmail_quote"><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid=
 rgb(204,204,204);padding-left:1ex">





<div lang=3D"EN-US">
<div class=3D"gmail-m_1249238387943332423WordSection1">
<p class=3D"MsoNormal"><span style=3D"font-size:11pt;font-family:Calibri,sa=
ns-serif;color:rgb(31,73,125)">Hi Wells,<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11pt;font-family:Calibri,sa=
ns-serif;color:rgb(31,73,125)">I am trying import CSV file in to Amazon RDS=
 PostgreSQL<u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11pt;font-family:Calibri,sa=
ns-serif;color:rgb(31,73,125)">I am getting following error , In internal P=
ostgreSQL it works<u></u><u></u></span></p>
<p class=3D"MsoNormal"><img width=3D"727" height=3D"36" style=3D"width: 7.5=
729in; height: 0.375in;" id=3D"gmail-m_1249238387943332423Picture_x0020_1" =
src=3D"cid:16f866c1fae4ce8e91"><span style=3D"font-size:11pt;font-family:Ca=
libri,sans-serif;color:rgb(31,73,125)"><u></u><u></u></span></p>
<p class=3D"MsoNormal"><span style=3D"font-size:11pt;font-family:Calibri,sa=
ns-serif;color:rgb(31,73,125)"><u></u>=C2=A0</span></p></div></div></blockq=
uote><div><br></div><div class=3D"gmail_default" style=3D"font-family:arial=
,helvetica,sans-serif"><a href=3D"https://www.postgresql.org/docs/12/sql-co=
py.html" style=3D"font-family:Arial,Helvetica,sans-serif">https://www.postg=
resql.org/docs/12/sql-copy.html</a></div><div class=3D"gmail_default" style=
=3D"font-family:arial,helvetica,sans-serif"><br></div><div class=3D"gmail_d=
efault" style=3D"font-family:arial,helvetica,sans-serif">&quot;COPY with a =
file name instructs the PostgreSQL server to directly read from or write to=
 a file.&quot; -=C2=A0 i.e., Even had the server accepted your command it w=
asn&#39;t going to find your file.</div><div class=3D"gmail_default" style=
=3D"font-family:arial,helvetica,sans-serif"><br></div><div class=3D"gmail_d=
efault" style=3D"font-family:arial,helvetica,sans-serif">Use either psql&#3=
9;s &quot;\copy&quot; meta command or &quot;COPY FROM STDIN&quot; (search a=
round for the needed syntax to make that work).</div><div class=3D"gmail_de=
fault" style=3D"font-family:arial,helvetica,sans-serif"><br></div><div clas=
s=3D"gmail_default" style=3D"font-family:arial,helvetica,sans-serif">David =
J.</div><div class=3D"gmail_default" style=3D"font-family:arial,helvetica,s=
ans-serif"><br></div></div></div>

--00000000000056171c059ba52312--
--00000000000056171e059ba52313
Content-Type: image/jpeg; name="image001.jpg"
Content-Disposition: inline; filename="image001.jpg"
Content-Transfer-Encoding: base64
Content-ID: <16f866c1fae4ce8e91>
X-Attachment-Id: 16f866c1fae4ce8e91

/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAoHBwkHBgoJCAkLCwoMDxkQDw4ODx4WFxIZJCAmJSMg
IyIoLTkwKCo2KyIjMkQyNjs9QEBAJjBGS0U+Sjk/QD3/wAALCAAkAtcBAREA/8QAHwAAAQUBAQEB
AQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1Fh
ByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZ
WmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXG
x8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/9oACAEBAAA/APZqKKKKKKKKKKKKKKKK
KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKSu
Rn07UB4pjtl1/UFhdDKUG3A56dOlWr/U7qz8W28QmJs/I/eoR0ycBqS61K7TSNalW4KvA2I3GPkF
R6CkbzJOniifUGVNzwGRGHTuAM1Gh1W/s5dcj1GWIJuaKzAHllR/e754Na11r32Xw4mqLaTXJKqT
DCMsSeuK07af7TaxTbGTzFDbW6jPY1LRXNeLbqSG406Eam+mwzSMJJlKjoOBk8Vc0NI7azmnGsy6
nF97zHdW24HIG2kj8V6bPdQQQNNMZuFeOJmQH3YcCpL/AMS6dp0ssM0jtNGAWijjLtg98Cn3Gv2V
tpsd7M0ixy/cTyz5jfRetP0/W7PU7N7i3d9qAl1dCrrj1U81Vj8WaZNcwwwPNMZuFeKJmQH0LDgV
tVzniPUbyx1bTRbSlYiWaZMfeUVctL95dculaXNssCuoPQZ70lp4nsLy++yRi4WQkhWkhZVb6MRg
0248WaXa3TW8kkpkUkHbExGR2z61ZXXbJtKGoK7mAjgBDvJ9NvXPtS6VrdrrCObbzVKHlJoyjfXB
7VHZ60bvWrvTzZXEQtgCJ3X5JPoaiuvFWmWd41tLJL5inB2xMQD6Z9asW2u2N1bQTxyNsnk8tAyk
Hd6Edqs/bYPtjWu/98qbyMdqzY/Femy6gLNTPuLbRIYWEZP+90qtr3i+30XULe1eOdmd8OVgZhjH
Yjqa2NO1GHVLQXFusqoTjEsZQ/kasSkrE5HBCnFctofjC1+yW8OoTTtcOxDSmI7M5wMt0rb1PXLT
SlQz+bIW5CwxmRseuB2qzZXsOoWyz27Eo3YjBHsR2qfpWP8A8JVp39ofZMzlt23zBC3l5/3ulaKX
sL3jWqvmZUDkY7Gq0uu2ENk93LPtgR/LZiOjdMVHp3iOw1S6ktoGlWVOdssZTcPUZ6imXPifT7W/
FpJ55bIBkSFmjH1YcCrN9rVjppjF1MFMoJjAGS+PT1qG28RWN1ZG6HnxxBwn72JkOT04NXnvYY7y
O1ZsTSKWUY6gVm3finTrK9+yyGdnBwXjhZkU+7DgVb1DV7XTbZZpi7K33ViQuzfQDmks9ZtL6xe7
iZ1jQEuHQqy49VPNVYfFOm3N5Db27TTGb7rxxMU/FugrZrD8TeJYfD1qGdJWlf7m2JmX8SOlT6Z4
hs9TtJJ086JIhl2niMYH50zTvFGnapdG3gaVZP4fNiZA/wDuk9fwrYrF1q5upb220yxlEElwCzzd
WRB12+9RMt74d0u5kkvpb/j90ZwNwY9uO1U8aloZtb651Ka7iuHVJoZQNse7pswPX1qpf3Pm6vqI
n8Ty6eYGAit1kRQeM9DyavjUr+Cy07U7iRvI+5cpjA2no9X9LupdT1C4ukmb7HGfKjjA4Yjq1bFc
zrHjS10rV4bJorhssRIVt2bjHG0jrWo2u2SaWt/I0iQt91WQhyfQL1zS6VrlnrKMbVpAyH5klQow
/A81o1lap4jsNIlWK4aV5G52wxGQqPU46VPJrNjFpov3uFFqcfvOwz61FYeILPUryS2thOWQZ3NE
yofox4Nadcx4u8SJpcaWaPcxTzMAZYoC+1T1IPTNWNGvYLHQje3Oo3lxAW/1t4m1l7dMDirVl4js
b++a1t/OLKu7zDEwjI9Q3Q1GnivTH1D7GJJN27aJDGfLJ/3+lPvvE2naddi2uJJBJxnbGWAz0yR0
qzpmq22rwNLaM5VWKsHUqQfoauZxWQ3ijT01D7I32gNu2+Z5LeXn/exitfrUN7eR2Fq9xMHKIMkI
pY/kK4/QdYOv6wJxqOpxfOcWwgKw7R6kjr+NX7zXku9esrK0vJ7VkmxKj2zbZh/dDdKm8WRXcNp9
vtdSubYw4/dR42Pz3yK3FmWO0WWZgoCBmJ+lZ+n+JLHUrpreEXCOOnnQsgb6E9a1qKKKKKKKSuSn
vdUPiiO6XQLxoUQxbxInPP3sZ6Voz6ZLd+InmkiItpLPyy2ejZrNttH1FPDeqWk0ReaRtseSPnUH
r+Vb+n6TaWdsnlWdvBKYwrNHGFPTnkVhCLVrCzl0SGweaOTcsV4GGxFP94HnI56Vq3QvdG8OomlW
q3l1EqqsZbaG9TmtO2eWS1ieePy5WUF0znae4qWiub8V2s0txp08emHUYoJGMkI29xxw3FWNNkku
dLu4k0V9M+RgkZCAOSO22rPh2yaw0S3gkhWKRV+ZRjrUFlp8sfijULySEBJI0WOTjJx1FN1y2uo9
Qs9Ttbc3ZttwaBSAxyMZXPFQ2FreXk99qVxaNaPcQGJLdiC31JHGaveHbJtP0WGCSFYXGSyrjrWp
WNqdjNc6/p0yxb4I1kEhz0yOKz9O0jUbK51VVxtaPbayPyD35FZVvo97eaha3EumXkV1BKrSzzXG
UfHBKoDgflWv/ZV55sZ8jgX5mPI+7zzVLUvDt1daajyW7zGC6eU2yylGkU9ww6GrvhXTYYLia5Gk
XljMyhS1zceYWHtya07O91OXWru3ubBYrGMDyJw+TJ68dqzZNLu2u5XEHyteLJnI5UDrUUunX9vY
meO1aaaG/acQhhmRfY+tWtOXUbvWpb+4sfssL24RI5GBfd33Y4rINrqkN+sGmafeWaNMGmZnWS3k
GeeCcj8K3fEUF0Vsru0ga5ktJfMaFCAzjGOCavaXez39sZbmxmsnzjy5SCfrxVqUFonA6lSBXGW9
vqy6KuhtpDjzCd1zuXy0BbOSOuaNb8PSR6ml41jdajGYViC29wYnQj8QCK6Dw3Yw2GlhYbKWzDsW
aKWTe2fc5NarZ2nGCcd64m4ttUhvmj0zT7y1eSXMsiur28n/AAFjkD6Vq3jahputrdw6dJepNCsT
+SwHlsD157Vk6la3lp4QlF7boZ2uhIIlOQQWyB9a0rZb7WdahurjTpdOitUZQZWUs5PpjtWEfDX2
W5ltbvRb69WSQsbiC7KxsCf4l3Dn1rprvS3k1nR5I7cG3tVbO7B2ccVZ8RWs93pTC1TzJUZXVM43
YOcVQtG1LU9ctb2fTms7eOJlZJWBfcT7cYrO1e21GK/nbTdOvYbmYjNzbupiYe6McfXipNf0K5uX
sb2e1kvTDHskgt5jEwJ7qQR+NWdH0pIdEv8A7NplzZT3CMClxN5jOccc5Na+hWhsdFtrdolidEG5
VHQ960KzPEFnNe6PNFbjMvDKufvEdqzna/8AEGjz2k2mz2EiqNpnZSshHbg9KhRdR1q5sYp9LksI
7JwzSyMpDYGMJj+tdVWNrVrdpeW2p2EazS24KvCeC6HqAfWoSb/xDplzFcWD6fkYjErAtuHc47VV
KanrT21hdae9rBbMrSzOwIl29NuOevrV2z0SB9U1C4vrC3kMkqtE8kascAe9aOpW32nS7i3EYffG
VCetQ6BaPY6HaW8sflyRphl9DWjWF4hivI7uxv7O1ku/szNuhjYBiCMcZ4qG/W+1G2sdSTT5I5rS
XzDaSMN7DpwRxmn6XFeajrX9qXVk9gqRmJYpCC7+5xxiugrm7gX+iavdXVvp0moR3ZB/dMA0ZHY5
7VXl0G7HhJ7Qwo8804lMQ6KC2cV1UahI1VQFAAAA6CnVma1bT3K2ot1LbJ1ZsHGAO9N8S2ct/okt
vBH5jsy/L6jIzVxrXfphtlAjzFsAA4XjFct9m1OXS08PnS2RVwpvSw8vAOcgdc1bvNJu3muvLiLq
5hCnI+YKea0tHtJ7e71F5k2rLMGQ56jGK03DGNguNxBxnpmvP73RtR1HzYrrTbyS8VsrcmfbCQDx
hAfT1Fd/ECsKBuoUA0+szQLae008x3KlX8xzgnPBNJq9rPcXWnNCm5YrgPJz0GOtL4hs5b/RbiC3
GZSMqvqR2rNuYbzxH4bmtJrGawlGAEmcfvMdsr0BrP0DR4l1SB5tB1C2kgHyzTXZkQHpwNxrtaKK
KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKrX1hBqMAhuVLIGDYBxyKs0UUUUUUUUUUU
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUlRJKzQlyBkUy8uXt7cSIFJPrTZ7p4rNJVC7m
xwelQQajLLOiMqYPoD/jSSalKkzIFTAbHQ/41duJmitTIoG4DPPSo4Ll5bIzMF3c8DpVe11GWe4E
bKgB9Af8a0qKKKzrvUJbecoioR7g1dhcyQq5xkjtUlFFFFFFRXEhigd1xlRnmm2czXFuJHABPpU9
FFFFFFM3nzQvGMZp9FFFFFFFFFFFUPt0n24Q7U25xnHNX6KKKKhupmggZ1AJHrVGLU5nlRSseG64
B/xrQmkMYBGOTjmpKKKKpX15JbMgQKd3qKbbX0kzOGVBtGRgGrqncoPrTqKQ9KzBqU32gJtTG7HQ
/wCNalFFFFFFVFu3a/MGF2gde9W6KKKKKKbIxVCw6ilU5UH1paKKKKKKKKK//9k=
--00000000000056171e059ba52313--