Re: ChatGPT for py teaching
Wes Turner <[email protected]> Wed, 4 Jan 2023 17:13:37 -0500
| Newsgroups | gmane.comp.python.education |
|---|---|
| Message-ID | <CACfEFw_g_xTKAjwPxWJ07177NpX-P-phpg8mO1LQ+_HB+wYFWg@mail.gmail.com> |
--===============0726037330301891171== Content-Type: multipart/alternative; boundary="000000000000ab946505f1777f41" --000000000000ab946505f1777f41 Content-Type: text/plain; charset="UTF-8" Should we expect the output of [ChatGPT] to be stable or deterministic given the same prompt? - Does [ChatGPT] "converge" on the same solutions, given the same inputs? Where is there additional entropy in the algorithm or implementation? - random seed(s) - Hash randomization - distributed system failure - "Which data series predict recession, and with what confidence?" - Known good: Bond Yield-Curve Inversion - "Which economic interventions are appropriate for the current conditions?" #EvidenceBasedPolicy On Wed, Jan 4, 2023, 4:34 PM Wes Turner <[email protected]> wrote: > What are the expected limitations of [ChatGPT]? > > What is "Prompt Engineering"? > [Prompt engineering - Wikipedia]( > https://en.wikipedia.org/wiki/Prompt_engineering ) > > What lessons about technology reliance could you teach, in regards to > Clippy? > > - "What is ChatGPT? Wrong answers only" > - Human_n: EDGES WITH REASONING > > - "Tell me IDK ("I don't know") when you don't know" > > - "How certain are you that that is the correct answer?" > > - "Are static analysis code metrics sufficient for Safety Critical code?" > > - "Whose code is this based on?" > > - "Where and when did you learn this?" > > - "Why would a US President abstain from using ChatGPT or similar to fill > speeches 'just like what I said before'?" > > #Burgundy > > GPT or similar trained on only Formally-Verified code with associated tests > and/or e.g. Lean Mathlib, or e.g. the Principia in SymPy & Cirq; that > could probably eliminate my job, but maybe still not teaching > > On Wed, Jan 4, 2023, 6:28 AM Christian Mascher <[email protected]> > wrote: > >> Hi, >> >> a student of mine was aware of this chatbot and asked it about a >> class-assignment of his own accord. We program in Java with some extra >> homemade library class used by some schools in our region. >> >> The bot came up with a "solution" which was flawed in several respects: >> 1. It used some other (unimported) classes - solution doesn't work and >> doesn't fit the assignment. >> 2. It put all the code into the constructor, a typical (design and >> style) error for students beginning with Java. >> >> When confronted with the problem number one above, it acknowledged the >> fault and produced a different unrelated solution. >> >> Sooo.... >> >> I was impressed how well the chatbot simulated a typical clueless human >> who even thinks he is smart, while his code is basically bullshit. >> (Probably a result of googling forums, where other learners posted their >> solutions to assignments with the given school library classes.) The bot >> clearly passed the Turing test ;-) >> >> But... >> >> I don't think the interaction was helpful for somebody who is learning >> to program. It is probably less helpful than conversing with other also >> not very knowledgeable students as they are at least reasoning humans. >> >> Talking to the bot might be fun to do in the last lesson before >> christmas or so. Entertaining until you realise the software is >> "simulating" intelligent conversation - not really talking with insight. >> And that could turn out to be a waste of time. >> >> Happy new year >> >> Christian >> >> Am 03.01.2023 um 04:06 schrieb Jurgis Pralgauskis: >> > Hi, happy NY! >> > >> > ChatGPT can create, fix and explain code >> > https://openai.com/blog/chatgpt/#samples >> > <https://openai.com/blog/chatgpt/#samples> >> > >> > Anyone tried to incorporate it into teaching process? >> > Or have ideas/doubts how it ciuld help? >> > >> > _______________________________________________ >> > Edu-sig mailing list -- [email protected] >> > To unsubscribe send an email to [email protected] >> > https://mail.python.org/mailman3/lists/edu-sig.python.org/ >> > Member address: [email protected] >> _______________________________________________ >> Edu-sig mailing list -- [email protected] >> To unsubscribe send an email to [email protected] >> https://mail.python.org/mailman3/lists/edu-sig.python.org/ >> Member address: [email protected] >> > --000000000000ab946505f1777f41 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto">Should we expect the output of [ChatGPT] to be stable or = deterministic given the same prompt?=C2=A0<div dir=3D"auto"><br></div><div = dir=3D"auto">- Does [ChatGPT] "converge" on the same solutions, g= iven the same inputs? Where is there additional entropy in the algorithm or= implementation?</div><div dir=3D"auto">=C2=A0 - random seed(s)</div><div d= ir=3D"auto">=C2=A0 - Hash randomization</div><div dir=3D"auto">=C2=A0 - dis= tributed system failure<br><div dir=3D"auto"><br></div><div dir=3D"auto">- = "Which data series predict recession, and with what confidence?"<= /div><div dir=3D"auto">=C2=A0 - Known good: Bond Yield-Curve Inversion</div= ><div dir=3D"auto"><br></div><div dir=3D"auto">- "Which economic inter= ventions are appropriate for the current conditions?"</div><div dir=3D= "auto"><br></div><div dir=3D"auto">#EvidenceBasedPolicy</div></div></div><b= r><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, = Jan 4, 2023, 4:34 PM Wes Turner <<a href=3D"mailto:[email protected]"= >[email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_qu= ote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex= "><div dir=3D"auto">What are the expected limitations of [ChatGPT]?<div dir= =3D"auto"><br></div><div dir=3D"auto">What is "Prompt Engineering"= ;?</div><div dir=3D"auto">[Prompt engineering - Wikipedia]( <a href=3D"http= s://en.wikipedia.org/wiki/Prompt_engineering" target=3D"_blank" rel=3D"nore= ferrer">https://en.wikipedia.org/wiki/Prompt_engineering</a> )<br><div dir= =3D"auto"><br></div><div dir=3D"auto">What lessons about technology relianc= e could you teach, in regards to Clippy?</div><div dir=3D"auto"><br></div><= div dir=3D"auto">- "What is ChatGPT? Wrong answers only"</div><di= v dir=3D"auto">=C2=A0 - Human_n: EDGES WITH REASONING</div><div dir=3D"auto= "><br></div><div dir=3D"auto">- "Tell me IDK ("I don't know&q= uot;) when you don't know"</div><div dir=3D"auto"><br></div><div d= ir=3D"auto">- "How certain are you that that is the correct answer?&qu= ot;</div><div dir=3D"auto"><br></div><div dir=3D"auto">- "Are static a= nalysis code metrics sufficient for Safety Critical code?"</div><div d= ir=3D"auto"><br></div><div dir=3D"auto">- "Whose code is this based on= ?"</div><div dir=3D"auto"><br></div><div dir=3D"auto">- "Where an= d when did you learn this?"</div><div dir=3D"auto"><br></div><div dir= =3D"auto">- "Why would a US President abstain from using ChatGPT or si= milar to fill speeches 'just like what I said before'?"</div><= div dir=3D"auto"><br></div><div dir=3D"auto">#Burgundy</div><div dir=3D"aut= o"><br></div><div dir=3D"auto">GPT or similar trained on only Formally-Veri= fied code with associated tests</div><div dir=3D"auto">and/or e.g. Lean Mat= hlib, or e.g. the Principia in SymPy & Cirq; that could probably elimin= ate my job, but maybe still not teaching=C2=A0</div></div></div><br><div cl= ass=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Jan 4, 20= 23, 6:28 AM Christian Mascher <<a href=3D"mailto:[email protected]= e" rel=3D"noreferrer noreferrer" target=3D"_blank">[email protected]= </a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:= 0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br> <br> a student of mine was aware of this chatbot and asked it about a<br> class-assignment of his own accord. We program in Java with some extra<br> homemade library class used by some schools in our region.<br> <br> The bot came up with a "solution" which was flawed in several res= pects:<br> 1. It used some other (unimported) classes - solution doesn't work and<= br> doesn't fit the assignment.<br> 2. It put all the code into the constructor, a typical (design and<br> style) error for students beginning with Java.<br> <br> When confronted with the problem number one above, it acknowledged the<br> fault and produced a different unrelated solution.<br> <br> Sooo....<br> <br> I was impressed how well the chatbot simulated a typical clueless human<br> who even thinks he is smart, while his code is basically bullshit.<br> (Probably a result of googling forums, where other learners posted their<br= > solutions to assignments with the given school library classes.) The bot<br= > clearly passed the Turing test ;-)<br> <br> But...<br> <br> I don't think the interaction was helpful for somebody who is learning<= br> to program. It is probably less helpful than conversing with other also<br> not very knowledgeable students as they are at least reasoning humans.<br> <br> Talking to the bot might be fun to do in the last lesson before<br> christmas or so. Entertaining until you realise the software is<br> "simulating" intelligent conversation - not really talking with i= nsight.<br> And that could turn out to be a waste of time.<br> <br> Happy new year<br> <br> Christian<br> <br> Am 03.01.2023 um 04:06 schrieb Jurgis Pralgauskis:<br> > Hi, happy NY!<br> ><br> > ChatGPT can create, fix and explain code<br> > <a href=3D"https://openai.com/blog/chatgpt/#samples" rel=3D"noreferrer= noreferrer noreferrer noreferrer" target=3D"_blank">https://openai.com/blo= g/chatgpt/#samples</a><br> > <<a href=3D"https://openai.com/blog/chatgpt/#samples" rel=3D"norefe= rrer noreferrer noreferrer noreferrer" target=3D"_blank">https://openai.com= /blog/chatgpt/#samples</a>><br> ><br> > Anyone tried to incorporate it into teaching process?<br> > Or have ideas/doubts how it ciuld help?<br> ><br> > _______________________________________________<br> > Edu-sig mailing list -- <a href=3D"mailto:[email protected]" rel=3D"n= oreferrer noreferrer noreferrer" target=3D"_blank">[email protected]</a><b= r> > To unsubscribe send an email to <a href=3D"mailto:edu-sig-leave@python= .org" rel=3D"noreferrer noreferrer noreferrer" target=3D"_blank">edu-sig-le= [email protected]</a><br> > <a href=3D"https://mail.python.org/mailman3/lists/edu-sig.python.org/"= rel=3D"noreferrer noreferrer noreferrer noreferrer" target=3D"_blank">http= s://mail.python.org/mailman3/lists/edu-sig.python.org/</a><br> > Member address: <a href=3D"mailto:[email protected]" rel=3D"nor= eferrer noreferrer noreferrer" target=3D"_blank">[email protected]</= a><br> _______________________________________________<br> Edu-sig mailing list -- <a href=3D"mailto:[email protected]" rel=3D"norefe= rrer noreferrer noreferrer" target=3D"_blank">[email protected]</a><br> To unsubscribe send an email to <a href=3D"mailto:[email protected]"= rel=3D"noreferrer noreferrer noreferrer" target=3D"_blank">edu-sig-leave@p= ython.org</a><br> <a href=3D"https://mail.python.org/mailman3/lists/edu-sig.python.org/" rel= =3D"noreferrer noreferrer noreferrer noreferrer" target=3D"_blank">https://= mail.python.org/mailman3/lists/edu-sig.python.org/</a><br> Member address: <a href=3D"mailto:[email protected]" rel=3D"noreferrer n= oreferrer noreferrer" target=3D"_blank">[email protected]</a><br> </blockquote></div> </blockquote></div> --000000000000ab946505f1777f41-- --===============0726037330301891171== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Edu-sig mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/edu-sig.python.org/ Member address: [email protected] --===============0726037330301891171==--