Re: ChatGPT for py teaching
Wes Turner <[email protected]> Fri, 6 Jan 2023 12:03:21 -0500
| Newsgroups | gmane.comp.python.education |
|---|---|
| Message-ID | <CACfEFw917gkzj47wn4f4EhDyq40QHczstUbgVqa3Nd70VjM3=Q@mail.gmail.com> |
--===============0530684309541577453== Content-Type: multipart/alternative; boundary="0000000000007f638205f19b6548" --0000000000007f638205f19b6548 Content-Type: text/plain; charset="UTF-8" On Wed, Jan 4, 2023 at 4:34 PM Wes Turner <[email protected]> wrote: > What are the expected limitations of [ChatGPT]? > > What is "Prompt Engineering"? > [Prompt engineering - Wikipedia]( > https://en.wikipedia.org/wiki/Prompt_engineering ) > > What lessons about technology reliance could you teach, in regards to > Clippy? > > - "What is ChatGPT? Wrong answers only" > - Human_n: EDGES WITH REASONING > > - "Tell me IDK ("I don't know") when you don't know" > - What are "Truthiness", Confidence Intervals and Error Propagation? - What is Convergence? - What does it mean for algorithmic outputs to converge given additional parametric noise? > > - "How certain are you that that is the correct answer?" > - How does [ChatGPT] handle known-to-be or presumed-to-be unsolved math and physics problems? - "How do we create room-temperature superconductivity?" - "How will planetary orbital trajectories change in the n-body gravity problem if another dense probably interstellar mass passes through our local system?" > > - "Are static analysis code metrics sufficient for Safety Critical code?" > - Where in the DevOpSec software development lifecycle should human code review for security best practices and common vulnerabilities and weaknesses? - Is the [ChatGPT] model trained from *only* Formally Verified code with associated tests? - Branch and line coverage metrics indicate which tests run which lines of which functions. Code coverage typically implies dynamic analysis? > > - "Whose code is this based on?" > > - "Where and when did you learn this?" > - Explainability (XAI) - How can models "Unlearn" or "Learn over" given a sufficient meta-analytic procedure given the information available at that point in spacetime? > - "Why would a US President abstain from using ChatGPT or similar to fill > speeches 'just like what I said before'?" > > #Burgundy > - How can we use [ChatGPT] and other Prompt Engineering approaches to perform Evidence-Based Policy with supporting computational analyses prepared in form for meta-analysis given verification of data quality and experimental controls? > > GPT or similar trained on only Formally-Verified code with associated tests > and/or e.g. Lean Mathlib, or e.g. the Principia in SymPy & Cirq; that > could probably eliminate my job, but maybe still not teaching > - "Q: ChatGPT etc. trained on mathlib and tests" https://github.com/leanprover-community/mathlib/issues/17919 - "Port the QuantumQ game to Cirq (SymPy (Python))?" - https://quantumai.google/cirq/start/start - https://colab.research.google.com/drive/11WGNX8TKOmjpmvJWO7v19gXKoEwIwxyw?usp=sharing - "How does [ChatGPT] compare to Critical Thinking, Reasoning, Logic, and Rationality? https://en.wikipedia.org/wiki/Critical_thinking#Logic_and_rationality > On Wed, Jan 4, 2023, 6:28 AM Christian Mascher <[email protected]> > wrote: > >> Hi, >> >> a student of mine was aware of this chatbot and asked it about a >> class-assignment of his own accord. We program in Java with some extra >> homemade library class used by some schools in our region. >> >> The bot came up with a "solution" which was flawed in several respects: >> 1. It used some other (unimported) classes - solution doesn't work and >> doesn't fit the assignment. >> 2. It put all the code into the constructor, a typical (design and >> style) error for students beginning with Java. >> >> When confronted with the problem number one above, it acknowledged the >> fault and produced a different unrelated solution. >> >> Sooo.... >> >> I was impressed how well the chatbot simulated a typical clueless human >> who even thinks he is smart, while his code is basically bullshit. >> (Probably a result of googling forums, where other learners posted their >> solutions to assignments with the given school library classes.) The bot >> clearly passed the Turing test ;-) >> >> But... >> >> I don't think the interaction was helpful for somebody who is learning >> to program. It is probably less helpful than conversing with other also >> not very knowledgeable students as they are at least reasoning humans. >> >> Talking to the bot might be fun to do in the last lesson before >> christmas or so. Entertaining until you realise the software is >> "simulating" intelligent conversation - not really talking with insight. >> And that could turn out to be a waste of time. >> >> Happy new year >> >> Christian >> >> Am 03.01.2023 um 04:06 schrieb Jurgis Pralgauskis: >> > Hi, happy NY! >> > >> > ChatGPT can create, fix and explain code >> > https://openai.com/blog/chatgpt/#samples >> > <https://openai.com/blog/chatgpt/#samples> >> > >> > Anyone tried to incorporate it into teaching process? >> > Or have ideas/doubts how it ciuld help? >> > >> > _______________________________________________ >> > Edu-sig mailing list -- [email protected] >> > To unsubscribe send an email to [email protected] >> > https://mail.python.org/mailman3/lists/edu-sig.python.org/ >> > Member address: [email protected] >> _______________________________________________ >> Edu-sig mailing list -- [email protected] >> To unsubscribe send an email to [email protected] >> https://mail.python.org/mailman3/lists/edu-sig.python.org/ >> Member address: [email protected] >> > --0000000000007f638205f19b6548 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Wed, Jan 4, 2023 at 4:34 PM Wes Tu= rner <<a href=3D"mailto:[email protected]">[email protected]</a>&g= t; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p= x 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div d= ir=3D"auto">What are the expected limitations of [ChatGPT]?<div dir=3D"auto= "><br></div><div dir=3D"auto">What is "Prompt Engineering"?</div>= <div dir=3D"auto">[Prompt engineering - Wikipedia]( <a href=3D"https://en.w= ikipedia.org/wiki/Prompt_engineering" target=3D"_blank">https://en.wikipedi= a.org/wiki/Prompt_engineering</a> )<br><div dir=3D"auto"><br></div><div dir= =3D"auto">What lessons about technology reliance could you teach, in regard= s to Clippy?</div><div dir=3D"auto"><br></div><div dir=3D"auto">- "Wha= t is ChatGPT? Wrong answers only"</div><div dir=3D"auto">=C2=A0 - Huma= n_n: EDGES WITH REASONING</div><div dir=3D"auto"><br></div><div dir=3D"auto= ">- "Tell me IDK ("I don't know") when you don't kno= w"</div></div></div></blockquote><div><br></div><div>- What are "= Truthiness", Confidence Intervals and Error Propagation?</div><div>- W= hat is Convergence?</div><div>- What does it mean for algorithmic outputs t= o converge given additional parametric noise?</div><div>=C2=A0</div><blockq= uote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1p= x solid rgb(204,204,204);padding-left:1ex"><div dir=3D"auto"><div dir=3D"au= to"><div dir=3D"auto"><br></div><div dir=3D"auto">- "How certain are y= ou that that is the correct answer?"</div></div></div></blockquote><di= v><br></div><div>- How does=C2=A0[ChatGPT] handle known-to-be or presumed-t= o-be unsolved math and physics problems?</div><div><br></div><div>- "H= ow do we create room-temperature superconductivity?"</div><div><br></d= iv><div>- "How will planetary orbital trajectories change in the n-bod= y gravity problem if another dense probably interstellar mass passes throug= h our local system?"</div><div>=C2=A0</div><blockquote class=3D"gmail_= quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,= 204);padding-left:1ex"><div dir=3D"auto"><div dir=3D"auto"><div dir=3D"auto= "><br></div><div dir=3D"auto">- "Are static analysis code metrics suff= icient for Safety Critical code?"</div></div></div></blockquote><div><= br></div><div>- Where in the DevOpSec software development lifecycle=C2=A0s= hould human code review for security best practices and common vulnerabilit= ies and weaknesses?</div><div>=C2=A0 - Is the [ChatGPT] model trained from = *only* Formally Verified code with associated tests?</div><div>=C2=A0 =C2= =A0 - Branch and line coverage metrics indicate which tests run which lines= of which functions. Code coverage typically implies dynamic analysis?</div= ><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px= 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div di= r=3D"auto"><div dir=3D"auto"><div dir=3D"auto"><br></div><div dir=3D"auto">= - "Whose code is this based on?"</div><div dir=3D"auto"><br></div= ><div dir=3D"auto">- "Where and when did you learn this?"</div></= div></div></blockquote><div><br></div><div>- Explainability (XAI)</div><div= >- How can models "Unlearn" or "Learn over" given a suf= ficient meta-analytic procedure given the information available at that poi= nt in spacetime?</div><div><br></div><div>=C2=A0</div><blockquote class=3D"= gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20= 4,204,204);padding-left:1ex"><div dir=3D"auto"><div dir=3D"auto"><div dir= =3D"auto"></div><div dir=3D"auto">- "Why would a US President abstain = from using ChatGPT or similar to fill speeches 'just like what I said b= efore'?"</div><div dir=3D"auto"><br></div><div dir=3D"auto">#Burgu= ndy</div></div></div></blockquote><div><br></div><div>- How can we use [Cha= tGPT] and other Prompt Engineering approaches to perform Evidence-Based Pol= icy with supporting computational analyses prepared in form for meta-analys= is given verification of data quality and experimental controls?</div><div>= =C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0= .8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"a= uto"><div dir=3D"auto"><div dir=3D"auto"><br></div><div dir=3D"auto">GPT or= similar trained on only Formally-Verified code with associated tests</div>= <div dir=3D"auto">and/or e.g. Lean Mathlib, or e.g. the Principia in SymPy = & Cirq; that could probably eliminate my job, but maybe still not teach= ing=C2=A0</div></div></div></blockquote><div><br></div><div>- "Q: Chat= GPT etc. trained on mathlib and tests"</div><div>=C2=A0 <a href=3D"htt= ps://github.com/leanprover-community/mathlib/issues/17919">https://github.c= om/leanprover-community/mathlib/issues/17919</a></div><div><br></div><div>-= "Port the QuantumQ game to Cirq (SymPy (Python))?"</div><div>=C2= =A0 -=C2=A0<a href=3D"https://quantumai.google/cirq/start/start">https://qu= antumai.google/cirq/start/start</a></div><div>=C2=A0 =C2=A0 -=C2=A0<a href= =3D"https://colab.research.google.com/drive/11WGNX8TKOmjpmvJWO7v19gXKoEwIwx= yw?usp=3Dsharing">https://colab.research.google.com/drive/11WGNX8TKOmjpmvJW= O7v19gXKoEwIwxyw?usp=3Dsharing</a></div><div><br></div><div>- "How doe= s [ChatGPT] compare to Critical Thinking, Reasoning, Logic, and Rationality= ?</div><div>=C2=A0=C2=A0<a href=3D"https://en.wikipedia.org/wiki/Critical_t= hinking#Logic_and_rationality">https://en.wikipedia.org/wiki/Critical_think= ing#Logic_and_rationality</a></div><div>=C2=A0=C2=A0</div><div><br></div><b= lockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-le= ft:1px solid rgb(204,204,204);padding-left:1ex"><br><div class=3D"gmail_quo= te"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Jan 4, 2023, 6:28 AM Chri= stian Mascher <<a href=3D"mailto:[email protected]" rel=3D"norefe= rrer" target=3D"_blank">[email protected]</a>> wrote:<br></div><b= lockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-le= ft:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br> <br> a student of mine was aware of this chatbot and asked it about a<br> class-assignment of his own accord. We program in Java with some extra<br> homemade library class used by some schools in our region.<br> <br> The bot came up with a "solution" which was flawed in several res= pects:<br> 1. It used some other (unimported) classes - solution doesn't work and<= br> doesn't fit the assignment.<br> 2. It put all the code into the constructor, a typical (design and<br> style) error for students beginning with Java.<br> <br> When confronted with the problem number one above, it acknowledged the<br> fault and produced a different unrelated solution.<br> <br> Sooo....<br> <br> I was impressed how well the chatbot simulated a typical clueless human<br> who even thinks he is smart, while his code is basically bullshit.<br> (Probably a result of googling forums, where other learners posted their<br= > solutions to assignments with the given school library classes.) The bot<br= > clearly passed the Turing test ;-)<br> <br> But...<br> <br> I don't think the interaction was helpful for somebody who is learning<= br> to program. It is probably less helpful than conversing with other also<br> not very knowledgeable students as they are at least reasoning humans.<br> <br> Talking to the bot might be fun to do in the last lesson before<br> christmas or so. Entertaining until you realise the software is<br> "simulating" intelligent conversation - not really talking with i= nsight.<br> And that could turn out to be a waste of time.<br> <br> Happy new year<br> <br> Christian<br> <br> Am 03.01.2023 um 04:06 schrieb Jurgis Pralgauskis:<br> > Hi, happy NY!<br> ><br> > ChatGPT can create, fix and explain code<br> > <a href=3D"https://openai.com/blog/chatgpt/#samples" rel=3D"noreferrer= noreferrer noreferrer" target=3D"_blank">https://openai.com/blog/chatgpt/#= samples</a><br> > <<a href=3D"https://openai.com/blog/chatgpt/#samples" rel=3D"norefe= rrer noreferrer noreferrer" target=3D"_blank">https://openai.com/blog/chatg= pt/#samples</a>><br> ><br> > Anyone tried to incorporate it into teaching process?<br> > Or have ideas/doubts how it ciuld help?<br> ><br> > _______________________________________________<br> > Edu-sig mailing list -- <a href=3D"mailto:[email protected]" rel=3D"n= oreferrer noreferrer" target=3D"_blank">[email protected]</a><br> > To unsubscribe send an email to <a href=3D"mailto:edu-sig-leave@python= .org" rel=3D"noreferrer noreferrer" target=3D"_blank">edu-sig-leave@python.= org</a><br> > <a href=3D"https://mail.python.org/mailman3/lists/edu-sig.python.org/"= rel=3D"noreferrer noreferrer noreferrer" target=3D"_blank">https://mail.py= thon.org/mailman3/lists/edu-sig.python.org/</a><br> > Member address: <a href=3D"mailto:[email protected]" rel=3D"nor= eferrer noreferrer" target=3D"_blank">[email protected]</a><br> _______________________________________________<br> Edu-sig mailing list -- <a href=3D"mailto:[email protected]" rel=3D"norefe= rrer noreferrer" target=3D"_blank">[email protected]</a><br> To unsubscribe send an email to <a href=3D"mailto:[email protected]"= rel=3D"noreferrer noreferrer" target=3D"_blank">[email protected]</= a><br> <a href=3D"https://mail.python.org/mailman3/lists/edu-sig.python.org/" rel= =3D"noreferrer noreferrer noreferrer" target=3D"_blank">https://mail.python= .org/mailman3/lists/edu-sig.python.org/</a><br> Member address: <a href=3D"mailto:[email protected]" rel=3D"noreferrer n= oreferrer" target=3D"_blank">[email protected]</a><br> </blockquote></div> </blockquote></div></div> --0000000000007f638205f19b6548-- --===============0530684309541577453== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Edu-sig mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/edu-sig.python.org/ Member address: [email protected] --===============0530684309541577453==--