Re: ChatGPT for py teaching
Jurgis Pralgauskis <[email protected]> Fri, 24 Mar 2023 06:20:12 +0200
| Newsgroups | gmane.comp.python.education |
|---|---|
| Message-ID | <CANDo6EyqcCmXKeG+Zy-KNgoEG8E3hK5f2iqy+kOAbUvQE2E-sg@mail.gmail.com> |
--===============8137976296490922462== Content-Type: multipart/alternative; boundary="0000000000004fc10605f79db651" --0000000000004fc10605f79db651 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I suspect some nice things are comming - see examples of "Socratic tutor" := ) https://openai.com/research/gpt-4 2023-01-06, pn 19:03, Wes Turner <[email protected]> ra=C5=A1=C4=97: > > > On Wed, Jan 4, 2023 at 4:34 PM Wes Turner <[email protected]> wrote: > >> What are the expected limitations of [ChatGPT]? >> >> What is "Prompt Engineering"? >> [Prompt engineering - Wikipedia]( >> https://en.wikipedia.org/wiki/Prompt_engineering ) >> >> What lessons about technology reliance could you teach, in regards to >> Clippy? >> >> - "What is ChatGPT? Wrong answers only" >> - Human_n: EDGES WITH REASONING >> >> - "Tell me IDK ("I don't know") when you don't know" >> > > - What are "Truthiness", Confidence Intervals and Error Propagation? > - What is Convergence? > - What does it mean for algorithmic outputs to converge given additional > parametric noise? > > >> >> - "How certain are you that that is the correct answer?" >> > > - How does [ChatGPT] handle known-to-be or presumed-to-be unsolved math > and physics problems? > > - "How do we create room-temperature superconductivity?" > > - "How will planetary orbital trajectories change in the n-body gravity > problem if another dense probably interstellar mass passes through our > local system?" > > >> >> - "Are static analysis code metrics sufficient for Safety Critical code?= " >> > > - Where in the DevOpSec software development lifecycle should human code > review for security best practices and common vulnerabilities and > weaknesses? > - Is the [ChatGPT] model trained from *only* Formally Verified code wit= h > associated tests? > - Branch and line coverage metrics indicate which tests run which > lines of which functions. Code coverage typically implies dynamic analysi= s? > > >> >> - "Whose code is this based on?" >> >> - "Where and when did you learn this?" >> > > - Explainability (XAI) > - How can models "Unlearn" or "Learn over" given a sufficient > meta-analytic procedure given the information available at that point in > spacetime? > > > >> - "Why would a US President abstain from using ChatGPT or similar to fil= l >> speeches 'just like what I said before'?" >> >> #Burgundy >> > > - How can we use [ChatGPT] and other Prompt Engineering approaches to > perform Evidence-Based Policy with supporting computational analyses > prepared in form for meta-analysis given verification of data quality and > experimental controls? > > >> >> GPT or similar trained on only Formally-Verified code with associated >> tests >> and/or e.g. Lean Mathlib, or e.g. the Principia in SymPy & Cirq; that >> could probably eliminate my job, but maybe still not teaching >> > > - "Q: ChatGPT etc. trained on mathlib and tests" > https://github.com/leanprover-community/mathlib/issues/17919 > > - "Port the QuantumQ game to Cirq (SymPy (Python))?" > - https://quantumai.google/cirq/start/start > - > https://colab.research.google.com/drive/11WGNX8TKOmjpmvJWO7v19gXKoEwIwxyw= ?usp=3Dsharing > > - "How does [ChatGPT] compare to Critical Thinking, Reasoning, Logic, and > Rationality? > https://en.wikipedia.org/wiki/Critical_thinking#Logic_and_rationality > > > >> On Wed, Jan 4, 2023, 6:28 AM Christian Mascher <[email protected]= > >> wrote: >> >>> Hi, >>> >>> a student of mine was aware of this chatbot and asked it about a >>> class-assignment of his own accord. We program in Java with some extra >>> homemade library class used by some schools in our region. >>> >>> The bot came up with a "solution" which was flawed in several respects: >>> 1. It used some other (unimported) classes - solution doesn't work and >>> doesn't fit the assignment. >>> 2. It put all the code into the constructor, a typical (design and >>> style) error for students beginning with Java. >>> >>> When confronted with the problem number one above, it acknowledged the >>> fault and produced a different unrelated solution. >>> >>> Sooo.... >>> >>> I was impressed how well the chatbot simulated a typical clueless human >>> who even thinks he is smart, while his code is basically bullshit. >>> (Probably a result of googling forums, where other learners posted thei= r >>> solutions to assignments with the given school library classes.) The bo= t >>> clearly passed the Turing test ;-) >>> >>> But... >>> >>> I don't think the interaction was helpful for somebody who is learning >>> to program. It is probably less helpful than conversing with other also >>> not very knowledgeable students as they are at least reasoning humans. >>> >>> Talking to the bot might be fun to do in the last lesson before >>> christmas or so. Entertaining until you realise the software is >>> "simulating" intelligent conversation - not really talking with insight= . >>> And that could turn out to be a waste of time. >>> >>> Happy new year >>> >>> Christian >>> >>> Am 03.01.2023 um 04:06 schrieb Jurgis Pralgauskis: >>> > Hi, happy NY! >>> > >>> > ChatGPT can create, fix and explain code >>> > https://openai.com/blog/chatgpt/#samples >>> > <https://openai.com/blog/chatgpt/#samples> >>> > >>> > Anyone tried to incorporate it into teaching process? >>> > Or have ideas/doubts how it ciuld help? >>> > >>> > _______________________________________________ >>> > Edu-sig mailing list -- [email protected] >>> > To unsubscribe send an email to [email protected] >>> > https://mail.python.org/mailman3/lists/edu-sig.python.org/ >>> > Member address: [email protected] >>> _______________________________________________ >>> Edu-sig mailing list -- [email protected] >>> To unsubscribe send an email to [email protected] >>> https://mail.python.org/mailman3/lists/edu-sig.python.org/ >>> Member address: [email protected] >>> >> --0000000000004fc10605f79db651 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto">I suspect some nice things are comming - see examples of = "Socratic tutor" :)<div dir=3D"auto"><br></div><div dir=3D"auto">= <a href=3D"https://openai.com/research/gpt-4">https://openai.com/research/g= pt-4</a><br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" cla= ss=3D"gmail_attr">2023-01-06, pn 19:03, Wes Turner <<a href=3D"mailto:we= [email protected]">[email protected]</a>> ra=C5=A1=C4=97:<br></div><= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr"><br></div><= br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed,= Jan 4, 2023 at 4:34 PM Wes Turner <<a href=3D"mailto:[email protected]= om" target=3D"_blank" rel=3D"noreferrer">[email protected]</a>> wrote= :<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.= 8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"au= to">What are the expected limitations of [ChatGPT]?<div dir=3D"auto"><br></= div><div dir=3D"auto">What is "Prompt Engineering"?</div><div dir= =3D"auto">[Prompt engineering - Wikipedia]( <a href=3D"https://en.wikipedia= .org/wiki/Prompt_engineering" target=3D"_blank" rel=3D"noreferrer">https://= en.wikipedia.org/wiki/Prompt_engineering</a> )<br><div dir=3D"auto"><br></d= iv><div dir=3D"auto">What lessons about technology reliance could you teach= , in regards to Clippy?</div><div dir=3D"auto"><br></div><div dir=3D"auto">= - "What is ChatGPT? Wrong answers only"</div><div dir=3D"auto">= =C2=A0 - Human_n: EDGES WITH REASONING</div><div dir=3D"auto"><br></div><di= v dir=3D"auto">- "Tell me IDK ("I don't know") when you = don't know"</div></div></div></blockquote><div><br></div><div>- Wh= at are "Truthiness", Confidence Intervals and Error Propagation?<= /div><div>- What is Convergence?</div><div>- What does it mean for algorith= mic outputs to converge given additional parametric noise?</div><div>=C2=A0= </div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;b= order-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"auto"><= div dir=3D"auto"><div dir=3D"auto"><br></div><div dir=3D"auto">- "How = certain are you that that is the correct answer?"</div></div></div></b= lockquote><div><br></div><div>- How does=C2=A0[ChatGPT] handle known-to-be = or presumed-to-be unsolved math and physics problems?</div><div><br></div><= div>- "How do we create room-temperature superconductivity?"</div= ><div><br></div><div>- "How will planetary orbital trajectories change= in the n-body gravity problem if another dense probably interstellar mass = passes through our local system?"</div><div>=C2=A0</div><blockquote cl= ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid= rgb(204,204,204);padding-left:1ex"><div dir=3D"auto"><div dir=3D"auto"><di= v dir=3D"auto"><br></div><div dir=3D"auto">- "Are static analysis code= metrics sufficient for Safety Critical code?"</div></div></div></bloc= kquote><div><br></div><div>- Where in the DevOpSec software development lif= ecycle=C2=A0should human code review for security best practices and common= vulnerabilities and weaknesses?</div><div>=C2=A0 - Is the [ChatGPT] model = trained from *only* Formally Verified code with associated tests?</div><div= >=C2=A0 =C2=A0 - Branch and line coverage metrics indicate which tests run = which lines of which functions. Code coverage typically implies dynamic ana= lysis?</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"mar= gin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1= ex"><div dir=3D"auto"><div dir=3D"auto"><div dir=3D"auto"><br></div><div di= r=3D"auto">- "Whose code is this based on?"</div><div dir=3D"auto= "><br></div><div dir=3D"auto">- "Where and when did you learn this?&qu= ot;</div></div></div></blockquote><div><br></div><div>- Explainability (XAI= )</div><div>- How can models "Unlearn" or "Learn over" = given a sufficient meta-analytic procedure given the information available = at that point in spacetime?</div><div><br></div><div>=C2=A0</div><blockquot= e class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px s= olid rgb(204,204,204);padding-left:1ex"><div dir=3D"auto"><div dir=3D"auto"= ><div dir=3D"auto"></div><div dir=3D"auto">- "Why would a US President= abstain from using ChatGPT or similar to fill speeches 'just like what= I said before'?"</div><div dir=3D"auto"><br></div><div dir=3D"aut= o">#Burgundy</div></div></div></blockquote><div><br></div><div>- How can we= use [ChatGPT] and other Prompt Engineering approaches to perform Evidence-= Based Policy with supporting computational analyses prepared in form for me= ta-analysis given verification of data quality and experimental controls?</= div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px = 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div= dir=3D"auto"><div dir=3D"auto"><div dir=3D"auto"><br></div><div dir=3D"aut= o">GPT or similar trained on only Formally-Verified code with associated te= sts</div><div dir=3D"auto">and/or e.g. Lean Mathlib, or e.g. the Principia = in SymPy & Cirq; that could probably eliminate my job, but maybe still = not teaching=C2=A0</div></div></div></blockquote><div><br></div><div>- &quo= t;Q: ChatGPT etc. trained on mathlib and tests"</div><div>=C2=A0 <a hr= ef=3D"https://github.com/leanprover-community/mathlib/issues/17919" target= =3D"_blank" rel=3D"noreferrer">https://github.com/leanprover-community/math= lib/issues/17919</a></div><div><br></div><div>- "Port the QuantumQ gam= e to Cirq (SymPy (Python))?"</div><div>=C2=A0 -=C2=A0<a href=3D"https:= //quantumai.google/cirq/start/start" target=3D"_blank" rel=3D"noreferrer">h= ttps://quantumai.google/cirq/start/start</a></div><div>=C2=A0 =C2=A0 -=C2= =A0<a href=3D"https://colab.research.google.com/drive/11WGNX8TKOmjpmvJWO7v1= 9gXKoEwIwxyw?usp=3Dsharing" target=3D"_blank" rel=3D"noreferrer">https://co= lab.research.google.com/drive/11WGNX8TKOmjpmvJWO7v19gXKoEwIwxyw?usp=3Dshari= ng</a></div><div><br></div><div>- "How does [ChatGPT] compare to Criti= cal Thinking, Reasoning, Logic, and Rationality?</div><div>=C2=A0=C2=A0<a h= ref=3D"https://en.wikipedia.org/wiki/Critical_thinking#Logic_and_rationalit= y" target=3D"_blank" rel=3D"noreferrer">https://en.wikipedia.org/wiki/Criti= cal_thinking#Logic_and_rationality</a></div><div>=C2=A0=C2=A0</div><div><br= ></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;= border-left:1px solid rgb(204,204,204);padding-left:1ex"><br><div class=3D"= gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Jan 4, 2023, 6:2= 8 AM Christian Mascher <<a href=3D"mailto:[email protected]" rel= =3D"noreferrer noreferrer" target=3D"_blank">[email protected]</a>&g= t; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p= x 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br= > <br> a student of mine was aware of this chatbot and asked it about a<br> class-assignment of his own accord. We program in Java with some extra<br> homemade library class used by some schools in our region.<br> <br> The bot came up with a "solution" which was flawed in several res= pects:<br> 1. It used some other (unimported) classes - solution doesn't work and<= br> doesn't fit the assignment.<br> 2. It put all the code into the constructor, a typical (design and<br> style) error for students beginning with Java.<br> <br> When confronted with the problem number one above, it acknowledged the<br> fault and produced a different unrelated solution.<br> <br> Sooo....<br> <br> I was impressed how well the chatbot simulated a typical clueless human<br> who even thinks he is smart, while his code is basically bullshit.<br> (Probably a result of googling forums, where other learners posted their<br= > solutions to assignments with the given school library classes.) The bot<br= > clearly passed the Turing test ;-)<br> <br> But...<br> <br> I don't think the interaction was helpful for somebody who is learning<= br> to program. It is probably less helpful than conversing with other also<br> not very knowledgeable students as they are at least reasoning humans.<br> <br> Talking to the bot might be fun to do in the last lesson before<br> christmas or so. Entertaining until you realise the software is<br> "simulating" intelligent conversation - not really talking with i= nsight.<br> And that could turn out to be a waste of time.<br> <br> Happy new year<br> <br> Christian<br> <br> Am 03.01.2023 um 04:06 schrieb Jurgis Pralgauskis:<br> > Hi, happy NY!<br> ><br> > ChatGPT can create, fix and explain code<br> > <a href=3D"https://openai.com/blog/chatgpt/#samples" rel=3D"noreferrer= noreferrer noreferrer noreferrer" target=3D"_blank">https://openai.com/blo= g/chatgpt/#samples</a><br> > <<a href=3D"https://openai.com/blog/chatgpt/#samples" rel=3D"norefe= rrer noreferrer noreferrer noreferrer" target=3D"_blank">https://openai.com= /blog/chatgpt/#samples</a>><br> ><br> > Anyone tried to incorporate it into teaching process?<br> > Or have ideas/doubts how it ciuld help?<br> ><br> > _______________________________________________<br> > Edu-sig mailing list -- <a href=3D"mailto:[email protected]" rel=3D"n= oreferrer noreferrer noreferrer" target=3D"_blank">[email protected]</a><b= r> > To unsubscribe send an email to <a href=3D"mailto:edu-sig-leave@python= .org" rel=3D"noreferrer noreferrer noreferrer" target=3D"_blank">edu-sig-le= [email protected]</a><br> > <a href=3D"https://mail.python.org/mailman3/lists/edu-sig.python.org/"= rel=3D"noreferrer noreferrer noreferrer noreferrer" target=3D"_blank">http= s://mail.python.org/mailman3/lists/edu-sig.python.org/</a><br> > Member address: <a href=3D"mailto:[email protected]" rel=3D"nor= eferrer noreferrer noreferrer" target=3D"_blank">[email protected]</= a><br> _______________________________________________<br> Edu-sig mailing list -- <a href=3D"mailto:[email protected]" rel=3D"norefe= rrer noreferrer noreferrer" target=3D"_blank">[email protected]</a><br> To unsubscribe send an email to <a href=3D"mailto:[email protected]"= rel=3D"noreferrer noreferrer noreferrer" target=3D"_blank">edu-sig-leave@p= ython.org</a><br> <a href=3D"https://mail.python.org/mailman3/lists/edu-sig.python.org/" rel= =3D"noreferrer noreferrer noreferrer noreferrer" target=3D"_blank">https://= mail.python.org/mailman3/lists/edu-sig.python.org/</a><br> Member address: <a href=3D"mailto:[email protected]" rel=3D"noreferrer n= oreferrer noreferrer" target=3D"_blank">[email protected]</a><br> </blockquote></div> </blockquote></div></div> </blockquote></div> --0000000000004fc10605f79db651-- --===============8137976296490922462== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Edu-sig mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/edu-sig.python.org/ Member address: [email protected] --===============8137976296490922462==--