Re: ChatGPT for py teaching
Wes Turner <[email protected]> Fri, 24 Mar 2023 16:12:31 -0400
| Newsgroups | gmane.comp.python.education |
|---|---|
| Message-ID | <CACfEFw87OFQBbYJn+-DzWG1BSRkmUHNRShEroOoLJMZfvDKakw@mail.gmail.com> |
--===============7107119619413045032== Content-Type: multipart/alternative; boundary="0000000000000d87f405f7ab04e9" --0000000000000d87f405f7ab04e9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Khanmigo is a new (2023-03-14) Khan Academy tutoring AI: https://blog.khanacademy.org/harnessing-ai-so-that-all-students-benefit-a-n= onprofit-approach-for-equal-access/ : """ Participants in our limited pilot can use Khanmigo to: - Get help with math - Write a story together through back-and-forth idea generation (but Khanmigo won=E2=80=99t write the story for them!) - Prep for AP exams - Practice new vocabulary words - Learn computer programming Interview a historical figure like Cleopatra or Jane Austen - Debate a topic like =E2=80=9CShould schools ban AI?=E2=80=9D Khanmigo does not, however, give students answers. Nobody learns anything by being given the answer. Instead, Khanmigo asks questions. Questions are thought-provoking. Questions are open-ended. And most importantly, questions lead to learning """ What is the Socratic method? https://en.wikipedia.org/wiki/Socratic_method Why are LLMs picking random positive integers 42 and 7 most frequently? On Fri, Mar 24, 2023, 12:20 AM Jurgis Pralgauskis < [email protected]> wrote: > I suspect some nice things are comming - see examples of "Socratic tutor" > :) > > https://openai.com/research/gpt-4 > > 2023-01-06, pn 19:03, Wes Turner <[email protected]> ra=C5=A1=C4=97: > >> >> >> On Wed, Jan 4, 2023 at 4:34 PM Wes Turner <[email protected]> wrote: >> >>> What are the expected limitations of [ChatGPT]? >>> >>> What is "Prompt Engineering"? >>> [Prompt engineering - Wikipedia]( >>> https://en.wikipedia.org/wiki/Prompt_engineering ) >>> >>> What lessons about technology reliance could you teach, in regards to >>> Clippy? >>> >>> - "What is ChatGPT? Wrong answers only" >>> - Human_n: EDGES WITH REASONING >>> >>> - "Tell me IDK ("I don't know") when you don't know" >>> >> >> - What are "Truthiness", Confidence Intervals and Error Propagation? >> - What is Convergence? >> - What does it mean for algorithmic outputs to converge given additional >> parametric noise? >> >> >>> >>> - "How certain are you that that is the correct answer?" >>> >> >> - How does [ChatGPT] handle known-to-be or presumed-to-be unsolved math >> and physics problems? >> >> - "How do we create room-temperature superconductivity?" >> >> - "How will planetary orbital trajectories change in the n-body gravity >> problem if another dense probably interstellar mass passes through our >> local system?" >> >> >>> >>> - "Are static analysis code metrics sufficient for Safety Critical code= ?" >>> >> >> - Where in the DevOpSec software development lifecycle should human code >> review for security best practices and common vulnerabilities and >> weaknesses? >> - Is the [ChatGPT] model trained from *only* Formally Verified code >> with associated tests? >> - Branch and line coverage metrics indicate which tests run which >> lines of which functions. Code coverage typically implies dynamic analys= is? >> >> >>> >>> - "Whose code is this based on?" >>> >>> - "Where and when did you learn this?" >>> >> >> - Explainability (XAI) >> - How can models "Unlearn" or "Learn over" given a sufficient >> meta-analytic procedure given the information available at that point in >> spacetime? >> >> >> >>> - "Why would a US President abstain from using ChatGPT or similar to >>> fill speeches 'just like what I said before'?" >>> >>> #Burgundy >>> >> >> - How can we use [ChatGPT] and other Prompt Engineering approaches to >> perform Evidence-Based Policy with supporting computational analyses >> prepared in form for meta-analysis given verification of data quality an= d >> experimental controls? >> >> >>> >>> GPT or similar trained on only Formally-Verified code with associated >>> tests >>> and/or e.g. Lean Mathlib, or e.g. the Principia in SymPy & Cirq; that >>> could probably eliminate my job, but maybe still not teaching >>> >> >> - "Q: ChatGPT etc. trained on mathlib and tests" >> https://github.com/leanprover-community/mathlib/issues/17919 >> >> - "Port the QuantumQ game to Cirq (SymPy (Python))?" >> - https://quantumai.google/cirq/start/start >> - >> https://colab.research.google.com/drive/11WGNX8TKOmjpmvJWO7v19gXKoEwIwxy= w?usp=3Dsharing >> >> - "How does [ChatGPT] compare to Critical Thinking, Reasoning, Logic, an= d >> Rationality? >> https://en.wikipedia.org/wiki/Critical_thinking#Logic_and_rationality >> >> >> >>> On Wed, Jan 4, 2023, 6:28 AM Christian Mascher <[email protected]= e> >>> wrote: >>> >>>> Hi, >>>> >>>> a student of mine was aware of this chatbot and asked it about a >>>> class-assignment of his own accord. We program in Java with some extra >>>> homemade library class used by some schools in our region. >>>> >>>> The bot came up with a "solution" which was flawed in several respects= : >>>> 1. It used some other (unimported) classes - solution doesn't work and >>>> doesn't fit the assignment. >>>> 2. It put all the code into the constructor, a typical (design and >>>> style) error for students beginning with Java. >>>> >>>> When confronted with the problem number one above, it acknowledged the >>>> fault and produced a different unrelated solution. >>>> >>>> Sooo.... >>>> >>>> I was impressed how well the chatbot simulated a typical clueless huma= n >>>> who even thinks he is smart, while his code is basically bullshit. >>>> (Probably a result of googling forums, where other learners posted the= ir >>>> solutions to assignments with the given school library classes.) The b= ot >>>> clearly passed the Turing test ;-) >>>> >>>> But... >>>> >>>> I don't think the interaction was helpful for somebody who is learning >>>> to program. It is probably less helpful than conversing with other als= o >>>> not very knowledgeable students as they are at least reasoning humans. >>>> >>>> Talking to the bot might be fun to do in the last lesson before >>>> christmas or so. Entertaining until you realise the software is >>>> "simulating" intelligent conversation - not really talking with insigh= t. >>>> And that could turn out to be a waste of time. >>>> >>>> Happy new year >>>> >>>> Christian >>>> >>>> Am 03.01.2023 um 04:06 schrieb Jurgis Pralgauskis: >>>> > Hi, happy NY! >>>> > >>>> > ChatGPT can create, fix and explain code >>>> > https://openai.com/blog/chatgpt/#samples >>>> > <https://openai.com/blog/chatgpt/#samples> >>>> > >>>> > Anyone tried to incorporate it into teaching process? >>>> > Or have ideas/doubts how it ciuld help? >>>> > >>>> > _______________________________________________ >>>> > Edu-sig mailing list -- [email protected] >>>> > To unsubscribe send an email to [email protected] >>>> > https://mail.python.org/mailman3/lists/edu-sig.python.org/ >>>> > Member address: [email protected] >>>> _______________________________________________ >>>> Edu-sig mailing list -- [email protected] >>>> To unsubscribe send an email to [email protected] >>>> https://mail.python.org/mailman3/lists/edu-sig.python.org/ >>>> Member address: [email protected] >>>> >>> --0000000000000d87f405f7ab04e9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto"><div dir=3D"auto">Khanmigo is a new (2023-03-14) Khan Aca= demy tutoring AI:=C2=A0</div><a href=3D"https://blog.khanacademy.org/harnes= sing-ai-so-that-all-students-benefit-a-nonprofit-approach-for-equal-access/= ">https://blog.khanacademy.org/harnessing-ai-so-that-all-students-benefit-a= -nonprofit-approach-for-equal-access/</a> :<div dir=3D"auto"><br></div><div= dir=3D"auto">"""</div><div dir=3D"auto">Participants in our= limited pilot can use Khanmigo to:<br></div><div dir=3D"auto"><br></div><d= iv dir=3D"auto">- Get help with math=C2=A0</div><div dir=3D"auto">- Write a= story together through back-and-forth idea generation (but Khanmigo won=E2= =80=99t write the story for them!)</div><div dir=3D"auto">- Prep for AP exa= ms</div><div dir=3D"auto">- Practice new vocabulary words</div><div dir=3D"= auto">- Learn computer programming</div><div dir=3D"auto">Interview a histo= rical figure like Cleopatra or Jane Austen</div><div dir=3D"auto">- Debate = a topic like =E2=80=9CShould schools ban AI?=E2=80=9D</div><div dir=3D"auto= "><br></div><div dir=3D"auto">Khanmigo does not, however, give students ans= wers. Nobody learns anything by being given the answer. Instead, Khanmigo a= sks questions. Questions are thought-provoking. Questions are open-ended. A= nd most importantly, questions lead to learning</div><div dir=3D"auto">&quo= t;""</div><div dir=3D"auto"><br></div><div dir=3D"auto">What is t= he Socratic method?=C2=A0</div><div dir=3D"auto"><a href=3D"https://en.wiki= pedia.org/wiki/Socratic_method">https://en.wikipedia.org/wiki/Socratic_meth= od</a></div><div dir=3D"auto"><br></div><div dir=3D"auto">Why are LLMs pick= ing random positive integers 42 and 7 most frequently?</div></div><br><div = class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Fri, Mar 24,= 2023, 12:20 AM Jurgis Pralgauskis <<a href=3D"mailto:jurgis.pralgauskis= @gmail.com">[email protected]</a>> wrote:<br></div><blockquot= e class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc sol= id;padding-left:1ex"><div dir=3D"auto">I suspect some nice things are commi= ng - see examples of "Socratic tutor" :)<div dir=3D"auto"><br></d= iv><div dir=3D"auto"><a href=3D"https://openai.com/research/gpt-4" target= =3D"_blank" rel=3D"noreferrer">https://openai.com/research/gpt-4</a><br></d= iv></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_att= r">2023-01-06, pn 19:03, Wes Turner <<a href=3D"mailto:wes.turner@gmail.= com" target=3D"_blank" rel=3D"noreferrer">[email protected]</a>> ra= =C5=A1=C4=97:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 = 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div= dir=3D"ltr"><br></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" clas= s=3D"gmail_attr">On Wed, Jan 4, 2023 at 4:34 PM Wes Turner <<a href=3D"m= ailto:[email protected]" rel=3D"noreferrer noreferrer" target=3D"_blank"= >[email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_qu= ote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,20= 4);padding-left:1ex"><div dir=3D"auto">What are the expected limitations of= [ChatGPT]?<div dir=3D"auto"><br></div><div dir=3D"auto">What is "Prom= pt Engineering"?</div><div dir=3D"auto">[Prompt engineering - Wikipedi= a]( <a href=3D"https://en.wikipedia.org/wiki/Prompt_engineering" rel=3D"nor= eferrer noreferrer" target=3D"_blank">https://en.wikipedia.org/wiki/Prompt_= engineering</a> )<br><div dir=3D"auto"><br></div><div dir=3D"auto">What les= sons about technology reliance could you teach, in regards to Clippy?</div>= <div dir=3D"auto"><br></div><div dir=3D"auto">- "What is ChatGPT? Wron= g answers only"</div><div dir=3D"auto">=C2=A0 - Human_n: EDGES WITH RE= ASONING</div><div dir=3D"auto"><br></div><div dir=3D"auto">- "Tell me = IDK ("I don't know") when you don't know"</div></div= ></div></blockquote><div><br></div><div>- What are "Truthiness", = Confidence Intervals and Error Propagation?</div><div>- What is Convergence= ?</div><div>- What does it mean for algorithmic outputs to converge given a= dditional parametric noise?</div><div>=C2=A0</div><blockquote class=3D"gmai= l_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,20= 4,204);padding-left:1ex"><div dir=3D"auto"><div dir=3D"auto"><div dir=3D"au= to"><br></div><div dir=3D"auto">- "How certain are you that that is th= e correct answer?"</div></div></div></blockquote><div><br></div><div>-= How does=C2=A0[ChatGPT] handle known-to-be or presumed-to-be unsolved math= and physics problems?</div><div><br></div><div>- "How do we create ro= om-temperature superconductivity?"</div><div><br></div><div>- "Ho= w will planetary orbital trajectories change in the n-body gravity problem = if another dense probably interstellar mass passes through our local system= ?"</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"ma= rgin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:= 1ex"><div dir=3D"auto"><div dir=3D"auto"><div dir=3D"auto"><br></div><div d= ir=3D"auto">- "Are static analysis code metrics sufficient for Safety = Critical code?"</div></div></div></blockquote><div><br></div><div>- Wh= ere in the DevOpSec software development lifecycle=C2=A0should human code r= eview for security best practices and common vulnerabilities and weaknesses= ?</div><div>=C2=A0 - Is the [ChatGPT] model trained from *only* Formally Ve= rified code with associated tests?</div><div>=C2=A0 =C2=A0 - Branch and lin= e coverage metrics indicate which tests run which lines of which functions.= Code coverage typically implies dynamic analysis?</div><div>=C2=A0</div><b= lockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-le= ft:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"auto"><div dir= =3D"auto"><div dir=3D"auto"><br></div><div dir=3D"auto">- "Whose code = is this based on?"</div><div dir=3D"auto"><br></div><div dir=3D"auto">= - "Where and when did you learn this?"</div></div></div></blockqu= ote><div><br></div><div>- Explainability (XAI)</div><div>- How can models &= quot;Unlearn" or "Learn over" given a sufficient meta-analyt= ic procedure given the information available at that point in spacetime?</d= iv><div><br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style= =3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding= -left:1ex"><div dir=3D"auto"><div dir=3D"auto"><div dir=3D"auto"></div><div= dir=3D"auto">- "Why would a US President abstain from using ChatGPT o= r similar to fill speeches 'just like what I said before'?"</d= iv><div dir=3D"auto"><br></div><div dir=3D"auto">#Burgundy</div></div></div= ></blockquote><div><br></div><div>- How can we use [ChatGPT] and other Prom= pt Engineering approaches to perform Evidence-Based Policy with supporting = computational analyses prepared in form for meta-analysis given verificatio= n of data quality and experimental controls?</div><div>=C2=A0</div><blockqu= ote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px= solid rgb(204,204,204);padding-left:1ex"><div dir=3D"auto"><div dir=3D"aut= o"><div dir=3D"auto"><br></div><div dir=3D"auto">GPT or similar trained on = only Formally-Verified code with associated tests</div><div dir=3D"auto">an= d/or e.g. Lean Mathlib, or e.g. the Principia in SymPy & Cirq; that cou= ld probably eliminate my job, but maybe still not teaching=C2=A0</div></div= ></div></blockquote><div><br></div><div>- "Q: ChatGPT etc. trained on = mathlib and tests"</div><div>=C2=A0 <a href=3D"https://github.com/lean= prover-community/mathlib/issues/17919" rel=3D"noreferrer noreferrer" target= =3D"_blank">https://github.com/leanprover-community/mathlib/issues/17919</a= ></div><div><br></div><div>- "Port the QuantumQ game to Cirq (SymPy (P= ython))?"</div><div>=C2=A0 -=C2=A0<a href=3D"https://quantumai.google/= cirq/start/start" rel=3D"noreferrer noreferrer" target=3D"_blank">https://q= uantumai.google/cirq/start/start</a></div><div>=C2=A0 =C2=A0 -=C2=A0<a href= =3D"https://colab.research.google.com/drive/11WGNX8TKOmjpmvJWO7v19gXKoEwIwx= yw?usp=3Dsharing" rel=3D"noreferrer noreferrer" target=3D"_blank">https://c= olab.research.google.com/drive/11WGNX8TKOmjpmvJWO7v19gXKoEwIwxyw?usp=3Dshar= ing</a></div><div><br></div><div>- "How does [ChatGPT] compare to Crit= ical Thinking, Reasoning, Logic, and Rationality?</div><div>=C2=A0=C2=A0<a = href=3D"https://en.wikipedia.org/wiki/Critical_thinking#Logic_and_rationali= ty" rel=3D"noreferrer noreferrer" target=3D"_blank">https://en.wikipedia.or= g/wiki/Critical_thinking#Logic_and_rationality</a></div><div>=C2=A0=C2=A0</= div><div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p= x 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br><d= iv class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Jan = 4, 2023, 6:28 AM Christian Mascher <<a href=3D"mailto:christian.mascher@= gmx.de" rel=3D"noreferrer noreferrer noreferrer" target=3D"_blank">christia= [email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_quote" = style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pa= dding-left:1ex">Hi,<br> <br> a student of mine was aware of this chatbot and asked it about a<br> class-assignment of his own accord. We program in Java with some extra<br> homemade library class used by some schools in our region.<br> <br> The bot came up with a "solution" which was flawed in several res= pects:<br> 1. It used some other (unimported) classes - solution doesn't work and<= br> doesn't fit the assignment.<br> 2. It put all the code into the constructor, a typical (design and<br> style) error for students beginning with Java.<br> <br> When confronted with the problem number one above, it acknowledged the<br> fault and produced a different unrelated solution.<br> <br> Sooo....<br> <br> I was impressed how well the chatbot simulated a typical clueless human<br> who even thinks he is smart, while his code is basically bullshit.<br> (Probably a result of googling forums, where other learners posted their<br= > solutions to assignments with the given school library classes.) The bot<br= > clearly passed the Turing test ;-)<br> <br> But...<br> <br> I don't think the interaction was helpful for somebody who is learning<= br> to program. It is probably less helpful than conversing with other also<br> not very knowledgeable students as they are at least reasoning humans.<br> <br> Talking to the bot might be fun to do in the last lesson before<br> christmas or so. Entertaining until you realise the software is<br> "simulating" intelligent conversation - not really talking with i= nsight.<br> And that could turn out to be a waste of time.<br> <br> Happy new year<br> <br> Christian<br> <br> Am 03.01.2023 um 04:06 schrieb Jurgis Pralgauskis:<br> > Hi, happy NY!<br> ><br> > ChatGPT can create, fix and explain code<br> > <a href=3D"https://openai.com/blog/chatgpt/#samples" rel=3D"noreferrer= noreferrer noreferrer noreferrer noreferrer" target=3D"_blank">https://ope= nai.com/blog/chatgpt/#samples</a><br> > <<a href=3D"https://openai.com/blog/chatgpt/#samples" rel=3D"norefe= rrer noreferrer noreferrer noreferrer noreferrer" target=3D"_blank">https:/= /openai.com/blog/chatgpt/#samples</a>><br> ><br> > Anyone tried to incorporate it into teaching process?<br> > Or have ideas/doubts how it ciuld help?<br> ><br> > _______________________________________________<br> > Edu-sig mailing list -- <a href=3D"mailto:[email protected]" rel=3D"n= oreferrer noreferrer noreferrer noreferrer" target=3D"_blank">edu-sig@pytho= n.org</a><br> > To unsubscribe send an email to <a href=3D"mailto:edu-sig-leave@python= .org" rel=3D"noreferrer noreferrer noreferrer noreferrer" target=3D"_blank"= >[email protected]</a><br> > <a href=3D"https://mail.python.org/mailman3/lists/edu-sig.python.org/"= rel=3D"noreferrer noreferrer noreferrer noreferrer noreferrer" target=3D"_= blank">https://mail.python.org/mailman3/lists/edu-sig.python.org/</a><br> > Member address: <a href=3D"mailto:[email protected]" rel=3D"nor= eferrer noreferrer noreferrer noreferrer" target=3D"_blank">christian.masch= [email protected]</a><br> _______________________________________________<br> Edu-sig mailing list -- <a href=3D"mailto:[email protected]" rel=3D"norefe= rrer noreferrer noreferrer noreferrer" target=3D"_blank">[email protected]= </a><br> To unsubscribe send an email to <a href=3D"mailto:[email protected]"= rel=3D"noreferrer noreferrer noreferrer noreferrer" target=3D"_blank">edu-= [email protected]</a><br> <a href=3D"https://mail.python.org/mailman3/lists/edu-sig.python.org/" rel= =3D"noreferrer noreferrer noreferrer noreferrer noreferrer" target=3D"_blan= k">https://mail.python.org/mailman3/lists/edu-sig.python.org/</a><br> Member address: <a href=3D"mailto:[email protected]" rel=3D"noreferrer n= oreferrer noreferrer noreferrer" target=3D"_blank">[email protected]</a>= <br> </blockquote></div> </blockquote></div></div> </blockquote></div> </blockquote></div> --0000000000000d87f405f7ab04e9-- --===============7107119619413045032== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Edu-sig mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/edu-sig.python.org/ Member address: [email protected] --===============7107119619413045032==--