Re: ChatGPT ???

Ditto for all of them, I have some success a while ago using formal proofs of correctness (Coq) and test and linting frameworks, but the truth is that the whole ecosystem has been rendered treacherous by the actions of various actors on the world stage. I have colabs to prove it...

the short answer is - it all depends on who you are, and where you are located, and what you are doing. This is nothing new, but is reinforced by AI itself now.

sovereign AI (indeed compute and data) is paramount, for all

for many, it is too late ...


Get Outlook for Mac <https://aka.ms/GetOutlookForMac>

From: [email protected] <[email protected]> on behalf of Eben Bruyns (as eben at sdk dot co dot nz) <[email protected]>
Date: Tuesday, 30 September 2025 at 06:21
To: David McClain <[email protected]>
Cc: Lispworks HUG <[email protected]>
Subject: Re: ChatGPT ???

I'll see you and raise you!

I had Claude code just flat out lie to me about running tests and the results. I've also had it refuse to do repetitive tasks, the output was "this is too tedious", it wrote a script that did the job wrong and called it production ready. It's honestly uncanny how much it behaves like an over confident bad junior dev! The number of times you'll see "this is perfect, flawless, ship it" when it doesn't even pass basic smoke tests is unreal.

Regards,

Eben Bruyns


On Mon, 29 Sept 2025 at 16:44, David McClain (as dbm at refined-audiometrics dot com) <[email protected]<mailto:[email protected]>> wrote:
I just had a session with ChatGPT, wherein I caught it outright fabricating information to me. Not just once, but three times.

It likes to present examples using Python code. I translated its Python code to Lisp to double check the answers it gives.

I asked it for a function to convert dBSPL to Phons, and back. If you don’t know what that is, it isn’t really important for this discussion. It has to do with Equal-Loudness contours and the Fletcher-Munson curves for human hearing.

By definition, the Phons scale equals the dBSPL scale at 1 kHz. It varies at other frequencies in a nonlinear manner.

But ChatGPT’s model produces 91 Phon for 40 dBSPL at 1 kHz. It is off by more than 50 dB !!

When I told it about the error, it apologized profusely, stating it knew where the error was. GPT said the error was because it didn’t take into account the threshold of hearing.

But that would have produced an error of absolute magnitude less than 1 dB. So once again, it is lying to me. I told it to stop giving me handwaving excuses for its errors.

It apologized again, with another silly handwaving argument, and produced another answer referring to Python’s iso226 model. I demanded to see that Python code and it finally gave me a URL that actually pointed to the source code. That is the final straight answer.

It took me an hour to get a straight, correct, answer from ChatGPT. It willfully misleads. So watch out when using it!!

I cannot imagine a clueless management class individual actually running their company on the promise of such flimsy AI. At best, ChatGPT might serve as an encyclopedia, but should not be trusted unless you know more about the subject that it does.



_______________________________________________
Lisp Hug - the mailing list for LispWorks users
[email protected]<mailto:[email protected]>
http://www.lispworks.com/support/lisp-hug.html