Re: Just take me out back and shoot me...

Er, yeah… Training against the Internet contents, which is rapidly becoming saturated by content generated by AI bots… 

I see this as the main Achilles heel of current AI systems. It will soon implode from the burden of its own crap… I’m sure many artists and musicians will be rejoicing when that happens.

- DM


> On Aug 7, 2025, at 13:28, Eben Bruyns (as eben at sdk dot co dot nz) <[email protected]> wrote:
> 
> The tools are constantly apologising and agreeing with me, but I've also seen it do this stuff in a loop:
> 
> gives me bad code
> me correcting it
> it agrees
> gives better code with another bug
> me pointing it out
> agrees
> goes back to fist bad code
> 
> This is a bit of a simplified version of how it goes, it's normally a few iterations before we're back at the first bad "idea" it had.
> 
> One thing I've also noticed with the "test your code" approach is that it will mock the data and call it good. My theory is that this is because of the TDD fad where every man and his dog made the moc tests that did nothing other than test data flow (I didn't say correct data flow). I'm guessing it has a lot to do with the training data that was used. So my hypothesis is that if they just trained it on the internet's code, we're going to have the same bad results you get when you copy and paste stack overflow code. That is now compounded by the fact that AI is producing a LOT of if not most of this bad code on the net which then feeds back into the training loop. I think the overall quality of these tools will decline as it converges on all AI generated code over time...
> 
> I'm both pessimistic and optimistic about this tech. I think there's a valid use case, I'm just struggling to nail down the specifics of it.
> 
> Regards,
> 
> Eben Bruyns
> 
> 
> On Fri, 8 Aug 2025 at 08:21, David McClain <[email protected] <mailto:[email protected]>> wrote:
>> > to find bug was casually introduced by these AI systems being "helpful". They've created many subtle race conditions that took hours or days to debug, and once I looked at the code I realised that I would never have written it that way to start with.
>> 
>> Yes, I noticed the race conditions being generated by ChatGPT this morning, and I called its attention to this defect. It then apologized, agreeing with me, and regenerated a safer version. 
>> 
>> So you really have to know more than the tool knows, at least at a fundamental level. You may not know as much about how all the pieces fit together (thinking of CAPI and GP here). But you have to be on the lookout for sloppiness, and offerings of consensus opinions as absolute facts.
>> 
>>