DBD::mysql::st execute failed: Out of memory

So I have a query joining two large tables - about 4 million rows each that
is throwing this error.  The query will work fine from the mysql cli, but
when being called from perl I get this:

DBD::mysql::st execute failed: Out of memory (Needed 56657250 bytes) at
./find.pl line 22.
DBD::mysql::st fetchrow_array failed: fetch() without execute() at ./find.pl
line 23.


A restart of mysql fixes the problem for a week or so.  The error log for
mysql also shows an error of:

[ERROR] /mysql/libexec/mysqld: Sort aborted


So it would appear to be running out of sort space.  The machine has 4G of
ram and mysql is only using about 3G.  I have adjusted the sort buffers way
up and it has not helped.  When it fails it always complains that it needs
around the exact same amount (54M).  The perl script was on a different
machine with virtually 4G to its disposal.

I asked on the forums and got no replies.  Any help would be appreciated.
Thank you.