Kmail/Kontact slows to a crawl during long VNC sessions?

I have noticed a problem that I thought was not Kontact related but which now, having noticed this for years, think perhaps it IS something to do with Kontact/Kmail...when running over a VNC session, it works fine for 30 minutes or so and then slows to a point where each letter takes seconds, minutes even, to appear.  My process is thus:

1.  Start Putty and TightVNC on local windows machine (skip to 2 if using linux locally)
2.  ssh to my home server  (myserver.mydomain.net).  
3.  Connect to localhost (which tunnels to the VNC server running on home machine)
4.  From the VNC session, I open a terminal and type "ssh -X myserver kontact"
5.  I wait a second, Kontact opens up.

For a few minutes or a few emails...sometimes 1-5 minutes and sometimes 20 or more...everything is fine.  Then quite rapidly the response drops to a lag that makes it impossible to work well.  

I am not sure this is actually connected to Kontact but I noticed just recently that when I leave the VNC session running but close Kontact after each time I send or read email, that the problem appears to go away unless I am reading/sending email for a long time.

Other applications seem to work just fine too.  I am posting this as a separate issue because I don't think it is connected to Akonadi...it has occurred for years on previous versions of Kmail.

Many thanks for help/thoughts/suggestions!