slow 304 response

I'm developing a server locally, and using 

user:file_search_path(static_files, assets).


http_reply_from_files(static_files(.), [], Request).


to serve the swipl owl the same size as off the swipl web site. Sometimes the server responds slowly.

After some amount of trying various server configurations, etc I discovered that the slow loads correspond
to the server responding 304 NOT MODIFIED   These are taking 2.00 sec on my machine, while 200 OK responses
are taking 5 msec. 
Firebug on Firefox reports this sequence of events:
within a few msec of the request, the html comes back.
(presumably shortly afterwards we request the js and css resources from the head)
1 second later the js and css come back, actual transfer taking a few sec
1 second after that the image comes back.

 The round number of 2.00 sec is quite reliable, which leads me to suspect the 1 sec is some constant deep within swipl http. 

directly asking for the owl (that is, typing localhost:7777/f/swipl.png into the URL bar) makes it always come back immediately, even if it 304's.

I've tried it with both chrome and firefox.