Re: Use of 304 - not modified

(First my apologies for not quoting, didn't really know where to start...)

My two cents worth:

I think investigating possible uses of 304 Not modified would definitely 
make sense, although I have the feeling it might actually turn out to 
not be much of a win. To find out when objects were last modified, you 
will have to 'wake up' all of the objects (retrieve them from the ZODB) 
you want to know the modification date for, which is usually one of the 
most expensive parts of a request. I'm not all that sure whether just 
processing the content on each request will make much of a difference in 
CPU time (although I think it will make a difference in network traffic, 
but I don't think that's the most interesting point to optimize, usually 
the network isn't Zope's bottleneck).

Perhaps using the ZCatalog to store modification times would make a 
large difference, since if we'd do that we could find out modification 
dates *without* 'waking up' objects, but I'm not so sure a setup without 
a ZCatalog would make a Silva system a lot faster per se. Of course it 
would make sense to investigate it better, do a minimal implementation 
and some benchmarking, since performace gains are always *very* 
interesting, but I wouldn't dare making assumptions here...

Cheers,

Guido

Bengt Giger wrote:

>Hi developers
>
>a system manager of a rather large Silva installation, I often think
>about performance. A lot of requests gets answered by the HTTP caches,
>but this leads to other problems, mainly pages in different caches being
>'not fresh enough'; or people, at the same moment, seeing different content.
>
>While looking at the HTTP protocol specs I found a way to optimize CPU
>usage without sacrificing document freshness. The protocol offers the
>'304 - not modified' answer, which was mainly designed to reduce network
>load: a cache will not reload the content of a page, even if that copy
>has expired, if it can be revalidated. Possible revalidation keys are
>Last-modified and ETag.
>
>For sites where even the most dynamic content changes only every few
>minutes, but most of the pages contain some of this dynamic content, a
>page freshness revalidation feature could save a lot of CPU time by
>revalidating pages instead of rendering them completely. And would make
>my users much happier by reducing the base caching time.
>
>Since this issue affects core components and includes the extension of
>the object methods, I would like to open the discussion.
>
>The HTTP header Last-modified must be set correctly, so we can use it as
>a freshness indicator. It depends on the save date of a document. It may
>also depend on some external source objects used in the document. So a
>Silva object should respond on this new method, 'last modified
>question', by internally asking all external object too, since we don't
>(and should not) know about their context.
>
>Every relevant dynamic component must provide a method which compares a
>given date to its own freshness, and gives back what it thinks would be
>the date for a document generated right now. This leads to an overall
>Last-modified date which can be used for validation.
>
>If a cache (public or browser internal) requests a new copy of a page,
>we add a relative freshness timer, the HTTP max-age header. After
>max-age, a number of seconds, has expired, the cache will ask the
>upstream server as usual, but add a 'If-modified-since' header.
>
>If we detect such a header, we could ask a document to compare its date
>with the content of the header. Maybe the Zope server can end the
>request by returning 304 without any content. Depending on what kind of
>site you have, the CPU savings could be huge: compared to the complete
>XML-HTML transformation,
>
>- date lookups are cheap
>- many external sources don't care about time (formatting stuff)
>  and would simply return the same value: very cheap
>. others will change regularly on a adjustable base: a comparison
>  with the value of If-modified-since is very cheap, too
>
>The 304 answer will make caches and clients happy, and due to the
>relative max-age header, the timer will be reset and start again. A
>proxy server can use the revalidated copy for answering further
>requests. So we could keep caching times low, while the proxy caches
>serve the same content even for weeks, as long as it validates.
>
>Regards
>
>Bengt
>
>_______________________________________________
>silva-dev mailing list
>silva-dev-IAPFreCvJWM6s/[email protected]
>http://lists.infrae.com/mailman/listinfo/silva-dev
>
>  
>