How do you compress roundup traffic?

Hi all:

I believe there are some python gurus here and wsgi people as well who
may be interested.

It may come as a surprise to some of you that roundup doesn't compress
its output using gzip, brotli etc. even if the client says it can
handle it.

So how are people compressing roundup assets? Static assets I assume
can be done by detecting @@file in the url and directing the server to
serve up the file possible with compression. Some other techniques
are listed under the header below.

I have a patch in development that enables gzip and brotli compression
on the fly for the dynamic http output and all served files
(@@file/...) as well as content served by designators like
.../file1. Gzip is part of the standard library, but you need to
install brotli.py for brotli support.

I am looking for a couple of testers for the code. Sadly it is still
too new to make it into the 2.1.0 release coming up next month. It is
also feature poor at the monent and doesn't support:

  * configuration settings, you have to edit the code to set
    a different compression level for example.
  * using pre-compressed assets. A request for
    "@@file/foo.js" should search for 'foo.js.br' if the
    'Accept-Encoding' header includes 'br'.  If 'foo.js.br' is found,
    it gets read from disk and returned. This way you can statically
    compress assets using maximum compression and not pay a runtime
    cost.
  * disabling compression methods or limiting compression to only
    html or ...fileN or @@file static assets.

For those not interested in trying th patch, how do you configure your
systems to compress roundup output?

Also I am interested in how you have roundup configured in
general. That will help the developers focus on the most used
deployment scenarios for future releases. I am looking for info on:

  web server: apache, nginx, hiawatha, roundup-server ...
  ssl deployed: yes/no
  roundup execution: mod-python,
         wsgi server (e.g. gunicorn, uwsgi, cherrypy)
	 roundup-server (http proxy),
         cgi ...
  compression: which assets are compressed and what settings you use.

I appreciate your feedback especially if you are usually a lurker on
the mailing list and don't post.

For other ways to configure compression and if my patch is even
permitted read below this header. 

Other Compression Methods and Should Roundup Compress??
=======================================================

One of the articles I saw discussing compression for wsgi claimed that
a wsgi app is not supposed to encode/compress the output. That's a job
for the wsgi caller/parent or middleware (which is just another wsgi
app so....). I assume this restriction came from (a misreading of)
https://www.python.org/dev/peps/pep-3333/:

   Similarly, a server may re-encode or transport-encode an
   application's response, but the application should use a suitable
   content encoding on its own, and must not apply a transport
   encoding.

But I think a transport (which I read as transfer) encoding is
something like chunked or gzip for hop-to-hop connections and is set
using the Transfer-Encoding not the Content-Encoding header. The list
includes gzip and other compression methods and I think this caused
confusion.

In setting up compression of resources (css, js etc.) my uwsgi
configuration works fine for gzip but there is no support for brotli,
zstd.... Nor can it compress the html response from roundup or
attached files (e.g. .../file1). (auto-gzip doesn't work because the
'content-length' header is set. Maybe we need a new config setting? )

The whitenoise (http://whitenoise.evans.io/en/stable/base.html) wsgi
middleware can be used to compress static assets, but that still
leaves the dynamic html responses uncompressed.

nginx can enable compression on proxied assets (html etc.)
(https://reinout.vanrees.org/weblog/2015/11/19/nginx-proxy-gzip.html
and https://docs.cherrypy.org/en/latest/deploy.html#id4). Again this
seems like gzip only
(https://docs.nginx.com/nginx/admin-guide/web-server/compression/) so
no zstd, brotli etc.

I believe apache with mod_wsgi and --compress-responses will compress
text based data on the fly. Not sure what modes is supports,
documentation is lacking.

Are there other techniques folks are using?

If you have read this far, I hope you have a great day.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.