[kde-artists] proposal to losslessly shrink jpeg files

Hello list,

there are quite a few jpeg files in KDE, most of them are wallpapers. By 
optimizing the entropy enconding with jpegtran and by removing thumbnails from 
exif metadata, they can easily be compressed stronger without losing quality.

For example from the kdeartwork-4.8.3 tarball, the file 
kdeartwork-4.8.3/WeatherWallpapers/JK_Bridge_at_Night/contents/images/1600x1200.jpg
has a size of 271474 bytes.

With a simple optimization run with jpegtran, the file size can be reduced to 
265162 bytes. With progressive mode, this can be even further shrunk to 254764 
bytes. Finally removing the exif embedded thumbnail yields a size of 247968 
bytes.

Applying these optimizations to all images saves several megabytes.

One can argue that embedded thumbnails are a good thing because a preview 
picture can be quickly shown. However, only some jpeg files in kdeartwork 
contain thumbnails, and they do not really work good together with the preview 
capabilities of KDE. If KDE does not find suitable embedded thumbnails, it will 
generate its own thumbnails, caching them in ~/.thumbnails. Once the 
thumbnails have been cached, all further preview accesses will be quick, no 
matter if the thumbnail was taken from exif metadata or autogenerated by a 
thumbnailer.

On IRC in #kde-devel, I was advised to propose this optimizations as a step in 
the normal release procedure. For this purpose, I have written the attached 
Python script compress-jpg.py.

It will probably run on any GNU/Linux and most Unix flavors including Mac OS X. 
It will probably fail on Windows, as it uses the function os.rename to 
atomically replace files, which fails on Windows according to the Python 
documentation.

compress-jpg.py makes use of some other programs that must exist to 
successfully run the script:

touch: part of coreutils. If you are using Linux, this is usually installed.
file: widely used command, probably installed on most Linux and Unix systems
jpegtran: from IJG's implementation or from libjpeg-turbo (probably installed)
exif: from libexif (libexif.sourceforge.net), usually in default repositories

And of course you need Python, which today can be found in the default 
installation of most Linux systems. compress-jpg.py should work with Python 
2.6, 2.7, 3.0, 3.1 and 3.2. (I tested it with Python 2.7.2 and Python 3.2.2.)

Its usage is pretty simple: invoke the script in a terminal and pass a file or 
a directory as argument. If you pass a file, this single file is optimized. If 
you pass a directory, the directory and all subdirectories are traversed and 
all jpg files that are found are processed. File format detection is done with 
the file command, so filename endings do not matter. Only files and directories 
will be processed (i. e. symlinks will be ignored).

If no other arguments are passed to the script, it will try optimization with 
jpegtran in progressive mode and in non-progressive mode and will use the 
result with the smaller file. If exif can reduce the file size by removing an 
embedded thumbnail, it will do so. The timestamps and access permissions of 
the files will be kept.

You can influence this behaviour with several switches:
--no-progressive: do not try progressive mode
A file that already *is* saved in progressive mode and can not be shrunk in 
non-progressive mode will stay in progressive mode.

--keep-thumbnail: do not try to remove embedded thumbnails from exif metadata
If there is no thumbnail or no exif information at all, there will not be any 
created.

--new-timestamp: update timestamps if files are modified
I found it useful to shrink my own files while keeping the last "real" change 
of the content. If you want to have new timestamps, then use this switch.

--quiet: suppress almost all text output
Without this switch you will get some status information like which file is 
currently being processed, its old and new size and a spinner when searching 
for jpeg files, to indicate that the program is still alive.

Interrupting the script (e. g. Ctrl-C, or SIGTERM) should be save, i. e. it 
does not corrupt any files, and it should not leave any garbage.

If compress-jpg.py crashes for some reason or it is killed with SIGKILL, there 
might remain a temporary file (its name begins with "tmp"), which should be 
manually removed.

As I am new to Python, you might encounter a bug. Therefore I suggest making 
simple sanity checks (like checking for empty files) before committing any 
changes (maybe you have a tool for easily comparing images? I tried compare 
from imagemagick some time ago, but did not find out how to simply test for 
equality)

If you find a bug or have a suggestion for improvement, please write me an e-
mail, with a subject line preferably starting with [compress-jpg.py].

Regards
Martin Walch
--

______________________________________________________________________________
[email protected] |  https://mail.kde.org/mailman/listinfo/kde-artists

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)

iEYEABECAAYFAk/ABzMACgkQM6S4K8IXn9e+sACg8wbhyN2b5c7Gnb+cHug/1MKy
qIIAoJE8+Cz2M9xXLeeu3ukGSduZd0LC
=WFLO
-----END PGP SIGNATURE-----