Fix for 687696 PDF 1.4 transparency can use large chunks of memory.

Fix for 687696 PDF 1.4 transparency can use large chunks of memory.
Prior to this fix the handling of PDF 1.4 transparency features used one
or more full size image buffers.  With high resolutions and large images,
these buffers can be multiple gigabytes.  As a result, we were not able
to handle these files.

DETAILS:

PDF 1.4 transparency is handled by a device which is inserted into the
device chain.  Prior to this fix, this device was via the 'device filter'
logic.  With this fix, the PDF 1.4 transparency device is inserted via
the compositor device mechanism.  This change was made since the device
filter logic did not allow other devices to control of where the PDF 1.4
transparency device is placed in the device chain.  There is a device
proc which implements the create compositor action.  This allows more
control than the device filter logic.

The first step in this fix was to convert the PDF 1.4 transparency
logic to use the create compositor mechanism.  For situations in which
the clist logic is not being used, the PDF 1.4 transparency device is
created and placed in the same location in the device chain as it was put
by the device filter logic.  In this situation, there is very little
logical difference in the operation of the PDF 1.4 transparency.

For situations in which the clist is used, there are two PDF 1.4
compositing devices created.  There is one device before the clist when the
clist is being written.  There is a second device after the clist when
the clist data is being read.

The device before the clist serves two purposes, it provides a means for
implementing the process color model for the PDF 1.4 compositing.  (The
PDF 1.4 compositing may used a different process color model from the output
device.)  Thus this device has color_info and encode/decode color procs
which match the PDF 1.4 blending color space.  This device also ensures
that the PDF 1.4 blending parameters (blend mode, shape, opacity, etc.)
are passed through the clist to the second PDF 1.4 device.

The second PDF 1.4 device (the one when the clist is being read) is the
same PDF 1.4 compositing device which is used when the clist is not being
used.  This is the device which does the blending operations.  Since the
device is after the clist, the buffers that it needs are the size of a
single band instead of the full page.  This device is placed prior to
the memory device which is used for drawing the raster data.

Two changes were made in the 'create compositor' logic.  The imager state
was being passed as a 'const'.  It is no longer const since the PDF 1.4
transparency compositing changes the cmap procs in the imager state to not
use transfer functions during color calculations.  The transfer functions
are applied later (and the cmap procs restored) when blended image data
is output from the PDF 1.4 compositing buffers.

The second change to the create compositor logic is the addition of two
more procedures to the gs_composite_type_t.procs list.  These procedures
are used by the clist device create compositor routines.  The first
is used to create the PDF 1.4 clist write device.  The second saves and
restores the color_info field for the clist device (see below).

Default versions of the added gs_composite_t.procs were also added for
use by the other compositor devices.  The defaults are no-ops.

The clist logic has a few changes.  Since the PDF 1.4 compositor device
may use a different process color model from the output device, the number
of colorants and the depth (bits per pixel) can be different for the PDF
1.4 compositor and the output device.  Thus changes were needed in parts
of the clist logic which assumed that these items were constant during the
processing of the clist.  An extra field was added to the device halftone
to indicate the number of colorants in effect when the device halftone was
created.  This data is included when the halftone is serialized for the
clist.  (Fortunately the PDF 1.4 compositing does not use halftones or
transfer functions.  So the device halftones and transfer functions do
not need to be changed when the PDF 1.4 compositor is installed.)

As previously mentioned the create compositor procedures for the clist
writer and reader now call a couple of compositor specific routines.  These
allow the compositor to implement actions related to the clist device.
(The PDF 1.4 compositor uses these routines to create the 'write clist'
PDF 1.4 compositor and to save and restore the clist device color info.
The color_info field of the clist device is changed to match the process
color model of the PDF 1.4 compositor while the PDF 1.4 compositor is active.

A additional device parameter (PageUsesTransparency) was added.  This
parameter is sent by the PDF interpreter to indicate if PDF 1.4 transparency
will be used on the page.  The banding/no banding decision and the size
of a band includes an estimate of the size of the PDF 1.4 blending buffers
if this parameter is true.  Note:  The estimate of the size of the
blending buffers is not exact since this calculation is made before the
actual number of buffers, the number of blending colors and alpha channels
is known.  Fixed values are used for the estimated buffer size.  This is
still much better than the pervious situation, in which the only the
output raster size was being used.  The blending buffers can easily be
30 or more times larger than the raster buffer.

This fix also involves the removal of the PDF 1.4 'marking' devices.
The marking devices were used to actually implement the pixel marking.
The marking devices were created at the start of each high level drawing
operation and deleted after the completion of the operation.  The marking
device action has been merged into the primary PDF 1.4 compositing device.
This change eliminates some device creation overhead.  The overhead would
have been worse since most information about high level drawing operations
is not passed through the clist.  As a result it would have been necessary
to create marking devices for each low level operations.



The following issues are not included in this fix but need to be considered.

1)  The device filter logic is no longer used.  It may be desired to remove
this logic.

2)  We currently have a 'create_compositor' device proc.  However there
is not a 'delete_compositor' device proc.  As a result, compositor devices
can stay in the device chain even when they are no longer needed.  The
current fix turns the PDF 1.4 compositor into a 'forwarding' device.  This
is the same method used by the overprint compositor device.  I do not like
leaving these devices since there is some extra overhead and there may be
possible side effects.  (Testing has been done to try to find side effects
and none has been found with the current code.)

3)  This change only affects devices which can use the clist logic.  This
excludes the 'high' level devices.  These include the 'display', 'x11',
and 'pdfwrite' devices.  Thus these device will continue to use a full
image buffer.  It is possible to extend this fix to use a clist device
for the PDF 1.4 device for these devices.


Dan

_______________________________________________
gs-code-review mailing list
[email protected]
http://www.ghostscript.com/mailman/listinfo/gs-code-review