Fix for 687364 rangecheck in buildshading2 from Quark XPress PS file

Fix for 687364 rangecheck in buildshading2 from Quark XPress PS file.
This fix involves the creation of the findcmykcustomcolor and
setcustomcolor functions.

DETAILS:

Ray found that the error message was due to an invalid shading
dictionary being given to shfill.  The dictionary has a function
which has a single input and a single output.  The numbers are
determined from the sizes of the Domain and Range arrays.  Since
the shading dictionary is using a DeviceCMYK color space, Ghostscript
is generating a rangecheck since the function does not have four
outputs.

The file is generating an invalid shading dictionary.  Specifically
it generates:

sh:  << /Coords [0 0 0 1] /ColorSpace [/DeviceCMYK] /Function <<
/BitsPerSample 8 /Ra
nge [0 1] /FunctionType 0 /Domain [0 1] /DataSource
(\200\200\200\201\201\202\202\203
\203\204\204\205\205\206\206\207\207\210\210\211\211\212\212\213\213\214\214
\215\215\
216\216\217\217\220\220\221\221\222\222\223\223\224\224\225\225\226\226\227\
227\230\2
30\231\231\232\232\233\233\234\234\235\235\236\236\237\237\240\240\241\241\2
42\242\24
3\243\244\244\245\245\246\246\247\247\250\250\251\251\252\252\253\253\254\25
4\255\255
\256\256\257\257\260\260\261\261\262\262\263\263\264\264\265\265\266\266\267
\267\270\
270\271\271\272\272\273\273\274\274\275\275\276\276\277\277\300\300\301\301\
302\302\3
03\303\304\304\305\305\306\306\307\307\310\310\311\311\312\312\313\313\314\3
14\315\31
5\316\316\317\317\320\320\321\321\322\322\323\323\324\324\325\325\326\326\32
7\327\330
\330\331\331\332\332\333\333\334\334\335\335\336\336\337\337\340\340\341\341
\342\342\
343\343\344\344\345\345\346\346\347\347\350\350\351\351\352\352\353\353\354\
354\355\3
55\356\356\357\357\360\360\361\361\362\362\363\363\364\364\365\365\366\366\3
67\367\37
0\370\371\371\372\372\373\373\374\374\375\375\376\376\377) /Order 1 /Size
[256] >> /S
hadingType 2 >>

The file was sent into Adobe Distiller 6.0.  Inside the resulting
PDF file there is a very different shading dictionary:  Specifically:

/sh: << /ColorSpace [/Separation /MEZZO__1 /DeviceCMYK -array-] /Coords [0 0
0 1] /Function << /Decode [0 1] /FilePosition 460412 /Range [0 1] /Filter
/FlateDecode /BitsPerSample 8 /Encode [0 255] /Length 255 /FunctionType 0
/File -file- /Domain [0 1] /DataSource
(\200\200\200\201\201\202\202\203\203\204\204\205\205\206\206\207\207\210\21
0\211\211\212\212\213\213\214\214\215\215\216\216\217\217\220\220\221\221\22
2\222\223\223\224\224\225\225\226\226\227\227\230\230\231\231\232\232\233\23
3\234\234\235\235\236\236\237\237\240\240\241\241\242\242\243\243\244\244\24
5\245\246\246\247\247\250\250\251\251\252\252\253\253\254\254\255\255\256\25
6\257\257\260\260\261\261\262\262\263\263\264\264\265\265\266\266\267\267\27
0\270\271\271\272\272\273\273\274\274\275\275\276\276\277\277\300\300\301\30
1\302\302\303\303\304\304\305\305\306\306\307\307\310\310\311\311\312\312\31
3\313\314\314\315\315\316\316\317\317\320\320\321\321\322\322\323\323\324\32
4\325\325\326\326\327\327\330\330\331\331\332\332\333\333\334\334\335\335\33
6\336\337\337\340\340\341\341\342\342\343\343\344\344\345\345\346\346\347\34
7\350\350\351\351\352\352\353\353\354\354\355\355\356\356\357\357\360\360\36
1\361\362\362\363\363\364\364\365\365\366\366\367\367\370\370\371\371\372\37
2\373\373\374\374\375\375\376\376\377) /Size [256] >> /ShadingType 2 >>

This dictionary has a Separation color space.  The given function
is a reasonable function to use with the Separation color space.
As Ray mentioned, it is not correct for a DeviceCMYK color space.

The file is generating a different dictionary for the two interpreters.
In the case of Ghostscript, the dictionary is invalid,

Backtracking through the various procsets inside of the given file
shows that is a mess of function definitions, followed by replacement
of the functions with new definitions.  Some of it may be Quarks.
Some of it is labeled:

%%BeginResource: file Jaws_custom_colour_blend_level3_emulation 2.1 0

The results of the search are:

1.  Ghostscript does not define 'setcustomcolor' but Adobe Distiller
6.0 does define it.

2.  The procsets in the file do provide a definition for
setcustomcolor.  Actually the file provides about 16 versions of
setcustomcolor.  However the versions provided in the file do
not work.

The PLRM does not mention setcustomcolor.  However it is
described in Adobe's "Technical Note #5044 Color Separation
Conventions for PostScript Language Programs".  TN 5044 says:

"The following "operators" are not defined in the PostScript
Language Reference Manual, but should be used as pseudo-operators
in your PostScript language output. Separation applications from
Adobe Systems and other vendors will redefine these convention
operators to separate your documents.  Your application should
conditionally define procedures with these special names, as shown
later in this document."

The fix to the problem is to provide the functions:
findcmykcustomcolor and setcustomcolor.  These are placed at the
end of the lib/gs_lev2.ps.  They are placed there because the
implementation requires Separation color spaces which is a level
2 feature.


Dan


The following code fragment is to be inserted into lib/gs_lev2.ps
just prior to the end of the file prior to the line that says:
  "end				% level2dict"
I am showing the code fragment like this instead as a diff simply
for reading ease.

% The following functions emulate the actions of findcmykcustomcolor and
% setcustomcolor.  These functions are described in Adobe's TN 5044.  That
% same document also says "The following “operators” are not defined in the
% PostScript Language Reference Manual, but should be used as
pseudo-operators
% in your PostScript language output. Separation applications from Adobe
% Systems and other vendors will redefine these convention operators to
% separate your documents.  Your application should conditionally define
% procedures with these special names, as shown later in this document."
%
% In the past we have not provided these pseudo operators since they are not
% a part of the language specification.  However we have found files from
% Quark Express which produce bad shading dictionaries if these operators
% are not defined.

% TN 5044 does not define the contents of the array.  We are simply putting
% the values given into an array.  This is consistent with what we see when
% testing with Adobe Distiller 6.0.
%   <cyan> <magenta> <yellow> <black> <key> findcmykcustomcolor <array>
/findcmykcustomcolor { 5 array astore } bind def

% Build a tint transform function for use by setcustomcolor.  This function
% is for a Separation color space which has a DeviceCMYK base color space
% (i.e. 1 input and 4 outputs).  The input to buildcustomtinttransform is
the
% array created by findcmykcustomcolor.  The resulting function is:
%   { dup cyan mul exch dup magenta mul exch dup yellow mul exch black mul }
%   Where cyan, magenta, yellow, and black are values from the array.
/buildcustomtinttransform	% <array> buildcustomtinttransform <function>
{ [ /dup load 2 index 0 get /mul load
  /exch load /dup load 6 index 1 get /mul load
  /exch load /dup load 10 index 2 get /mul load
  /exch load 13 index 3 get /mul load
  ] cvx bind
  exch pop				% Remove the input array
} bind def

% Set a custom color based upon a tint and array which describes the custom
% color.  See finccmykcustomcolor.  First we create and then set a
Separation
% colorspace.  Then we set the specified color.
/setcustomcolor			% <array> <tint> setcustomcolor -
{ % Start building Separation colorspace
  [ /Separation 3 index 4 get	% Get separation name from array's key
  /DeviceCMYK
  5 index buildcustomtinttransform ]	% build the tint transform function
  setcolorspace			% Set the Separation color space as current
  setcolor				% Set the tint as the current color
  pop					% Remove the input array
} bind def