Re: Bug 687560 "Invalid PDF if /BP pdfmarks withnon-unique /_objdef")

Dear SaGS and other reviewers,

Likely we need another round of this tour, starting form the beginning.

1. First of all I need to state that the reported problem doesn't happen when
processing an output of popular PS writers. At least nobody could bring
a practical example.

2. In same time I can bring an example how to rise the problem
artificially, using an putput of a public PDF writer, which contains any objdef.
Let the document file name is a.ps. To rise the problem run :

    gs -sDEVICE=pdfwrite, sOutputFile=a.pdf a.ps a.ps

Since the document is being processed twice in same session, the objdef 
with same name appears twice in same session. Also there is a similar case,
when two different document are being concatenated, both define a specefic object
using same name, but objects appear different. This example gives us
a practical case, and likely gs generates an invalid output for it.

3. There are two ways to resolve the case (2) :
3.1. Perform a specal namespace cleanup before the second
document is started;
3.2. Define a variable-like behavior for objdef.

4. The appriach (3.1) requires a change to the device interface,
because now it doesn't pass a "start document" signal.
Such kind of changes must not get bountiable status, 
since a resolution requires a high reliabiliry.
I have no strong opinion, whether we need (3.1) or not,
but I would vote for it, because the signal would be useful 
from a general point of view.

5. Independently on whether we need (3.1), we could work on (3.2).
Another consequence of the example above is:
an object definition MUST PRECEED any object usage,
otherwise concatenating two correct documents 
with same object name and different values will give a
wrong result (unless we implement the signal; 
but as explained above, now we have no signal).

6. So now we get a strong consequence, that we need a variable-like 
behavior without forward references. It may be implemented as 
I suggested a week ago :

6.1. When a name is being defined at the first time,
create the object, store it with the given name in the object directory
(I mean the distill-time association of distill-time names with cos objects).

6.2. When an usage happens, the object exists and has a specific ID.
Create a reference to it as usial by the object directory;

6.3. When the object name appears in objdef at second time,
create a new object, and replace the association in the object directory.
The old object continues to exist.

6.4. When another usage happens later, use the object directory
to create the reference to the last object assotiated.

6.5. If an usage happens when no object associated in the object directory,
signal error and discontinue the processing (since at this moment 
we don't know whether we'll start another document or not, we can't resolve
whether this an error or a forward reference. A practical
resolution for now is to emit error. Maybe we'll do more for it after 
(a) the "start document" signal is implemented and 
(2) we recieve practical examples).

6.6. For future development inmplement a reserved function,
which purges the object directory; It will be called after we
implement the "start document" signal.

7. The approach (6) does not need any surgery or a fundamental change of
pdfwrite invariants. In same time, patches suggested by SaGS,
do change basic invariants, such as the resource lifetime and semantics.
Therefore I would approve an implementation of (6), but I can't approve
SaGS's patches (well, my opinion would be different if someone brings
a practical example of a document with a non-unique objdef
denerated by a popular PS writer, but as I already said, we did not
recieve any).

8. There exists a disirable optimisation, which
merges equal instances of  objects defined by consequtive objdefs.
Particularly it would be useful if a document is being concatenated with itself.
To implement this optimization I would use same technique,
as pdfwrite uses for other resource types : write a new object to a stream,
then compare the stream for equality with existing streams.
If en equal one appears, replace the new object with the old one.
This technique already works for most types of cos objects,
and I woild approve it for XForm objects created by pdfmark. 
But I would consider this optimization separately from the reported problem,
and since it is big enough, I would assign a separate bounty for it.
(Let me skip the details of an implementation of the optimisation
from this message just to simplify the subject. 
On request I'll post them separately).

9. So IMO now there exist several alternatives :

9.1. SaGS or somebody else will submit a patch for (6),
I'll approve it, commit and Artifex shall pay the bounty.

9.2. Somebody submits a patch, which just signals error and stops.
I would approve it as a temporary solution, since
there is no claims from customers. But I'm not sure that
the full bounty sum is good for such small patch.

9.3. We'll cancel the bountiable status,
pay to SaGS for the problem analysis and trial implementations,
patch with (9.2), open an enhancement project with the
"start document" implementation, assign a proper priority to it 
and put into the queue.

10. Sine we already spent too big time for this problem,
now we strongly need to turn it to a productive way.
I'll not consider any patches until we decide,
which way to choose. I'm sorry.

--------------------------------------

Well, now I explained my position.
Please ask questions if any point of it appears unclear.
If you have a different opinions, please bring it 
after all points of my one are clarified. Thank you.

Igor.