Theora Fast Quality Reduction Transcoding

Hi.

I have been working on a tool whose goal is to reduce the bit rate of theora
video by decoding to DCT coefficients, reducing the entropy of the
coefficients, and re-tokenizing the stream.

I have successfully used the decoder source to extract the DCT coefficients
for each block, and I am able to capture any and all relevant information
about where the block of coefficients falls in the image, the frag list, the
MCU and so on. That is to say the entire decoder state for a given block of
coefficients is available to me in a callback.

I had thought at this point to use the tokenizer directly to construct a new
token stream. However the complexity of that module alone is daunting. It
relies on state information retained and perhaps also generated by the
encoder. I think the retained information like the huff tables and maybe
even lambda can be constructed easily enough. But I am concerned about the
possible use of generated data since we are not doing a full encode, and
that there may be traps waiting to be sprung on me.

I have also considered running a full encoder but without any real image
data being fed to it. The processing required to generate coefficients would
be replaced with a callback to retrieve coefficients generated by the
decoder. However, this approach is very likely to be unworkable as, at the
very least, the inter frame motion vectors and quite probably other useful
information would be missing without further intervention from me.

So I am still inclined to use the tokenizer directly on the modified
coefficient stream. I would be very grateful if anyone knowledgeable about
the tokenizing process or the encode process in general could offer advice
or warnings about either approach I described.

Thanks in advance,
Chris