UCI misc

I think I can tackle some of these.  I've CC'd the uci list where this 
discussion should probably continue.


Said by Steve Lhomme <[email protected]>:
--------------------------------
- What is the difference between the API and the CPI (Codec Programming
Interface)

The API is the interface used by programs that want to use the codecs.  
It encapsulates functionality that all programs want that no codec 
implementer should have to write.  (or that shouldn't have to be 
written by every codec implementer) The CPI is the interface that the 
codec implementer is to support.

On BeOS the API corresponds to MediaDecoder/MediaEncoder and a few 
others, and the CPI is the formerly private interface that I am working 
on for OpenBeOS.

- Can you allow an input of 1 type and 2 output of different types
(audio + video), like for decoding DV data; unless you see DV data as a
container, stored in a container (AVI, MCF, whatever)

I think that the latter is a common & reasonable way to view DV data.

- I also see that you plan to use the BSD license. Is it subject to
change ? We have chosen LGPL on libmcf for different reasons. One is
that we want to avoid public forks of the library. The viral aspect of
LGPL being a good thing too.

For usability with OpenBeOS, BSD is preferred.  It's possible that LGPL 
is also compatibile with the OpenBeOS license although everyone I talk 
to is fuzzy on the issue.  (same here)  BSD may also be preferred to 
encourage hardware (mpeg2 cards, etc.) developers to supply UCI codecs 
for their cards.

- for video encode_supported_formats must output RGB or YUV only. But 
it
could decode to... MPEG (could be usefull for DV and then chaining
codecs, even though I don't know if MPEG is used in DV). And no mention
of audio :'(

If it is "decoding" to mpeg it obviously hasn't finished it's job. :-)  
Anything that is decoded should be a bitmap, effectively speaking.  The 
specifications so far do seem to lack a bit in the audio department.  
And as far as format descriptions go this is a good point.  There 
should be different format descriptions for video and audio.  (a union 
here works well)

- according to the codec_type parameter, I assume this is very
monolithic. ie you couldn't have audio on input and output video.

That doesn't sound like any codec I know. :-)  It does sound like some 
filters I know but that is outside the scope of this project. (as I 
believe it should be)

- in the *codec_format_info* structure, what is the *id* field for ?

I assume it's a unique id assigned to the codec on load time by the 
codec management library.

- in the same structure, fourcc might only be usefull for VfW video
codec and QT codec. Not at all for audio. For ACM, it is a 16 bit ID.
Maybe this should be a union depending on video or audio.

I'd prefer to just carve up the space of codec types.  Like this for 
example:

enum {
   CODEC_FMTID_VIDEO_BASE = 0,

   CODEC_FMTID_FOURCC,
   CODEC_FMTID_DSHOW,
   ...(more ids follow)

  CODEC_FMTID_AUDIO_BASE = 0x100000,
  CODEC_FMTID_WAV,
  CODEC_FMTID_ACM,
  ...(more ids follow)

  CODEC_FMTID_<some other>_BASE = 0x200000,
  ....
}

This is way more bits than we need to represent all the codec id types 
that exist or will exist in the reasonable future.  Having the video 
and audio codec id types in the same enum means that you don't have to 
have another field to distinguish whether it is video or audio type.  
It also means that when you visually look at the number you can tell 
which category it falls into.

- The binary data that the codec output (on encoding) or input (on
decoding) from the container, it depends on the input/output used (VfW,
DShow, MCF, whatever). The current CPI is not clear on this (or maybe
doesn't support this).

Hmm.. I don't understand this.

- on a programming note, as you know I like C++. Especially because you
use SO many pointers -> the way to allow bugs.

I like C++ too and the UCI interface will have a clean C++ wrapper on 
OpenBeOS in any case.  It would probably be a good idea to have a 
standard C++ interface to supplement the C API.  Certainly there will 
be people who will prefer a C++ API even if there is no C++ CPI.  Other 
codecs already have separate supported interfaces for C and C++. (for 
example: FLAC)

- There is no timecode mention on encode or decode. I strongly advise
you that there should be one ! This is the basis for good containers.

In addition to timestamps, frame numbers and other information which 
may be embedded in the encoded data should also be supplied.  It seems 
like these would be returned in the dresults codec_params.  BTW, there 
should probably be a set of standard parameters covering all of the 
common parameters such as bitrate, horizontal resolution, vertical 
resolution, frame numbers, etc.

- Nor there is a mention of latency of the codec (as found on BeOS)
which can help the system to better handle buffers and minimize 
latency.

And latency. :-)

- On the same level, there is no way of choosing between 2 codec that
both handle MPEG4 decoding. (like a priority number)

I suggest self-reported capability ratings like the beos "quality".  
Generally speaking this is a tricky issue because the decision between 
two codecs ultimately rests (or should rest) in the hands of the user.  
So there has to be a user interface component here, which complicates 
things a little bit.  This interface component could be as simple as a 
command line application to list the loaded codecs and their priority 
ordering, and then allow modifying the priority of the codecs.  Such an 
application could display self-reported capability ratings and even 
allow the user to prefer a codec with a lower capability.

- "# Transparently perform some common data conversions to allow 
various
applications to interact with various codecs easily" I suggest not to
use anything transparent. Or at least not customisable. Because it can
be good to have different transformation "filters" that would be either
fast or best quality (for example). Chaining of codecs (you can call
that filters if you prefer) is better and more flexible.

I'm not sure which conversions this refers to, but I believe that a 
colorspace transform component to the codec UI would be very valuable.  
That is, a different CPI-like interface for straight colorspace 
transformations.  Then someone can write the world's fastest RGB->YUV 
conversion and all the codecs can benefit from it.  As Steve points 
out, these transformations should be optional and controllable from the 
API.

Andrew Bachmann






-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf