Re: [gst-devel] Re: Common Opensource codec API

On Wed, Dec 31, 2003 at 03:27:13PM +0200, Tuukka Toivonen wrote:
> On Tue, 30 Dec 2003, Artem Baguinski wrote:

> I try to enlighten you. 

thanks

> Everything that happens to video/audio/multimedia
> can be divided into three parts:
>
> - Source
> - A pipeline of format conversions
> - Sink

everything that happens in your computer is:

- getting some input
- processing it
- creating some output

i'm afraid you're taking Keep It Simple principle to the extreme.

[watching tv in mplayer skipped]
>
> What I mean is that vloopback could in principle replace codecs in mplayer.
> Have some new weird codec in AVIs? Why bother adding support only to
> mplayer, when one could write a program that decodes frames from the AVI
> and outputs to all other programs from /dev/video1 using vloopback?
> 
> This would work with every program, since V4L API is quite well supported.
> In fact, V4L with vloopback could be considered a kind of plugin standard
> in Linux. But the problem, probably, would be the low efficiency and
> flexibility.

i agree that this is a nice thing to think about, yes. exchanging
media, especially such heavy one as video, between programs is one of
the issues piksel group is working on. we had to reject vloopback though
because of non-portability. 

as for separating the whole video playback into little simple programs
connected with vloopback [or similar] - that's not such a good idea.
the work of the input / demux / decoder / displayer and whatever you
wanna insert inbetween has to be synchronized, they have to know when to
stop, when to queue incoming data, when to skip and only media input /
media output don't give enough information about that. 

dont forget that you have an unpredictable user who will wanna change
sources / parameters / seek and he wants to see the results of his work
as soon as possible. 

running a bunch of simple processes to watch video eats more resources
then, their synchronization and exchange of [other then media] data and
mutual control is much more difficult to program. the only case when you
should use vloopback-like exchange is where exchanging processes have
minimal dependancy on each other - which is one of golden rules of
complex systems design anyway.

that's basicly why i wondered what vloopback has to do with common codec
api. 

> So no, I don't think vloopback is a good long term solution, although it
> has one good thing: compatibility with most programs.

?

> Kind of another extreme is mplayer, its plugins are efficient but they are
> bound to a single program. Except that, of cource, libavcodec is not just
> for a single program. But I'm still a bit suspicious whether it could
> provide the necessary format conversions. Are dynamically loadable plugins
> possible? Can the formats be well specified, i.e. if an AVI contains frames
> in unknown format to mplayer, can mplayer be made aware of the format by
> just writing libavcodec plugin or does it require modification of mplayer
> too?

i haven't look into mplayer for a couple of years but i guess you'll
have to add something to mplayer code as well. all different projects do
format detection differently and have different plugins registry
implementation, so there must be some bit of a avcodec wrapper for
mplayer that takes care of mplayer-specific metainformation about the
wrapped codec.

> On the other hand, if implementing good codec/format conversion plugin
> system, why not enhance the API slightly to allow also codecs with no
> input, just output, and codecs with only input, no outputs--or source and
> sinks.


and again it won't be "codec api", it'll be something like "media
pipelining api" which indeed sounds like ...

> Enter Gstreamer.
>
> >> Yeah, and Gstreamer. Probably others too. Multiplicity is the problem.
> >Gstreamer isn't a codec api, it's a glue layer implented in C.
> 
> Glue layer? Gstreamer is an API for multimedia pipelines consisting of
> source, format conversions and sink. And certainly format conversions
> include codecs too.

hmm. didn't i just agreed on that? :-)

> >i guess before trying to invent it people should look at OpenML i
> 
> Hmm, good idea. Thanks for this, i didn't know about it.

-- 
gr{oe|ee}t{en|ings}
artm