Re: Design ideas for the editing system.

----- Original Message -----
From: Steve Lhomme <[email protected]>
Date: Wed, 28 Jan 2004 21:14:26 +0100
To: "The attempt to create a new,  open standard codec API" <[email protected]>
Subject: Re: [Media-api] Design ideas for the editing system.

> Toby Hudon wrote:
> 
> >>I agree if the architecture can be extended later. Any hacker will be 
> >>able to add it's own format then. As long as we can handle the most 
> >>tricky cases.
> >>
> > 
> > 
> > I agree, extensibility should be a major goal here. What's considered the tricky cases at this point?
> 
> WARP ? ;)
> Also control tracks (which we partly know what they'll do), menus, frame 
> references.
> 

Reference frames by source timecode? Menus I could see being a little tricky. The complexity with WARP should be hidden by whatever kind of decoder plugin is contsructed. I'd suspect for editing we won't need buffering for realtime playback as much. Also, some codecs do not like to play backwards, so we might consider implementing a way to buffer a limited number of frames in the editor so we can do something like play from the last keyframe foward, store and reverse internally, then the previous keyframe forward, etc. This would make things run a little faster in some cases, so I could see it being something some codecs will want or need.

> > I think our best best is the photoshop-like approach. Force everything into a single multitrack MKV for editing when we load it as a source, then perform conversion back to other formats like AVI at final render time if the user specifies that in options. Since editing assumes we're going to be making changes, there's no point in preserving an existing container structure since we'll need to completely tear it down to edit and rebuild it for final output anyway. For transcoding I suggest we just completely rip the existing streams out of the source file with appropriate container demuxers and re-mux them into the project MKV as direct stream copy, so there's no decode/encode. That should be possible, right?
> 
> Well, if the script is just an XML file it can be attached to an MKV 
> file. The problem of storing everything in Matroska is that each stream 
> has to be in sync. Or that will not be a file that is meant to be 
> played, just stored. As so there's no need to use Matroska in this case. 
>   A .tar should be enough (or .KAR/.MKR an EBML archive as planned a 
> long time ago).
> 

Well I'm not sure exactly what the difference is but I'll take your word for it. Obviously we'll have to sync the control stream to the file, or vice versa actually. We know the editing stream is in sync because it defines sync. Whatever it says for a specific timecode needs to be inserted at that timecode. Even if it's just a storage file I can think we'll be wanting to play it for previewing at some point.

> > How hard is it to just link up say an avi demuxer and plug the outputs into an mkv muxer? Does that actually work as a direct stream copy?
> 
> Sure, that's what mkvmerge does well. Direct Stream Copy just meant that 
> coded data (in the sense of codec) are used untouched.
> 

Ok so it shouldn't be that hard then.

> BTW, that makes me think that we should be able to edit a video based on 
> the audio timecodes, not video. So if the audio to start/stop the movie 
> is in the middle of a video frame, we just cut the timecode of the video 
> frame.
> 

Umm... but if the audio and video are both synced what does it matter? Their control streams should have the same timecodes, just the source streams will be different. Or are you talking about when a cut falls in the middle of a video or audio frame's duration? I say we enclude the whole frame of course but if control says stop then the stream stops. Otherwise pasting another stream on the end of this in the editor won't come out quite right. If the next frame is only displayed for 0.01ms then so be it, that's where the timecode said stop. If we want to do a "cut at nearest frame" that's a function that could be specified by an editor tool, and it could be audio or video, and nearest keyframe would likely be desired as well.

> The problem of a pull model is that you know when a frame (audio/video) 
> starts, but not when it ends. Or maybe it should be included in the 
> pulling request (ending timecode)...
> 

Yeah I'd think we might need to include that for VFR. For CFR we could assume it from the specified framerate of course.
-- 
___________________________________________________________
Sign-up for Ads Free at Mail.com
http://promo.mail.com/adsfreejump.htm