From: lilo_booter Date: Wed, 10 Mar 2004 12:08:13 +0000 (+0000) Subject: Minor fixes to westley and mlt_consumer; first draft westley docs X-Git-Url: https://git.sesse.net/?a=commitdiff_plain;h=6a9365cbc88b77e48d6f249808e39d32388c7094;p=mlt Minor fixes to westley and mlt_consumer; first draft westley docs git-svn-id: https://mlt.svn.sourceforge.net/svnroot/mlt/trunk/mlt@198 d19143bc-622f-0410-bfdd-b5b2a6649095 --- diff --git a/docs/inigo.txt b/docs/inigo.txt index d62031f4..ed417d99 100644 --- a/docs/inigo.txt +++ b/docs/inigo.txt @@ -90,12 +90,11 @@ Properties: $ inigo file in=50 out=100 something="something else" - Note that while some properties have meaning to all producers - (for example: in, out and length are guaranteed to be valid for - all, though typically, length is determined automatically), the - validity of others are dependent on the producer - however, - properties will always be assigned, but it doesn't mean they - will be used. + Note that while some properties have meaning to all producers (for + example: in, out and length are guaranteed to be valid for all, though + typically, length is determined automatically), the validity of others + are dependent on the producer - however, properties will always be + assigned and silently ignored if they won't be used. Multiple Files: @@ -108,36 +107,29 @@ Multiple Files: $ inigo a.dv in=50 out=100 b.mpg out=500 c.png out=500 + MLT will take care of 'normalising' the output of a producer to ensure + that the consumer gets what it needs. So, in the case above, the mlt + framework will ensure that images are rescaled and audio resampled to meet + the requirements of your configuration (which, by default, will be PAL). + See 'Appendix A: Normalisation Rules' below. -Filters: - The Multiple Files examples above will logically playout one - after the other. - - However, inigo doesn't care too much about changes in frame - dimensions or audio specification, so you may need to add - additional normalising filters to that, ie: - - $ inigo a.dv b.mpg c.png -filter resize -filter resample - - These filters are designed to guarantee that the consumer gets - what it asks for. - - It should also be stressed that filters are applied in the order - in which they're specified. +Filters: + Filters are frame modifiers - they can change the contents of the audio or + the images associated to a frame. -Filter Properties: + $ inigo a.dv -filter greyscale As with producers, properties may be specified on filters too. - Again, in and out properties are common to all, so to apply a - filter to a range of frames, you would use something like: + Again, in and out properties are common to all, so to apply a filter to a + range of frames, you would use something like: $ inigo a.dv -filter greyscale in=0 out=50 - Again, filters have their own set of rules about properties and - will silently ignore properties that do not apply. + Again, filters have their own set of rules about properties and will + silently ignore properties that do not apply. Groups: @@ -165,8 +157,8 @@ Groups: Introducing Tracks and Blanks: - So far, all of the examples have shown the definition of a - single playlist, or more accurately, track. + So far, all of the examples have shown the definition of a single + playlist, or more accurately, track. When multiple tracks exist, the consumer will receive a frame from the 'lowest numbered' track that is generating a non-blank @@ -202,8 +194,8 @@ Introducing Tracks and Blanks: +-------------------+ Now playout will continue as though a and b clips are on the - same track (which is about as useful as reversing the process of - slicing bread). + same track (which on its own, is about as useful as reversing the + process of slicing bread). Transitions: @@ -213,9 +205,10 @@ Transitions: Here we need tracks to overlap, so a useful multitrack definition could be given as: - $ inigo a.dv out=49 -transition luma in=25 out=49 \ + $ inigo a.dv out=49 \ -track \ - -blank 24 b.dv + -blank 24 b.dv \ + -transition luma in=25 out=49 a_track=0 b_track=1 Now we're cooking - our visualisation would be something like: @@ -246,23 +239,10 @@ Reversing a Transition: In this scenario, we define a command line as follows: $ inigo a.dv out=49 -blank 49 a2.dv \ - -transition luma in=25 out=49 \ - -transition luma in=100 out=124 reverse=1 \ -track \ - -blank 24 b.dv out=99 - - -Filters and Tracks: - - A filter applies to a [specified region of a] single track, so - normalisation filters need to be applied to each track when - applicable. - - This user specification is a necessary evil (you do not want to - resize a text or png overlay to be the size of the frame that - the consumer is requesting, and you may not want to unecessarily - resize a video track if you will be later rescaling it for - composition). + -blank 24 b.dv out=99 \ + -transition luma in=25 out=49 a_track=0 b_track=1 \ + -transition luma in=100 out=124 reverse=1 a_track=0 b_track=1 Serialisation: @@ -275,14 +255,22 @@ Serialisation: miracle or inigo. Take care though - paths to files are saved as provided on the command line.... + A more expressive serialisation can be obtained with the westley consumer + - this will provide an xml document which can be used freely in inigo and + miracle. + + See westley.txt for more information. + Missing Features: - Some filters/transitions should be applied on the output frame - regardless of which track it comes from - for example, you might - have a 3rd text track or a watermark which you want composited - on every frame, and of course, there's the obscure feature.... + Some filters/transitions should be applied on the output frame regardless + of which track it comes from - for example, you might have a 3rd text + track or a watermark which you want composited on every frame, and of + course, there's the obscure filter.... - A -post switch will be added to provided this feature at some - point soon. + inigo only supports this in two invocations - as a simple example: + + $ inigo a.dv -track -blank 100 b.dv -consumer westley:basic.westley + $ inigo basic.westley -filter watermark:watermark.png diff --git a/docs/westley.txt b/docs/westley.txt new file mode 100644 index 00000000..5e37b0a8 --- /dev/null +++ b/docs/westley.txt @@ -0,0 +1,280 @@ +WESTLEY +------- + +Preamble: + + Westley is the MLT projects XML serialisation/deserialisation format - + as such, it closely mirrors the internal structure of the MLT API. + + +Introduction: + + A westley document is essentially a list of 'producers' - a producer is + an mlt object which generates mlt frames (images and associated audio + samples). + + There are 3 types of producer: + + * Basic Producers - these are typically file or device oriented feeds; + * Playlists - these are arrangements of multiple producers; + * Tractors - these are multitrack fx encapsulators. + + Although westley was defined as a serialisation mechanism for running MLT + components, this document will concentrate on the hand authoring of westley + documents. + + +Rules: + + As shall become apparent through the remainder of this document, the basic + tenet of westley authoring is to organise the document in the following + manner: + + 1) create producer elements for each unique media clip in the project; + 2) create playlists for each track; + 3) create a tractor for track specific filters and transitions; + 4) create another tractor for filters that are common to the output. + + While other uses of westley exist, the approach taken here is to maximise + efficiency for complex projects. + + +Basic Producers: + + The simplest westley document is: + + + + clip1.dv + + + + The westley wrapping is of course superfluous here - loading this document + with MLT is identical to loading the clip directly. + + Of course, you can specify additional properties. For example, consider an + MPEG file with multiple soundtracks - you could define a westley document to + ensure that the second audio track is loaded: + + + + clip1.mpeg + 1 + + + + NB: This relies on the mpeg being handled by the avformat producer, rather + than the mcmpeg one. See services.txt for more details. + + A more useful example comes with the pango producer for a text producer. + + TODO: pango example... + + Notes: + + 1) It is better not to specify in/out points when defining basic producers + as these can be specified in the playlists. The reasoning is that in/out + restricts the amount of the clip available, and could lead to the same clip + being loaded multiple times if you need different regions of the clip + elsewhere; + 2) A westley can be specified as a resource, so westleys can naturally + encapsulate other westleys. + + +Playlists: + + Playlists provide a 'collection' structure for producers. These can be used + to define 'tracks' in the multitrack approach, or simple playlists for + sequential, single track playout. + + As an example, the following defines two basic producers and a playlist with 3 + items: + + + + clip1.dv + + + clip2.dv + + + + + + + + + Here we see how the playlist defines the in/out points of the basic + producers. + + Notes: + + 1) All in/out points are absolute frame positions - we support PAL and + NTSC at a system level, but westley documents are currently authored + for a specific normalisation; + 2) The last 'producer' in the document is the default for play out; + 3) Playlists can reference the same producer multiple times. In/out regions + do not need to be contiguous - duplication and skipping is acceptable. + + +Interlude - Introducing Tractors: + + So far, we've defined basic producers and playlists/tracks - the tractor is + the element that allows us to arrange our tracks and specify filters and + transitions. Similarly to a playlist, a tractor is a container. + + Note that MLT doesn't see a filter or a transition as a producer in the + normal sense - filters and transitions are passive when it comes to seeking. + Internally, seeks are carried out on the producers. This is an important + point - MLT does not follow a traditional graph oriented model. + + Visualising an MLT tractor and it's interaction with the consumer will + assist here: + + +----------------------------------------------+ + |tractor | + | +----------+ +-+ +-+ +-+ +-+ | + | |multitrack| |f| |f| |t| |t| | + | | +------+ | |i| |i| |r| |r| | + | | |track0|-|--->|l|- ->|l|- ->|a|--->|a|\ | + | | +------+ | |t| |t| |n| |n| \ | + | | | |e| |e| |s| |s| \ | + | | +------+ | |r| |r| |i| |i| \ | +--------+ + | | |track1|-|- ->|0|--->|1|--->|t|--->|t|-----|--->|consumer| + | | +------+ | | | | | |i| |i| / | +--------+ + | | | | | | | |o| |o| / | ^ + | | +------+ | | | | | |n| |n| / | | + | | |track2|-|- ->| |- ->| |--->|0|- ->|1|/ | | + | | +------+ | | | | | | | | | | | + | +----------+ +-+ +-+ +-+ +-+ | | + +----------------------------------------------+ | + ^ | + | | + +-----------+ | + |APPLICATION|--------------------------------------------+ + +-----------+ + + Internally, all frames from all tracks pass through all the filters and + transitions - these are told which tracks to deal and which regions of the + tracks to work on. + + Note that the application communicates with the producer - it can alter + playback speed, position, or even which producer is connected to which + consumer. + + In a later phase of MLT development, the application will be able to + manipulate the tractors make up, by adding and removing tracks, filters + and transitions. The consumer itself remains connected to the same object + (the tractor). + + The consumer receives the first non-blank frame (see below). It has no say + in the order in which gets them (the sdl consumer when used with inigo might + appear to be an exception - it isn't - it simply has a route back to the + application to allow the application to interpret key presses). + + Whether this is better or worse than a traditional graph approach is a moot + point. The author of this document likes it anyway :-). + + +Tractors: + + To create a multitrack westley, we can use two playlists and introduce a + tractor. For the purposes of demonstration, I'll add a filter here too: + + + + clip1.dv + + + clip2.dv + + + + + + + + + + + + + + + + 0 + greyscale + + + + + Here we see that blank frames are inserted into the first playlist and a + blank is provided at the beginning of the second - this can be visualised in + the traditional timeline widget as follows: + + +-------+ +-------------+ + |a | |a | + +-------+---+-------------+ + |b | + +---+ + + Adding the filter on the top track, gives us: + + +-------+ +-------------+ + |a | |a | + +-------+---+-------------+ + |greyscale | + --------+---+-------------+ + |b | + +---+ + + Note that it's only applied to the visible parts of the top track. + + The requirement to apply a filter to the output, as opposed to a specific track + leads us to the final item in the Rules section above. As an example, let's + assume we wish to watermark all output, then we could use the following: + + + + clip1.dv + + + clip2.dv + + + + + + + + + + + + + + + + 0 + greyscale + + + + + + + + watermark + watermark1.png + + + + + Here we employ another tractor and we define a single track (being the + tractor we previously defined) and apply a watermarking filter there. + + This is simply provided as an example - the watermarking functionality could + be better handled at the playout stage itself (ie: as a filter automatically + placed between all producers and the consumer). + + TODO: transition example diff --git a/src/framework/mlt_consumer.c b/src/framework/mlt_consumer.c index ac84f90e..5ca552f5 100644 --- a/src/framework/mlt_consumer.c +++ b/src/framework/mlt_consumer.c @@ -222,6 +222,7 @@ static void *consumer_read_ahead_thread( void *arg ) // Average time for get_frame and get_image int count = 1; + int skipped = 0; int64_t time_wait = 0; int64_t time_frame = 0; int64_t time_image = 0; @@ -262,6 +263,22 @@ static void *consumer_read_ahead_thread( void *arg ) mlt_frame_get_image( frame, &image, &this->format, &width, &height, 0 ); mlt_properties_set_int( mlt_frame_properties( frame ), "rendered", 1 ); time_image += time_difference( &ante ); + + // Reset the skipped count + skipped = 0; + } + else + { + // Increment the number of sequentially skipped frames + skipped ++; + + // If we've reached an unacceptable level, reset everything + if ( skipped > 10 ) + { + skipped = 0; + time_frame = 0; + time_image = 0; + } } } diff --git a/src/modules/westley/producer_westley.c b/src/modules/westley/producer_westley.c index 49e808b4..094558be 100644 --- a/src/modules/westley/producer_westley.c +++ b/src/modules/westley/producer_westley.c @@ -93,6 +93,9 @@ static void on_start_tractor( deserialise_context context, const xmlChar *name, mlt_properties_set_position( properties, "length", length ); } + if ( mlt_properties_get( properties, "id" ) != NULL ) + mlt_properties_set_data( context->producer_map, mlt_properties_get( properties, "id" ), service, 0, NULL, NULL ); + context_push_service( context, service ); } @@ -452,7 +455,7 @@ static void on_end_filter( deserialise_context context, const xmlChar *name ) properties = mlt_service_properties( service ); // Set in and out -//fprintf( stderr, "setting filter in %lld out %lld\n", mlt_properties_get_position( properties, "in" ), mlt_properties_get_position( properties, "out" ) ); +//fprintf( stderr, "setting filter in %d out %d\n", mlt_properties_get_position( properties, "in" ), mlt_properties_get_position( properties, "out" ) ); mlt_filter_set_in_and_out( MLT_FILTER( service ), mlt_properties_get_position( properties, "in" ), mlt_properties_get_position( properties, "out" ) ); @@ -471,7 +474,7 @@ static void on_end_filter( deserialise_context context, const xmlChar *name ) context_push_service( context, tractor ); } -//fprintf( stderr, "setting filter in %lld out %lld\n", mlt_properties_get_position( properties, "in" ), mlt_properties_get_position( properties, "out" ) ); +//fprintf( stderr, "setting filter in %d out %d\n", mlt_properties_get_position( properties, "in" ), mlt_properties_get_position( properties, "out" ) ); // If a producer alias is in the producer_map, get it snprintf( key, 10, "%p", producer ); if ( mlt_properties_get_data( context->producer_map, key, NULL ) != NULL )