X-Git-Url: https://git.sesse.net/?a=blobdiff_plain;f=docs%2Fframework.txt;h=56610fd5b23d46d58daf1bad1de2a390eb761d63;hb=HEAD;hp=a1276db855b69ec6ac3f340965b09efc936cfba6;hpb=a83b8624f7a504b52e6432e049df69f41cc9ce75;p=mlt diff --git a/docs/framework.txt b/docs/framework.txt index a1276db8..56610fd5 100644 --- a/docs/framework.txt +++ b/docs/framework.txt @@ -1,8 +1,8 @@ Framework Documentation -Copyright (C) 2004 Ushodaya Enterprises Limited +Copyright (C) 2004-2009 Ushodaya Enterprises Limited Author: Charles Yates -Last Revision: 2004-03-20 +Last Revision: 2005-05-08 MLT FRAMEWORK @@ -23,8 +23,8 @@ Preamble: This document is split roughly into 3 sections. The first section provides a basic overview of MLT, the second section shows how it's used and the final - section shows shows structure and design, with an emphasis on how the system - is extended. + section shows structure and design, with an emphasis on how the system is + extended. Target Audience: @@ -44,7 +44,7 @@ Target Audience: opposed to the implementation details. It is not required reading for the MLT client/server integration - please - refer to valerie.txt and dvcp.txt for more details on this area. + refer to libmvsp.txt and mvsp.txt for more details on this area. SECTION 1 - BASIC OVERVIEW @@ -187,12 +187,12 @@ Hello World: The defaults requested here are a special case - the NULL usage requests that we use the default producers and consumers. - The default producer is "fezzik". This producer matches file names to + The default producer is "loader". This producer matches file names to locate a service to use and attaches 'normalising filters' (such as scalers, deinterlacers, resamplers and field normalisers) to the loaded content - these filters ensure that the consumer gets what it asks for. - The default consumer is "sdl". The combination of fezzik and sdl will + The default consumer is "sdl". The combination of loader and sdl will provide a media player. In this example, we connect the producer and then start the consumer. We @@ -205,11 +205,11 @@ Hello World: Also note, you can override the defaults as follows: - $ MLT_CONSUMER=westley ./hello file.avi + $ MLT_CONSUMER=xml ./hello file.avi - This will create a westley xml document on stdout. + This will create a XML document on stdout. - $ MLT_CONSUMER=westley MLT_PRODUCER=avformat ./hello file.avi + $ MLT_CONSUMER=xml MLT_PRODUCER=avformat ./hello file.avi This will play the video using the avformat producer directly, thus it will bypass the normalising functions. @@ -256,10 +256,12 @@ Factories: +------------------+------------------------------------+------------------+ |MLT_NORMALISATION |The normalisation of the system |PAL or NTSC | +------------------+------------------------------------+------------------+ - |MLT_PRODUCER |The default producer |"fezzik" or other | + |MLT_PRODUCER |The default producer |"loader" or other | +------------------+------------------------------------+------------------+ |MLT_CONSUMER |The default consumer |"sdl" or other | +------------------+------------------------------------+------------------+ + |MLT_TEST_CARD |The default test card producer |any producer | + +------------------+------------------------------------+------------------+ These values are initialised from the environment variables of the same name. @@ -301,11 +303,7 @@ Playlists: Instead of invoking mlt_factory_producer directly, we'll create a new function called create_playlist. This function is responsible for creating - the playlist, creating each producer, appending to the playlist and ensuring - that all the producers are cleaned up when the playlist is destroyed. The - last point is important - a close on the playlist won't explicitly close these - producers. In this example, we use unique "data" properties with destructors - to ensure closing. + the playlist, creating each producer and appending to the playlist. mlt_producer create_playlist( int argc, char **argv ) { @@ -319,26 +317,28 @@ Playlists: int i = 0; for ( i = 1; i < argc; i ++ ) { - // Define the unique key - char key[ 256 ]; - // Create the producer mlt_producer producer = mlt_factory_producer( NULL, argv[ i ] ); // Add it to the playlist mlt_playlist_append( playlist, producer ); - // Create a unique key for this producer - sprintf( key, "producer%d", i ); - - // Now we need to ensure the producers are destroyed - mlt_properties_set_data( properties, key, producer, 0, ( mlt_destructor )mlt_producer_close, NULL ); + // Close the producer (see below) + mlt_producer_close( producer ); } // Return the playlist as a producer return mlt_playlist_producer( playlist ); } + Notice that we close the producer after the append. Actually, what we're + doing is closing our reference to it - the playlist creates its own reference + to the producer on append and insert, and it will close its reference + when the playlist is destroyed[*]. + + Note also that if you append multiple instances of the same producer, it + will create multiple references to it. + Now all we need do is to replace these lines in the main function: // Create a normalised producer @@ -351,6 +351,10 @@ Playlists: and we have a means to play multiple clips. + [*] This reference functionality was introduced in mlt 0.1.2 - it is 100% + compatable with the early mechanism of registering the reference and + destructor with the properties of the playlist object. + Filters: @@ -385,6 +389,193 @@ Filters: section, even multiple tracks have a single track output. +Attached Filters: + + All services can have attached filters. + + Consider the following example: + + // Create a producer + mlt_producer producer = mlt_factory_producer( NULL, clip ); + + // Get the service object of the producer + mlt_producer service = mlt_producer_service( producer ); + + // Create a filter + mlt_filter filter = mlt_factory_filter( "greyscale" ); + + // Create a playlist + mlt_playlist playlist = mlt_playlist_init( ); + + // Attach the filter to the producer + mlt_service_attach( producer, filter ); + + // Construct a playlist with various cuts from the producer + mlt_playlist_append_io( producer, 0, 99 ); + mlt_playlist_append_io( producer, 450, 499 ); + mlt_playlist_append_io( producer, 200, 399 ); + + // We can close the producer and filter now + mlt_producer_close( producer ); + mlt_filter_close( filter ); + + When this is played out, the greyscale filter will be executed for each frame + in the playlist which comes from that producer. + + Further, each cut can have their own filters attached which are executed after + the producer's filters. As an example: + + // Create a new filter + filter = mlt_factory_filter( "invert", NULL ); + + // Get the second 'clip' in the playlist + producer = mlt_playlist_get_clip( 1 ); + + // Get the service object of the clip + service = mlt_producer_service( producer ); + + // Attach the filter + mlt_service_attach( producer, filter ); + + // Close the filter + mlt_filter_close( filter ); + + Even the playlist itself can have an attached filter: + + // Create a new filter + filter = mlt_factory_filter( "watermark", "+Hello.txt" ); + + // Get the service object of the playlist + service = mlt_playlist_service( playlist ); + + // Attach the filter + mlt_service_attach( service, filter ); + + // Close the filter + mlt_filter_close( filter ); + + And, of course, the playlist, being a producer, can be cut up and placed on + another playlist, and filters can be attached to those cuts or on the new + playlist itself and so on ad nauseum. + + The main advantage of attached filters is that they remain attached and don't + suffer from the maintenance problems associated with items being inserted and + displacing calculated in/out points - this being a major issue if you + exclusively use the connect or insert detached filters in a multitrack field + (described below). + + +Introducing the Mix: + + The mix is the simplest way to introduce transitions between adjacent clips + on a playlist. + + Consider the following playlist: + + +-+----------------------+----------------------------+-+ + |X|A |B |X| + +-+----------------------+----------------------------+-+ + + Let's assume that the 'X' is a 'black clip' of 50 frames long. + + When you play this out, you'll get a 50 frames of black, abrupt cut into + A, followed by an abrupt cut into B, and finally into black again. + + The intention is to convert this playlist into something like: + + +-+---------------------+-+------------------------+-+ + |X|A |A|B |B| + |A| |B| |X| + +-+---------------------+-+------------------------+-+ + + Where the clips which refer to 2 clips represent a transition. Notice that + the representation of the second playlist is shorter than the first - this is + to be expected - a single transition of 50 frames between two clips will + reduce the playtime of the result by 50 frames. + + This is done via the use of the mlt_playlist_mix method. So, assuming you get + a playlist as shown in the original diagram, to do the first mix, you could do + something like: + + // Create a transition + mlt_transition transition = mlt_factor_transition( "luma", NULL ); + + // Mix the first and second clips for 50 + mlt_playlist_mix( playlist, 0, 50, transition ); + + // Close the transition + mlt_transition_close( transition ); + + This would give you the first transition, subsequently, you would apply a similar + technique to mix clips 1 and 2. Note that this would create a new clip on the + playlist, so the next mix would be between 3 and 4. + + As a general hint, to simplify the requirement to know the next clip index, + you might find the following simpler: + + // Get the number of clips on the playlist + int i = mlt_playlist_count( ); + + // Iterate through them in reverse order + while ( i -- ) + { + // Create a transition + mlt_transition transition = mlt_factor_transition( "luma", NULL ); + + // Mix the first and second clips for 50 + mlt_playlist_mix( playlist, i, 50, transition ); + + // Close the transition + mlt_transition_close( transition ); + } + + There are other techniques, like using the mlt_playlist_join between the + current clip and the newly created one (you can determine if a new clip was + created by comparing the playlist length before and after the mix call). + + Internally, the mlt_playlist_mix call generates a tractor and multitrack as + described below. Like the attached filters, the mix makes life very simple + when you're inserting items into the playlist. + + Also note that it allows a simpler user interface - instead of enforcing the + use of a complex multitrack object, you can do many operations on a single + track. Thus, additional tracks can be used to introduce audio dubs, mixes + or composites which are independently positioned and aren't affected by + manipulations on other tracks. But hey, if you want a bombastic, confusing + and ultimately frustrating traditional NLE experience, that functionality + is provided too ;-). + + +Practicalities and Optimisations: + + In the previous two sections I've introduced some powerful functionality + designed to simplify MLT usage. However, a general issue comes into this - + what happens when you introduce a transition between two cuts from the same + bit of video footage? + + Anyone who is familiar with video compression will be aware that seeking + isn't always without consequence from a performance point of view. So if + you happen to require two frames from the same clip for a transition, the + processing is going to be excessive and the result will undoubtedly be very + unpleasant, especially if you're rendering in realtime... + + So how do we get round this? + + Actually, it's very simple - you invoke mlt_producer_optimise on the top + level object after a modification and MLT will determine how to handle it. + Internally, it determines the maximum number of overlapping instances + throughout the object and creates clones and assigns clone indexes as + required. + + In the mix example above, you can simply call: + + // Optimise the playlist + mlt_producer_optimise( mlt_playlist_producer( playlist ) ); + + after the mix calls have be done. Note that this is automatically applied + to deserialised MLT XML. + + Multiple Tracks and Transitions: MLT's approach to multiple tracks is governed by two requirements: @@ -403,8 +594,9 @@ Multiple Tracks and Transitions: +------------------------------+ The overlapping areas of track 0 and 1 would (presumably) have some kind of - transition - without a transition, the frames from a1 and a2 would be shown - during the areas of overlap. + transition - without a transition, the frames from b1 and b2 would be shown + during the areas of overlap (ie: by default, the higher numbered track takes + precedence over the lower numbered track). MLT has a multitrack object, but it is not a producer in the sense that it can be connected directly to a consumer and everything will work correctly. @@ -441,7 +633,7 @@ Multiple Tracks and Transitions: +----------+ With a combination of the two, we can now connect multitracks to consumers. - The first non-test card will be retrieved and passed on. + The last non-test card will be retrieved and passed on. The tracks can be producers, playlists, or even other tractors. @@ -450,7 +642,7 @@ Multiple Tracks and Transitions: tractor and the multitrack, but this involves a lot of connecting and reconnecting left and right producers and consumers, and it seemed only fair that we should be able to automate that process. - + So in keeping with our agricultural theme, the concept of the 'field' was born. We 'plant' filters and transitions in the field and the tractor pulls the multitrack (think of a combine harvester :-)) over the field and @@ -473,13 +665,11 @@ Multiple Tracks and Transitions: | +------+ | +-------------+ +-------+ +----------+ - In reality, we create a field first, and from that we obtain a multitrack - and a tractor. We can then populate the multitrack, field and finally, - connect the tractor to the consumer. - - The reasoning behind this is possibly flawed - it might have made more - sense to produce the tractor and have it encapsulate the field and the - multitrack as that is how it looks to a connected consumer: + So, we need to create the tractor first, and from that we obtain the + multitrack and field objects. We can populate these and finally + connect the tractor to a consumer. + + In essence, this is how it looks to the consumer: +-----------------------------------------------+ |tractor +--------------------------+ | @@ -509,23 +699,23 @@ Multiple Tracks and Transitions: mlt_producer create_tracks( int argc, char **argv ) { - // Create the field - mlt_field field = mlt_field_init( ); + // Create the tractor + mlt_tractor tractor = mlt_tractor_new( ); + + // Obtain the field + mlt_field field = mlt_tractor_field( tractor ); // Obtain the multitrack - mlt_multitrack multitrack = mlt_field_multitrack( field ); - - // Obtain the tractor - mlt_tractor tractor = mlt_field_tractor( field ); + mlt_multitrack multitrack = mlt_tractor_multitrack( tractor ); - // Obtain a composite transition - mlt_transition transition = mlt_factory_transition( "composite", "10%,10%:15%x15%" ); + // Create a composite transition + mlt_transition transition = mlt_factory_transition( "composite", "10%/10%:15%x15%" ); // Create track 0 mlt_producer track0 = create_playlist( argc, argv ); - // Create the watermark track - note we NEED fezzik for scaling here - mlt_producer track1 = mlt_factory_producer( "fezzik", "pango" ); + // Create the watermark track - note we NEED loader for scaling here + mlt_producer track1 = mlt_factory_producer( "loader", "pango" ); // Get the length of track0 mlt_position length = mlt_producer_get_playtime( track0 ); @@ -536,7 +726,9 @@ Multiple Tracks and Transitions: mlt_properties_set_position( properties, "in", 0 ); mlt_properties_set_position( properties, "out", length - 1 ); mlt_properties_set_position( properties, "length", length ); - + mlt_properties_set_int( properties, "a_track", 0 ); + mlt_properties_set_int( properties, "b_track", 1 ); + // Now set the properties on the transition properties = mlt_transition_properties( transition ); mlt_properties_set_position( properties, "in", 0 ); @@ -548,15 +740,12 @@ Multiple Tracks and Transitions: // Now plant the transition mlt_field_plant_transition( field, transition, 0, 1 ); - - // Now set the properties on the tractor - properties = mlt_tractor_properties( tractor ); - mlt_properties_set_data( properties, "multitrack", multitrack, 0, ( mlt_destructor )mlt_multitrack_close, NULL ); - mlt_properties_set_data( properties, "field", field, 0, ( mlt_destructor )mlt_field_close, NULL ); - mlt_properties_set_data( properties, "track0", track0, 0, ( mlt_destructor )mlt_producer_close, NULL ); - mlt_properties_set_data( properties, "track1", track1, 0, ( mlt_destructor )mlt_producer_close, NULL ); - mlt_properties_set_data( properties, "transition", transition, 0, ( mlt_destructor )mlt_transition_close, NULL ); - + + // Close our references + mlt_producer_close( track0 ); + mlt_producer_close( track1 ); + mlt_transition_close( transition ); + // Return the tractor return mlt_tractor_producer( tractor ); } @@ -574,6 +763,9 @@ Multiple Tracks and Transitions: and we have a means to play multiple clips with a horribly obtrusive watermark - just what the world needed, right? ;-) + Incidentally, the same thing could be achieved with the more trivial + watermark filter inserted between the producer and the consumer. + SECTION 3 - STRUCTURE AND DESIGN -------------------------------- @@ -987,7 +1179,7 @@ mlt_service: A service does not define any properties when constructed. It should be noted that producers, filters and transitions my be serialised (say, via the - westley consumer), and care should be taken to distinguish between + xml consumer), and care should be taken to distinguish between serialisable and transient properties. The convention used is to prefix transient properties with an underscore.