--- /dev/null
+MLT FRAMEWORK
+-------------
+
+Preamble:
+
+ MLT is a multimedia framework designed for television broadcasting. As such,
+ it provides a pluggable architecture for the inclusion of new audio/video
+ sources, filters, transitions and playback devices.
+
+ The MLT framework provides the structure and utility functionality on which
+ all of the MLT applications and services are defined.
+
+ On its own, the framework provides little more than 'abstract classes' and
+ utilities for managing resources, such as memory, properties, dynamic object
+ loading and service instantiation.
+
+ This document is split roughly into 3 sections. The first section provides a
+ basic overview of MLT, the second section shows how it's used and the final
+ section shows shows structure and design, with an emphasis on how the system
+ is extended.
+
+
+Target Audience:
+
+ This document is provided as a 'road map' for the framework and should be
+ considered mandatory reading for anyone wishing to develop code at the MLT
+ level.
+
+ This includes:
+
+ 1. framework maintainers;
+ 2. module developers;
+ 3. application developers;
+ 4. anyone interested in MLT.
+
+ The emphasis of the document is in explaining the public interfaces, as
+ opposed to the implementation details.
+
+ It is not required reading for the MLT client/server integration - please
+ refer to valerie.txt and dvcp.txt for more details on this area.
+
+
+SECTION 1 - BASIC OVERVIEW
+--------------------------
+
+Basic Design Information:
+
+ MLT is written in C.
+
+ The framework has no dependencies other than the standard C99 and POSIX
+ libraries.
+
+ It follows a basic Object Oriented design paradigm, and as such, much of the
+ design is loosely based on the Producer/Consumer design pattern.
+
+ It employs Reverse Polish Notation for the application of audio and video FX.
+
+ The framework is designed to be colour space neutral - the currently
+ implemented modules, however, are very much 8bit YUV422 oriented. In theory,
+ the modules could be entirely replaced.
+
+ A vague understanding of these terms is assumed throughout the remainder of
+ this document.
+
+
+Structure and Flow:
+
+ The general structure of an MLT 'network' is simply the connection of a
+ 'producer' to a 'consumer':
+
+ +--------+ +--------+
+ |Producer|-->|Consumer|
+ +--------+ +--------+
+
+ A typical consumer requests MLT Frame objects from the producer, does
+ something with them and when finished with a frame, closes it.
+
+ /\ A common confusion with the producer/consumer terminoligy used here is
+ /!!\ that a consumer may 'produce' something. For example, the libdv consumer
+ \!!/ produces DV and the libdv producer seems to consume DV. However, the
+ \/ naming conventions refer only to producers and consumers of MLT Frames.
+
+ To put it another way - a producer produces MLT Frame objects and a consumer
+ consumes MLT Frame objects.
+
+ An MLT Frame essentially provides an uncompressed image and its associated
+ audio samples.
+
+ Filters may also be placed between the producer and the consumer:
+
+ +--------+ +------+ +--------+
+ |Producer|-->|Filter|-->|Consumer|
+ +--------+ +------+ +--------+
+
+ A service is the collective name for producers, filters, transitions and
+ consumers.
+
+ The communications between a connected consumer and producer or service are
+ carried out in 3 phases:
+
+ * get the frame
+ * get the image
+ * get the audio
+
+ MLT employs 'lazy evaluation' - the image and audio need not be extracted
+ from the source until the get image and audio methods are invoked.
+
+ In essence, the consumer pulls from what it's connected to - this means that
+ threading is typically in the domain of the consumer implementation and some
+ basic functionality is provided on the consumer class to ensure realtime
+ throughput.
+
+
+SECTION 2 - USAGE
+-----------------
+
+Hello World:
+
+ Before we go in to the specifics of the framework architecture, a working
+ example of usage is provided.
+
+ The following simply provides a media player:
+
+ #include <stdio.h>
+ #include <unistd.h>
+ #include <framework/mlt.h>
+
+ int main( int argc, char *argv[] )
+ {
+ // Initialise the factory
+ if ( mlt_factory_init( NULL ) == 0 )
+ {
+ // Create the default consumer
+ mlt_consumer hello = mlt_factory_consumer( NULL, NULL );
+
+ // Create via the default producer
+ mlt_producer world = mlt_factory_producer( NULL, argv[ 1 ] );
+
+ // Connect the producer to the consumer
+ mlt_consumer_connect( hello, mlt_producer_service( world ) );
+
+ // Start the consumer
+ mlt_consumer_start( hello );
+
+ // Wait for the consumer to terminate
+ while( !mlt_consumer_is_stopped( hello ) )
+ sleep( 1 );
+
+ // Close the consumer
+ mlt_consumer_close( hello );
+
+ // Close the producer
+ mlt_producer_close( world );
+
+ // Close the factory
+ mlt_factory_close( );
+ }
+ else
+ {
+ // Report an error during initialisation
+ fprintf( stderr, "Unable to locate factory modules\n" );
+ }
+
+ // End of program
+ return 0;
+ }
+
+ This is a simple example - it doesn't provide any seeking capabilities or
+ runtime configuration options.
+
+ The first step of any MLT application is the factory initialisation - this
+ ensures that the environment is configured and MLT can function. The factory
+ is covered in more detail below.
+
+ All services are instantiated via the factories, as shown by the
+ mlt_factory_consumer and mlt_factory_producer calls above. There are similar
+ factories for filters and transitions. There are details on all the standard
+ services in services.txt.
+
+ The defaults requested here are a special case - the NULL usage requests
+ that we use the default producers and consumers.
+
+ The default producer is "fezzik". This producer matches file names to
+ locate a service to use and attaches 'normalising filters' (such as scalers,
+ deinterlacers, resamplers and field normalisers) to the loaded content -
+ these filters ensure that the consumer gets what it asks for.
+
+ The default consumer is "sdl". The combination of fezzik and sdl will
+ provide a media player.
+
+ In this example, we connect the producer and then start the consumer. We
+ then wait until the consumer is stopped (in this case, by the action of the
+ user closing the SDL window) and finally close the consumer, producer and
+ factory before exiting the application.
+
+ Note that the consumer is threaded - waiting for an event of some sort is
+ always required after starting and before stopping or closing the consumer.
+
+ Also note, you can override the defaults as follows:
+
+ $ MLT_CONSUMER=westley ./hello file.avi
+
+ This will create a westley xml document on stdout.
+
+ $ MLT_CONSUMER=westley MLT_PRODUCER=avformat ./hello file.avi
+
+ This will play the video using the avformat producer directly, thus it will
+ bypass the normalising functions.
+
+ $ MLT_CONSUMER=libdv ./hello file.avi > /dev/dv1394
+
+ This might, if you're lucky, do on the fly, realtime conversions of file.avi
+ to DV and broadcast it to your DV device.
+
+
+Factories:
+
+ As shown in the 'Hello World' example, factories create service objects.
+
+ The framework itself provides no services - they are provided in the form of
+ a plugin structure. A plugin is organised in the form of a 'module' and a
+ module can provide many services of different types.
+
+ Once the factory is initialised, all the configured services are available
+ for use.
+
+ The complete set of methods associated to the factory are as follows:
+
+ int mlt_factory_init( char *prefix );
+ const char *mlt_factory_prefix( );
+ char *mlt_environment( char *name );
+ mlt_producer mlt_factory_producer( char *name, void *input );
+ mlt_filter mlt_factory_filter( char *name, void *input );
+ mlt_transition mlt_factory_transition( char *name, void *input );
+ mlt_consumer mlt_factory_consumer( char *name, void *input );
+ void mlt_factory_close( );
+
+ The mlt_factory_prefix returns the path to the location of the installed
+ modules directory. This can be specified in the mlt_factory_init call
+ itself, or it can be specified via the MLT_REPOSITORY environment variable,
+ or in the absence of either of those, it will default to the install
+ prefix/shared/mlt/modules.
+
+ The mlt_environment provides read only access to a collection of name=value
+ pairs as shown in the following table:
+
+ +------------------+------------------------------------+------------------+
+ |Name |Description |Values |
+ +------------------+------------------------------------+------------------+
+ |MLT_NORMALISATION |The normalisation of the system |PAL or NTSC |
+ +------------------+------------------------------------+------------------+
+ |MLT_PRODUCER |The default producer |"fezzik" or other |
+ +------------------+------------------------------------+------------------+
+ |MLT_CONSUMER |The default consumer |"sdl" or other |
+ +------------------+------------------------------------+------------------+
+
+ These values are initialised from the environment variables of the same
+ name.
+
+ As shown above, a producer can be created using the 'default normalising'
+ producer, and they can also be requested by name. Filters and transitions
+ are always requested by name - there is no concept of a 'default' for these.
+
+
+Service Properties:
+
+ As shown in the services.txt document, all services have their own set of
+ properties than can be manipulated to affect their behaviour.
+
+ In order to set properties on a service, we need to retrieve the properties
+ object associated to it. For producers, this is done by invoking:
+
+ mlt_properties properties = mlt_producer_properties( producer );
+
+ All services have a similar method associated to them.
+
+ Once retrieved, setting and getting properties can be done directly on this
+ object, for example:
+
+ mlt_properties_set( properties, "name", "value" );
+
+ A more complete description of the properties object is found below.
+
+
+Playlists:
+
+ So far, we've shown a simple producer/consumer configuration - the next
+ phase is to organise producers in playlists.
+
+ Let's assume that we're adapting the Hello World example, and wish to queue
+ a number of files for playout, ie:
+
+ hello *.avi
+
+ Instead of invoking mlt_factory_producer directly, we'll create a new
+ function called create_playlist. This function is responsible for creating
+ the playlist, creating each producer, appending to the playlist and ensuring
+ that all the producers are cleaned up when the playlist is destroyed. The
+ last point is important - a close on the playlist won't explicitly these
+ producers. In this example, we use unique "data" properties with destructors
+ to ensure closing.
+
+ mlt_producer create_playlist( int argc, char **argv )
+ {
+ // We're creating a playlist here
+ mlt_playlist playlist = mlt_playlist_init( );
+
+ // We need the playlist properties to ensure clean up
+ mlt_properties properties = mlt_playlist_properties( playlist );
+
+ // Loop through each of the arguments
+ int i = 0;
+ for ( i = 1; i < argc; i ++ )
+ {
+ // Definie the unique key
+ char key[ 256 ];
+
+ // Create the producer
+ mlt_producer producer = mlt_factory_producer( NULL, argv[ i ] );
+
+ // Add it to the playlist
+ mlt_playlist_append( playlist, producer );
+
+ // Create a unique key for this producer
+ sprintf( key, "producer%d", i );
+
+ // Now we need to ensure the producers are destroyed
+ mlt_properties_set_data( properties, key, producer, 0, ( mlt_destructor )mlt_producer_close, NULL );
+ }
+
+ // Return the playlist as a producer
+ return mlt_playlist_producer( playlist );
+ }
+
+ Now all we need do is to replace these lines in the main function:
+
+ // Create a normalised producer
+ mlt_producer world = mlt_factory_producer( NULL, argv[ 1 ] );
+
+ with:
+
+ // Create a playlist
+ mlt_producer world = create_playlist( argc, argv );
+
+ and we have a means to play multiple clips.
+
+
+Filters:
+
+ Inserting filters between the producer and consumer is just a case of
+ instantiating the filters, connecting the first to the producer, the next
+ to the previous filter and the last filter to the consumer.
+
+ For example:
+
+ // Create a producer from something
+ mlt_producer producer = mlt_factory_producer( ... );
+
+ // Create a consumer from something
+ mlt_consumer consumer = mlt_factory_consumer( ... );
+
+ // Create a greyscale filter
+ mlt_filter filter = mlt_factory_filter( "greyscale", NULL );
+
+ // Connect the filter to the producer
+ mlt_filter_connect( filter, mlt_producer_service( producer ), 0 );
+
+ // Connect the consumer to filter
+ mlt_consumer_connect( consumer, mlt_filter_service( filter ) );
+
+ As with producers and consumers, filters can be manipulated via their
+ properties object - the mlt_filter_properties method can be invoked and
+ properties can be set as needed.
+
+ The additional argument in the filter connection is an important one as it
+ dicates the 'track' on which the filter operates. For basic producers and
+ playlists, there's only one track (0), and as you will see in the next
+ section, even multiple tracks have a single track output.
+
+
+Multiple Tracks and Transitions:
+
+ MLT's approach to mutliple tracks is governed by two requirements:
+
+ 1) The need for a consumer and producer to communicate with one another via
+ a single frame;
+ 2) The desire to be able to serialise and manipulate a 'network' (or filter
+ graph if you prefer).
+
+ We can visualise a multitrack in the way that an NLE presents it:
+
+ +-----------------+ +-----------------------+
+ 0: |a1 | |a2 |
+ +---------------+-+--------------------------+-+---------------------+
+ 1: |b1 |
+ +------------------------------+
+
+ The overlapping areas of track 0 and 1 would (presumably) have some kind of
+ transition - without a transition, the frames from a1 and a2 would be shown
+ during the areas of overlap.
+
+ MLT has a multitrack object, but it is not a producer in the sense that it
+ can be connected directly to a consumer and everything will work correctly.
+ A consumer would treat it precisely as it would a normal producer, and, in
+ the case of the multitrack above, you would never see anything from track 1
+ other than the transitions between the clips - the gap between a1 and a2
+ would show test frames.
+
+ This happens because a consumer pulls one frame from the producer it's
+ connected to while a multitrack will provide one frame per track.
+ Something, somewhere, must ensure that all frames are pulled from the
+ multitrack and elect the correct frame to pass on.
+
+ Hence, MLT provides a wrapper for the multitrack, which is called a
+ 'tractor', and its the tractors task to ensure that all tracks are pulled
+ evenly, the correct frame is output and that we have 'producer like'
+ behaviour.
+
+ Thus, a mulitrack is conceptually 'pulled' by a tractor as shown here:
+
+ +----------+
+ |multitrack|
+ | +------+ | +-------+
+ | |track0|-|--->|tractor|
+ | +------+ | |\ |
+ | | | \ |
+ | +------+ | | \ |
+ | |track1|-|--->|---o---|--->
+ | +------+ | | / |
+ | | | / |
+ | +------+ | |/ |
+ | |track2|-|--->| |
+ | +------+ | +-------+
+ +----------+
+
+ With a combination of the two, we can now connect multitracks to consumers.
+ The first non-test card will be retrieved and passed on.
+
+ The tracks can be producers, playlists, or even other tractors.
+
+ Now we wish to insert filters and transitions between the mulitrack and the
+ tractor. We can do this directly by inserting filters directly between the
+ tractor and the multitrack, but this involves a lot of connecting and
+ reconnecting left and right producers and consumers, and it seemed only fair
+ that we should be able to automate that process.
+
+ So in keeping with our agricultural theme, the concept of the 'field' was
+ born. We 'plant' filters and transitions in the field and the tractor pulls
+ the multitrack (think of a combine harvester :-)) over the field and
+ produces a 'bail' (sorry - kidding - frame :-)).
+
+ Conceptually, we can see it like this:
+
+ +----------+
+ |multitrack|
+ | +------+ | +-------------+ +-------+
+ | |track0|-|--->|field |--->|tractor|
+ | +------+ | | | |\ |
+ | | | filters | | \ |
+ | +------+ | | and | | \ |
+ | |track1|-|--->| transitions |--->|---o---|--->
+ | +------+ | | | | / |
+ | | | | | / |
+ | +------+ | | | |/ |
+ | |track2|-|--->| |--->| |
+ | +------+ | +-------------+ +-------+
+ +----------+
+
+ In reality, we create a field first, and from that we obtain a multitrack
+ and a tractor. We can then populate the multitrack, field and finally,
+ connect the tractor to the consumer.
+
+ The reasoning behind this is possibly flawed - it might have made more
+ sense to produce the tractor and have it encapsulate the field and the
+ multitrack as that is how it looks to a connected consumer:
+
+ +-----------------------------------------------+
+ |tractor +--------------------------+ |
+ | +----------+ | +-+ +-+ +-+ +-+ | |
+ | |multitrack| | |f| |f| |t| |t| | |
+ | | +------+ | | |i| |i| |r| |r| | |
+ | | |track0|-|--->| |l|- ->|l|- ->|a|--->|a|\| |
+ | | +------+ | | |t| |t| |n| |n| | |
+ | | | | |e| |e| |s| |s| |\ |
+ | | +------+ | | |r| |r| |i| |i| | \|
+ | | |track1|-|- ->| |0|--->|1|--->|t|--->|t|-|--o--->
+ | | +------+ | | | | | | |i| |i| | /|
+ | | | | | | | | |o| |o| |/ |
+ | | +------+ | | | | | | |n| |n| | |
+ | | |track2|-|- ->| | |- ->| |--->|0|- ->|1|/| |
+ | | +------+ | | | | | | | | | | | |
+ | +----------+ | +-+ +-+ +-+ +-+ | |
+ | +--------------------------+ |
+ +-----------------------------------------------+
+
+ An example will hopefully clarify this.
+
+ Let's assume that we want to provide a 'watermark' to our hello world
+ example. We have already extended the example to play multiple clips,
+ and now we will place a text based watermark, reading 'Hello World' in
+ the top left hand corner:
+
+ mlt_producer create_tracks( int argc, char **argv )
+ {
+ // Create the field
+ mlt_field field = mlt_field_init( );
+
+ // Obtain the multitrack
+ mlt_multitrack multitrack = mlt_field_multitrack( field );
+
+ // Obtain the tractor
+ mlt_tractor tractor = mlt_field_tractor( field );
+
+ // Obtain a composite transition
+ mlt_transition transition = mlt_factory_transition( "composite", "10%,10%:15%x15%" );
+
+ // Create track 0
+ mlt_producer track0 = create_playlist( argc, argv );
+
+ // Create the watermark track - note we NEED fezzik for scaling here
+ mlt_producer track1 = mlt_factory_producer( "fezzik", "pango" );
+
+ // Get the length of track0
+ mlt_position length = mlt_producer_get_playtime( track0 );
+
+ // Set the properties of track1
+ mlt_properties properties = mlt_producer_properties( track1 );
+ mlt_properties_set( properties, "text", "Hello\nWorld" );
+ mlt_properties_set_position( properties, "in", 0 );
+ mlt_properties_set_position( properties, "out", length - 1 );
+ mlt_properties_set_position( properties, "length", length );
+
+ // Now set the properties on the transition
+ properties = mlt_transition_properties( transition );
+ mlt_properties_set_position( properties, "in", 0 );
+ mlt_properties_set_position( properties, "out", length - 1 );
+
+ // Add our tracks to the multitrack
+ mlt_multitrack_connect( multitrack, track0, 0 );
+ mlt_multitrack_connect( multitrack, track1, 1 );
+
+ // Now plant the transition
+ mlt_field_plant_transition( field, transition, 0, 1 );
+
+ // Now set the properties on the tractor
+ properties = mlt_tractor_properties( tractor );
+ mlt_properties_set_data( properties, "multitrack", multitrack, 0, ( mlt_destructor )mlt_multitrack_close, NULL );
+ mlt_properties_set_data( properties, "field", field, 0, ( mlt_destructor )mlt_field_close, NULL );
+ mlt_properties_set_data( properties, "track0", track0, 0, ( mlt_destructor )mlt_producer_close, NULL );
+ mlt_properties_set_data( properties, "track1", track1, 0, ( mlt_destructor )mlt_producer_close, NULL );
+ mlt_properties_set_data( properties, "transition", transition, 0, ( mlt_destructor )mlt_transition_close, NULL );
+
+ // Return the tractor
+ return mlt_tractor_producer( tractor );
+ }
+
+ Now all we need do is to replace these lines in the main function:
+
+ // Create a playlist
+ mlt_producer world = create_playlist( argc, argv );
+
+ with:
+
+ // Create a watermarked playlist
+ mlt_producer world = create_tracks( argc, argv );
+
+ and we have a means to play multiple clips with a horribly obtrusive
+ watermark - just what the world needed, right? ;-)
+
+
+SECTION 3 - STRUCTURE AND DESIGN
+--------------------------------
+
+Class Heirachy:
+
+ The mlt framework consists of an OO class heirachy which consists of the
+ following public classes and abstractions:
+
+ mlt_properties
+ mlt_frame
+ mlt_service
+ mlt_producer
+ mlt_playlist
+ mlt_tractor
+ mlt_filter
+ mlt_transition
+ mlt_consumer
+ mlt_deque
+ mlt_pool
+ mlt_factory
+
+ Each class defined above can be read as extending the classes above and to
+ the left.
+
+ The following sections describe the properties, stacking/queuing and memory
+ pooling functionality provided by the framework - these are key components
+ and a basic understanding of these is required for the remainder of the
+ documentation.
+
+
+mlt_properties:
+
+ The properties class is the base class for the frame and service classes.
+
+ It is designed to provide an efficient lookup table for various types of
+ information, such as strings, integers, floating points values and pointers
+ to data and data structures.
+
+ All properties are indexed by a unique string.
+
+ The most basic use of properties is as follows:
+
+ // 1. Create a new, empty properties set;
+ mlt_properties properties = mlt_properties_new( );
+
+ // 2. Assign the value "world" to the property "hello";
+ mlt_properties_set( properties, "hello", "world" );
+
+ // 3. Retrieve and print the value of "hello";
+ printf( "%s\n", mlt_properties_get( properties, "hello" ) );
+
+ // 4. Reassign "hello" to "world!";
+ mlt_properties_set( properties, "hello", "world!" );
+
+ // 5. Retrieve and print the value of "hello";
+ printf( "%s\n", mlt_properties_get( properties, "hello" ) );
+
+ // 6. Assign the value "0" to "int";
+ mlt_properties_set( properties, "int", "0" );
+
+ // 7. Retrieve and print the integer value of "int";
+ printf( "%d\n", mlt_properties_get_int( properties, "int" ) );
+
+ // 8. Assign the integer value 50 to "int2";
+ mlt_properties_set_int( properties, "int2", 50 );
+
+ // 9. Retrieve and print the double value of "int2";
+ printf( "%s\n", mlt_properties_get( properties, "int2" ) );
+
+ Steps 2 through 5 demonstrate that the "name" is unique - set operations on
+ an existing "name" change the value. They also free up memory associated to
+ the previous value. Note that it also possible to change type in this way
+ too.
+
+ Steps 6 and 7 demonstrate that the properties object handles deserialisation
+ from strings. The string value of "50" is set, the integer value of 50 is
+ retrieved.
+
+ Steps 8 and 9 demonstrate that the properties object handles serialisation
+ to strings.
+
+ To show all the name/value pairs in a properties, it is possible to iterate
+ through them:
+
+ int i = 0;
+ for ( i = 0; i < mlt_properties_count( properties ); i ++ )
+ printf( "%s = %s\n", mlt_properties_get_name( properties, i ),
+ mlt_properties_get_value( properties, i ) );
+
+ Note that properties are retrieved in the order in which they are set.
+
+ Properties are also used to hold pointers to memory. This is done via the
+ set_data call:
+
+ uint8_t *image = malloc( size );
+ mlt_properties_set_data( properties, "image", image, size, NULL, NULL );
+
+ In this example, we specify that the pointer can be retrieved from
+ properties by a subsequent request to get_data:
+
+ image = mlt_properties_get_data( properties, "image", &size );
+
+ or:
+
+ image = mlt_properties_get_data( properties, "image", NULL );
+
+ if we don't wish to retrieve the size.
+
+ Two points here:
+
+ 1) The allocated memory remains after the properties object is closed unless
+ you specify a destructor. In the case above, this can be done with:
+
+ mlt_properties_set_data( properties, "image", image, size, free, NULL );
+
+ When the properties are closed, or the value of "image" is changed, the
+ destructor is invoked.
+
+ 2) The string value returned by mlt_properties_get is NULL. Typically, you
+ wouldn't wish to serialise an image as a string, but other structures
+ might need such functionality - you can specify a serialiser as the last
+ argument if required (declaration is char *serialise( void * )).
+
+ Properties also provides some more advanced usage capabilities.
+
+ It has the ability to inherit all serialisable values from another properties
+ object:
+
+ mlt_properties_inherit( this, that );
+
+ It has the ability to mirror properties set on this on another set of
+ properties:
+
+ mlt_properties_mirror( this, that );
+
+ After this call, all serialisable values set on this are passed on to that.
+
+
+mlt_deque:
+
+ Stacks and queues are essential components in the MLT framework. Being of a
+ lazy disposition, we elected to implement a 'Double Ended Queue' (deque) -
+ this encapsulates the functionality of both.
+
+ The API of the deque is defined as follows:
+
+ mlt_deque mlt_deque_init( );
+ int mlt_deque_count( mlt_deque this );
+ int mlt_deque_push_back( mlt_deque this, void *item );
+ void *mlt_deque_pop_back( mlt_deque this );
+ int mlt_deque_push_front( mlt_deque this, void *item );
+ void *mlt_deque_pop_front( mlt_deque this );
+ void *mlt_deque_peek_back( mlt_deque this );
+ void *mlt_deque_peek_front( mlt_deque this );
+ void mlt_deque_close( mlt_deque this );
+
+ The stacking operations are used in a number of places:
+
+ * Reverse Polish Notation (RPN) image and audio operations
+ * memory pooling
+
+ The queuing operations are used in typical frame based consumers to allow
+ buffering.
+
+
+mlt_pool:
+
+ The MLT framework provides memory pooling capabilities through the mlt_pool
+ API. Once initilialised, these can be seen as a straightforward drop in
+ replacement for malloc/realloc/free functionality.
+
+ The background behind this API is that malloc/free operations are
+ notoriously inefficient, especially when dealing with large blocks of memory
+ (such as an image). On linux, malloc is optimised for memory allocations
+ less than 128k - memory blocks allocated of these sizes or less are retained
+ in the process heap for subsequent reuse, thus bypassing the kernel calls
+ for repeated allocation/frees for small blocks of memory. However, blocks of
+ memory larger than that require kernel calls and this has a detrimental
+ impact on performance.
+
+ The mlt_pool design is simply to hold a list of stacks - there is one stack
+ per 2^n bytes (where n is between 8 and 31). When an alloc is called, the
+ requested size is rounded to the next 2^n, the stack is retrieved for that
+ size, and an item is popped or created if the stack is empty.
+
+ Each item has a 'header', situated immediately before the returned address -
+ this holds the 'stack' to which the item belongs.
+
+ When an item is released, we retrieve the header, obtain the stack and push
+ it back.
+
+ Thus, from the programmers point of view, the API is the same as the
+ traditional malloc/realloc/free calls:
+
+ void *mlt_pool_alloc( int size );
+ void *mlt_pool_realloc( void *ptr, int size );
+ void mlt_pool_release( void *release );
+
+
+mlt_frame:
+
+ A frame object is essentially defined as:
+
+ +------------+
+ |frame |
+ +------------+
+ | properties |
+ | image stack|
+ | audio stack|
+ +------------+
+
+ The life cycle of a frame can be represented as follows:
+
+ +-----+----------------------+-----------------------+---------------------+
+ |Stage|Producer |Filter |Consumer |
+ +-----+----------------------+-----------------------+---------------------+
+ | 0.0 | | |Request frame |
+ +-----+----------------------+-----------------------+---------------------+
+ | 0.1 | |Receives request | |
+ | | |Request frame | |
+ +-----+----------------------+-----------------------+---------------------+
+ | 0.2 |Receives request | | |
+ | |Generates frame for | | |
+ | |current position | | |
+ | |Increments position | | |
+ +-----+----------------------+-----------------------+---------------------+
+ | 0.3 | |Receives frame | |
+ | | |Updates frame | |
+ +-----+----------------------+-----------------------+---------------------+
+ | 0.4 | | |Receives frame |
+ +-----+----------------------+-----------------------+---------------------+
+
+ Note that neither the filter nor the consumer have any conception of
+ 'position' until they receive a frame. Speed and position are properties of
+ the producer, and they are assigned to the frame object when the producer
+ creates it.
+
+ Step 0.3 is a critical one here - if the filter determines that the frame is
+ of interest to it, then it should manipulate the image and/or audio stacks
+ and properties as required.
+
+ Assuming that the filter deals with both image and audio, then it should
+ push data and methods on to the stacks which will deal with the processing.
+ This can be done with the mlt_frame_push_image and audio methods. In order for
+ the filter to register interest in the frame, the stacks should hold:
+
+ image stack:
+ [ producer_get_image ] [ data1 ] [ data2 ] [ filter_get_image ]
+
+ audio stack:
+ [ producer_get_audio ] [ data ] [ filter_get_audio ]
+
+ The filter_get methods are invoked automatically when the consumer invokes a
+ get_image on the frame.
+
+ +-----+----------------------+-----------------------+---------------------+
+ |Stage|Producer |Filter |Consumer |
+ +-----+----------------------+-----------------------+---------------------+
+ | 1.0 | | |frame_get_image |
+ +-----+----------------------+-----------------------+---------------------+
+ | 1.1 | |filter_get_image: | |
+ | | | pop data2 and data1 | |
+ | | | frame_get_image | |
+ +-----+----------------------+-----------------------+---------------------+
+ | 1.2 |producer_get_image | | |
+ | | Generates image | | |
+ +-----+----------------------+-----------------------+---------------------+
+ | 1.3 | |Receives image | |
+ | | |Updates image | |
+ +-----+----------------------+-----------------------+---------------------+
+ | 1.4 | | |Receives image |
+ +-----+----------------------+-----------------------+---------------------+
+
+ Obviously, if the filter isn't interested in the image, then it should leave
+ the stack alone, and then the consumer will retrieve its image directly from
+ the producer.
+
+ Similarly, audio is handled as follows:
+
+ +-----+----------------------+-----------------------+---------------------+
+ |Stage|Producer |Filter |Consumer |
+ +-----+----------------------+-----------------------+---------------------+
+ | 2.0 | | |frame_get_audio |
+ +-----+----------------------+-----------------------+---------------------+
+ | 2.1 | |filter_get_audio: | |
+ | | | pop data | |
+ | | | frame_get_audio | |
+ +-----+----------------------+-----------------------+---------------------+
+ | 2.2 |producer_get_audio | | |
+ | | Generates audio | | |
+ +-----+----------------------+-----------------------+---------------------+
+ | 2.3 | |Receives audio | |
+ | | |Updates audio | |
+ +-----+----------------------+-----------------------+---------------------+
+ | 2.4 | | |Receives audio |
+ +-----+----------------------+-----------------------+---------------------+
+
+ And finally, when the consumer is done with the frame, it should close it.
+
+ Note that a consumer may not evaluate both image and audio for any given
+ frame, especially in a realtime environment. See 'Realtime Considerations'
+ below.
+
+ By default, a frame has the following properties:
+
+ +------------------+------------------------------------+------------------+
+ |Name |Description |Values |
+ +------------------+------------------------------------+------------------+
+ |_position |The producers frame position |0 to n |
+ +------------------+------------------------------------+------------------+
+ |_speed |The producers speed |double |
+ +------------------+------------------------------------+------------------+
+ |image |The generated image |NULL or pointer |
+ +------------------+------------------------------------+------------------+
+ |alpha |The generated alpha mask |NULL or pointer |
+ +------------------+------------------------------------+------------------+
+ |width |The width of the image | |
+ +------------------+------------------------------------+------------------+
+ |height |The height of the image | |
+ +------------------+------------------------------------+------------------+
+ |normalised_width |The normalised width of the image |720 |
+ +------------------+------------------------------------+------------------+
+ |normalised_height |The normalised height of the image |576 or 480 |
+ +------------------+------------------------------------+------------------+
+ |progressive |Indicates progressive/interlaced |0 or 1 |
+ +------------------+------------------------------------+------------------+
+ |top_field_first |Indicates top field first |0 or 1 |
+ +------------------+------------------------------------+------------------+
+ |audio |The generated audio |NULL or pointer |
+ +------------------+------------------------------------+------------------+
+ |frequency |The frequency of the audio | |
+ +------------------+------------------------------------+------------------+
+ |channels |The channels of the audio | |
+ +------------------+------------------------------------+------------------+
+ |samples |The samples of the audio | |
+ +------------------+------------------------------------+------------------+
+ |aspect_ratio |The aspect ratio of the image |double |
+ +------------------+------------------------------------+------------------+
+ |test_image |Used to indicate no image available |0 or 1 |
+ +------------------+------------------------------------+------------------+
+ |test_audio |Used to indicate no audio available |0 or 1 |
+ +------------------+------------------------------------+------------------+
+
+ The consumer can attach the following properties which affect the default
+ behaviour of a frame:
+
+ +------------------+------------------------------------+------------------+
+ |test_card_producer|Synthesise test images from here |NULL or pointer |
+ +------------------+------------------------------------+------------------+
+ |consumer_aspect_ |Apply this aspect ratio to the test |double |
+ |ratio |card producer | |
+ +------------------+------------------------------------+------------------+
+ |rescale.interp |Use this scale method for test image|"string" |
+ +------------------+------------------------------------+------------------+
+
+ While most of these are mainly self explainatory, the normalised_width and
+ normalised_height values require a little explaination. These are required
+ to ensure that effects are consistently handled as PAL or NTSC, regardless
+ of the consumers or producers width/height image request.
+
+ The test_image and audio flags are used to determine when images and audio
+ should be synthesised.
+
+ Additional properties may be provided by the producer implementation, and
+ filters, transitions and consumers may add additional properties to
+ communicate specific requests. These are documented in modules.txt.
+
+ The complete API for the mlt frame is as follows:
+
+ mlt_frame mlt_frame_init( );
+ mlt_properties mlt_frame_properties( mlt_frame this );
+ int mlt_frame_is_test_card( mlt_frame this );
+ int mlt_frame_is_test_audio( mlt_frame this );
+ double mlt_frame_get_aspect_ratio( mlt_frame this );
+ int mlt_frame_set_aspect_ratio( mlt_frame this, double value );
+ mlt_position mlt_frame_get_position( mlt_frame this );
+ int mlt_frame_set_position( mlt_frame this, mlt_position value );
+ int mlt_frame_get_image( mlt_frame this, uint8_t **buffer, mlt_image_format *format, int *width, int *height, int writable );
+ uint8_t *mlt_frame_get_alpha_mask( mlt_frame this );
+ int mlt_frame_get_audio( mlt_frame this, int16_t **buffer, mlt_audio_format *format, int *frequency, int *channels, int *samples );
+ int mlt_frame_push_get_image( mlt_frame this, mlt_get_image get_image );
+ mlt_get_image mlt_frame_pop_get_image( mlt_frame this );
+ int mlt_frame_push_frame( mlt_frame this, mlt_frame that );
+ mlt_frame mlt_frame_pop_frame( mlt_frame this );
+ int mlt_frame_push_service( mlt_frame this, void *that );
+ void *mlt_frame_pop_service( mlt_frame this );
+ int mlt_frame_push_audio( mlt_frame this, void *that );
+ void *mlt_frame_pop_audio( mlt_frame this );
+ void mlt_frame_close( mlt_frame this );
+
+mlt_service:
+
+ The service base class extends properties and allows 0 to m inputs and 0 to
+ n outputs and is represented as follows:
+
+ +-----------+
+ - ->| |- ->
+ - ->| Service |- ->
+ - ->| |
+ +-----------+
+ | properties|
+ +-----------+
+
+ Descendents of service impose restrictions on how inputs and outputs can be
+ connected and will provide a basic set of properties. Typically, the service
+ instance is encapsulated by the descendent in order for it to ensure that
+ its connection rules are followed.
+
+ A service does not define any properties when constructed. It should be
+ noted that producers, filters and transitions my be serialised (say, via the
+ westley consumer), and care should be taken to distinguish between
+ serialisable and transient properties. The convention used is to prefix
+ transient properties with an underscore.
+
+ The public interface is defined by the following functions:
+
+ int mlt_service_init( mlt_service this, void *child );
+ mlt_properties mlt_service_properties( mlt_service this );
+ int mlt_service_connect_producer( mlt_service this, mlt_service producer, int index );
+ int mlt_service_get_frame( mlt_service this, mlt_frame_ptr frame, int index );
+ void mlt_service_close( mlt_service this );
+
+ Typically, only direct descendents of services need invoke these methods and
+ developers are encouraged to use those extensions when definining new
+ services.
+
+
+mlt_producer:
+
+ A producer has 0 inputs and 1 output:
+
+ +-----------+
+ | |
+ | Producer |--->
+ | |
+ +-----------+
+ | service |
+ +-----------+
+
+ A producer provides an abstraction for file readers, pipes, streams or any
+ other image or audio input.
+
+ When instantiated, a producer has the following properties:
+
+ +------------------+------------------------------------+------------------+
+ |Name |Description |Values |
+ +------------------+------------------------------------+------------------+
+ |mlt_type |The producers type |mlt_producer |
+ +------------------+------------------------------------+------------------+
+ |_position |The producers frame position |0 to n |
+ +------------------+------------------------------------+------------------+
+ |_speed |The producers speed |double |
+ +------------------+------------------------------------+------------------+
+ |fps |The output frames per second |25 or 29.97 |
+ +------------------+------------------------------------+------------------+
+ |in |The in point in frames |0 to length - 1 |
+ +------------------+------------------------------------+------------------+
+ |out |The out point in frames |in to length - 1 |
+ +------------------+------------------------------------+------------------+
+ |length |The length of the input in frames |0 to n |
+ +------------------+------------------------------------+------------------+
+ |aspect_ratio |aspect_ratio of the source |0 to n |
+ +------------------+------------------------------------+------------------+
+ |eof |end of clip behaviour |"pause" or "loop" |
+ +------------------+------------------------------------+------------------+
+ |resource |Constructor argument (ie: file name)|"<resource>" |
+ +------------------+------------------------------------+------------------+
+
+ Additional properties may be provided by the producer implementation.
+
+ The public interface is defined by the following functions:
+
+ int mlt_producer_init( mlt_producer this, void *child );
+ mlt_service mlt_producer_service( mlt_producer this );
+ mlt_properties mlt_producer_properties( mlt_producer this );
+ int mlt_producer_seek( mlt_producer this, mlt_position position );
+ mlt_position mlt_producer_position( mlt_producer this );
+ mlt_position mlt_producer_frame( mlt_producer this );
+ int mlt_producer_set_speed( mlt_producer this, double speed );
+ double mlt_producer_get_speed( mlt_producer this );
+ double mlt_producer_get_fps( mlt_producer this );
+ int mlt_producer_set_in_and_out( mlt_producer this, mlt_position in, mlt_position out );
+ mlt_position mlt_producer_get_in( mlt_producer this );
+ mlt_position mlt_producer_get_out( mlt_producer this );
+ mlt_position mlt_producer_get_playtime( mlt_producer this );
+ mlt_position mlt_producer_get_length( mlt_producer this );
+ void mlt_producer_prepare_next( mlt_producer this );
+ void mlt_producer_close( mlt_producer this );
+
+ For the sake of discussion here, we'll assume that someone wants to provide
+ a new producer which simply generates green frames and silent audio :-) -
+ we'll call the producer 'green'.
+
+ // Forward reference
+ static int producer_get_frame( mlt_service service, mlt_frame_ptr frame, int index );
+
+ mlt_producer producer_green( void *arg )
+ {
+ // Create a new producer
+ mlt_producer this = mlt_producer_new( );
+
+ // Check that we were allocated a new producer
+ if ( this != NULL )
+ {
+ // Get the service
+ mlt_service service = mlt_producer_service( this );
+
+ // We need to override the get_frame method
+ service->get_frame = producer_get_frame;
+ }
+
+ // Return this producer
+ return this;
+ }
+
+ static int producer_get_frame( mlt_service service, mlt_frame_ptr frame, int index )
+ {
+ // Create a new frame
+ *frame = mlt_frame_init( );
+
+ // Specify the get_image
+ mlt_frame_push_get_image( *frame, producer_get_image );
+
+ // Specify the get_audio
+ mlt_frame_push_audio( *frame, producer_get_audio );
+
+ // Return that all was successful
+ return 0;
+ }
+
+mlt_filter:
+
+ The public interface is defined by the following functions:
+
+ int mlt_filter_init( mlt_filter this, void *child );
+ mlt_filter mlt_filter_new( );
+ mlt_service mlt_filter_service( mlt_filter this );
+ mlt_properties mlt_filter_properties( mlt_filter this );
+ mlt_frame mlt_filter_process( mlt_filter this, mlt_frame that );
+ int mlt_filter_connect( mlt_filter this, mlt_service producer, int index );
+ void mlt_filter_set_in_and_out( mlt_filter this, mlt_position in, mlt_position out );
+ int mlt_filter_get_track( mlt_filter this );
+ mlt_position mlt_filter_get_in( mlt_filter this );
+ mlt_position mlt_filter_get_out( mlt_filter this );
+ void mlt_filter_close( mlt_filter );
+
+
+mlt_transition:
+
+ The public interface is defined by the following functions:
+
+ int mlt_transition_init( mlt_transition this, void *child );
+ mlt_transition mlt_transition_new( );
+ mlt_service mlt_transition_service( mlt_transition this );
+ mlt_properties mlt_transition_properties( mlt_transition this );
+ int mlt_transition_connect( mlt_transition this, mlt_service producer, int a_track, int b_track );
+ void mlt_transition_set_in_and_out( mlt_transition this, mlt_position in, mlt_position out );
+ int mlt_transition_get_a_track( mlt_transition this );
+ int mlt_transition_get_b_track( mlt_transition this );
+ mlt_position mlt_transition_get_in( mlt_transition this );
+ mlt_position mlt_transition_get_out( mlt_transition this );
+ mlt_frame mlt_transition_process( mlt_transition this, mlt_frame a_frame, mlt_frame b_frame );
+ void mlt_transition_close( mlt_transition this );
+
+
+mlt_consumer:
+
+ The public interface is defined by the following functions:
+
+ int mlt_consumer_init( mlt_consumer this, void *child );
+ mlt_service mlt_consumer_service( mlt_consumer this );
+ mlt_properties mlt_consumer_properties( mlt_consumer this );
+ int mlt_consumer_connect( mlt_consumer this, mlt_service producer );
+ int mlt_consumer_start( mlt_consumer this );
+ mlt_frame mlt_consumer_get_frame( mlt_consumer this );
+ mlt_frame mlt_consumer_rt_frame( mlt_consumer this );
+ int mlt_consumer_stop( mlt_consumer this );
+ int mlt_consumer_is_stopped( mlt_consumer this );
+ void mlt_consumer_close( mlt_consumer );
+
+
+Specialised Producers:
+
+ There are two major types of specialised producers - playlists and tractors.
+
+ The following sections describe these.
+
+
+mlt_playlist:
+
+ mlt_playlist mlt_playlist_init( );
+ mlt_producer mlt_playlist_producer( mlt_playlist this );
+ mlt_service mlt_playlist_service( mlt_playlist this );
+ mlt_properties mlt_playlist_properties( mlt_playlist this );
+ int mlt_playlist_count( mlt_playlist this );
+ int mlt_playlist_clear( mlt_playlist this );
+ int mlt_playlist_append( mlt_playlist this, mlt_producer producer );
+ int mlt_playlist_append_io( mlt_playlist this, mlt_producer producer, mlt_position in, mlt_position out );
+ int mlt_playlist_blank( mlt_playlist this, mlt_position length );
+ mlt_position mlt_playlist_clip( mlt_playlist this, mlt_whence whence, int index );
+ int mlt_playlist_current_clip( mlt_playlist this );
+ mlt_producer mlt_playlist_current( mlt_playlist this );
+ int mlt_playlist_get_clip_info( mlt_playlist this, mlt_playlist_clip_info *info, int index );
+ int mlt_playlist_insert( mlt_playlist this, mlt_producer producer, int where, mlt_position in, mlt_position out );
+ int mlt_playlist_remove( mlt_playlist this, int where );
+ int mlt_playlist_move( mlt_playlist this, int from, int to );
+ int mlt_playlist_resize_clip( mlt_playlist this, int clip, mlt_position in, mlt_position out );
+ void mlt_playlist_close( mlt_playlist this );
+
+mlt_tractor:
+
+
+mlt_factory
+
projects / mlt / commitdiff