1 MLT XML Schema Documentation
3 Copyright (C) 2004-2009 Ushodaya Enterprised Limited
4 Authors: Charles Yates <charles.yates@pandora.be>
5 Last Revision: 2009-05-08
13 This is the MLT projects XML serialisation/deserialisation format -
14 as such, it closely mirrors the internal structure of the MLT API.
16 If you just want to go straight to the DTD, then see
17 mlt/src/modules/xml/mlt-xml.dtd, which gets installed at
18 $(prefix)/share/mlt/modules/xml/mlt-xml.dtd. Currently, the XML parser is
24 A MLT XML document is essentially a list of 'producers' - a producer is
25 an mlt object which generates mlt frames (images and associated audio
28 There are 3 types of producer:
30 * Basic Producers - these are typically file or device oriented feeds;
31 * Playlists - these are arrangements of multiple producers;
32 * Multitracks - these are the fx encapsulators.
34 In the mlt model, producers are created and attached to 'consumers' -
35 consumers are software playback components (such as SDL), or wrappers for
36 hardware drivers (such as sdi) or even the XML serialising
37 consumer itself (the latter doesn't receive frames - it merely
38 interrogates the connected producer for its configuration).
40 Although MLT XML was defined as a serialisation mechanism for instantiated
41 MLT components, this document will concentrate on the hand authoring of
47 As shall become apparent through the remainder of this document, the basic
48 tenet of MLT XML authoring is to organise the document in the following
51 1) create producer elements for each unique media clip in the project;
52 2) create playlists for each track;
53 3) create a multitrack and specify filters and transitions;
54 4) adding global filters.
56 While other uses of MLT XML exist, the approach taken here is to maximise
57 efficiency for complex projects.
62 The simplest MLT XML document is:
65 <producer id="producer0">
66 <property name="resource">clip1.dv</property>
70 The XML wrapping is of course superfluous here - loading this document
71 with MLT is identical to loading the clip directly.
73 Of course, you can specify additional properties. For example, consider an
74 MPEG file with multiple soundtracks - you could define a MLT XML document to
75 ensure that the second audio track is loaded:
78 <producer id="producer0">
79 <property name="resource">clip1.mpeg</property>
80 <property name="audio_track">1</property>
84 NB: This relies on the mpeg being handled by the avformat producer, rather
85 than the mcmpeg one. See services.txt for more details.
87 A more useful example comes with the pango producer for a text producer.
89 TODO: pango example...
93 1) It is better not to specify in/out points when defining basic producers
94 as these can be specified in the playlists. The reasoning is that in/out
95 restricts the amount of the clip available, and could lead to the same clip
96 being loaded multiple times if you need different regions of the clip
98 2) A MLT XML doc can be specified as a resource, so XML docs can naturally
99 encapsulate other XML docs.
104 Playlists provide a 'collection' structure for producers. These can be used
105 to define 'tracks' in the multitrack approach, or simple playlists for
106 sequential, single track playout.
108 As an example, the following defines two basic producers and a playlist with 3
112 <producer id="producer0">
113 <property name="resource">clip1.dv</property>
115 <producer id="producer1">
116 <property name="resource">clip2.dv</property>
118 <playlist id="playlist0">
119 <entry producer="producer0" in="0" out="2999"/>
120 <entry producer="producer1" in="0" out="999"/>
121 <entry producer="producer0" in="3000" out="6999"/>
125 Here we see how the playlist defines the in/out points of the basic
130 1) All in/out points are absolute frame positions relative to the producer
131 being appended to the playlist;
132 2) MLT XML documents are currently authored for a specific normalisation;
133 3) The last 'producer' in the document is the default for play out;
134 4) Playlists can reference the same producer multiple times. In/out regions
135 do not need to be contiguous - duplication and skipping is acceptable.
138 Interlude - Introducing Multitracks:
140 So far we've defined basic producers and playlists/tracks - the tractor is
141 the element that allows us to arrange our tracks and specify filters and
142 transitions. Similarly to a playlist, a tractor is a container.
144 Note that MLT doesn't see a filter or a transition as a producer in the
145 normal sense - filters and transitions are passive when it comes to seeking.
146 Internally, seeks are carried out on the producers. This is an important
147 point - MLT does not follow a traditional graph oriented model.
149 Visualising an MLT tractor and it's interaction with the consumer will
152 +----------------------------------------------+
154 | +----------+ +-+ +-+ +-+ +-+ |
155 | |multitrack| |f| |f| |t| |t| |
156 | | +------+ | |i| |i| |r| |r| |
157 | | |track0|-|--->|l|- ->|l|- ->|a|--->|a|\ |
158 | | +------+ | |t| |t| |n| |n| \ |
159 | | | |e| |e| |s| |s| \ |
160 | | +------+ | |r| |r| |i| |i| \ | +--------+
161 | | |track1|-|- ->|0|--->|1|--->|t|--->|t|-----|--->|consumer|
162 | | +------+ | | | | | |i| |i| / | +--------+
163 | | | | | | | |o| |o| / | ^
164 | | +------+ | | | | | |n| |n| / | |
165 | | |track2|-|- ->| |- ->| |--->|0|- ->|1|/ | |
166 | | +------+ | | | | | | | | | | |
167 | +----------+ +-+ +-+ +-+ +-+ | |
168 +----------------------------------------------+ |
172 |APPLICATION|--------------------------------------------+
175 Internally, all frames from all tracks pass through all the filters and
176 transitions - these are told which tracks to deal and which regions of the
179 Note that the application communicates with the producer - it can alter
180 playback speed, position, or even which producer is connected to which
183 The consumer receives the first non-blank frame (see below). It has no say
184 in the order in which gets them (the sdl consumer when used with melt might
185 appear to be an exception - it isn't - it simply has a route back to the
186 application to allow the application to interpret key presses).
191 To create a multitrack XML, we can use two playlists and introduce a
192 tractor. For the purposes of demonstration, I'll add a filter here too:
195 <producer id="producer0">
196 <property name="resource">clip1.dv</property>
198 <producer id="producer1">
199 <property name="resource">clip2.dv</property>
201 <playlist id="playlist0">
202 <entry producer="producer0" in="0" out="2999"/>
203 <blank length="1000"/>
204 <entry producer="producer0" in="3000" out="6999"/>
206 <playlist id="playlist1">
207 <blank length="3000"/>
208 <entry producer="producer1" in="0" out="999"/>
210 <tractor id="tractor0">
212 <track producer="playlist0"/>
213 <track producer="playlist1"/>
216 <property name="track">0</property>
217 <property name="mlt_service">greyscale</property>
222 Here we see that blank frames are inserted into the first playlist and a
223 blank is provided at the beginning of the second - this can be visualised in
224 the traditional timeline widget as follows:
226 +-------+ +-------------+
228 +-------+---+-------------+
232 Adding the filter on the top track, gives us:
234 +-------+ +-------------+
236 +-------+---+-------------+
238 --------+---+-------------+
242 Note that it's only applied to the visible parts of the top track.
244 The requirement to apply a filter to the output, as opposed to a specific
245 track leads us to the final item in the Rules section above. As an example,
246 let's assume we wish to watermark all output, then we could use the
250 <producer id="producer0">
251 <property name="resource">clip1.dv</property>
253 <producer id="producer1">
254 <property name="resource">clip2.dv</property>
256 <playlist id="playlist0">
257 <entry producer="producer0" in="0" out="2999"/>
258 <blank length="1000"/>
259 <entry producer="producer0" in="3000" out="6999"/>
261 <playlist id="playlist1">
262 <blank length="3000"/>
263 <entry producer="producer1" in="0" out="999"/>
265 <tractor id="tractor0">
267 <track producer="playlist0"/>
268 <track producer="playlist1"/>
271 <property name="track">0</property>
272 <property name="mlt_service">greyscale</property>
275 <tractor id="tractor1">
277 <track producer="tractor0"/>
280 <property name="mlt_service">watermark</property>
281 <property name="resource">watermark1.png</property>
286 Here we employ another tractor and we define a single track (being the
287 tractor we previously defined) and apply a watermarking filter there.
289 This is simply provided as an example - the watermarking functionality could
290 be better handled at the playout stage itself (ie: as a filter automatically
291 placed between all producers and the consumer).
293 Tracks act like "layers" in an image processing program like the GIMP. The
294 bottom-most track takes highest priority and higher layers are overlays
295 and do not appear unless there are gaps in the lower layers or unless
296 a transition is applied that merges the tracks on the specifed region.
297 Practically speaking, for A/B video editing it does not mean too much,
298 and it will work as expected; however, as a general rule apply any CGI
299 (graphic overlays with pixbuf or titles with pango) on tracks higher than
300 your video tracks. Also, this means that any audio-only tracks that are
301 lower than your video tracks will play rather than the audio from the video
302 clip. Remember, nothing is affected like mixing or compositing until one
303 applies a transition or appropriate filter.
306 <producer id="producer0">
307 <property name="resource">clip1.dv</property>
309 <playlist id="playlist0">
310 <entry producer="producer0"/>
312 <producer id="producer1">
313 <property name="resource">clip2.mpeg</property>
315 <playlist id="playlist1">
317 <entry producer="producer1"/>
319 <tractor id="tractor0" in="0" out="315">
320 <multitrack id="multitrack0">
321 <track producer="playlist0"/>
322 <track producer="playlist1"/>
324 <transition id="transition0" in="50" out="74">
325 <property name="a_track">0</property>
326 <property name="b_track">1</property>
327 <property name="mlt_service">luma</property>
329 <transition id="transition1" in="50" out="74">
330 <property name="a_track">0</property>
331 <property name="b_track">1</property>
332 <property name="mlt_service">mix</property>
333 <property name="start">0.0</property>
334 <property name="end">1.0</property>
339 A "luma" transition is a video wipe processor that takes a greyscale bitmap
340 for the wipe definition. When one does not specify a bitmap, luma performs
341 a dissolve. The "mix" transition does an audio mix, but it interpolates
342 between the gain scaling factors between the start and end properties -
343 in this example, from 0.0 (none of track B) to 1.0 (all of track B).
344 Because the bottom track starts out with a gap specified using the <blank>
345 element, the upper track appears during the blank segment. See the demos and
346 services.txt to get an idea of the capabilities of the included transitions.
350 The information presented above is considered the MLT XML "normal"
351 form. This is the output generated by the xml consumer, for example,
352 when used with melt. It is the output generated when you use the
353 "XML to File" consumer in the demo script, which beginners will find
354 most useful for learning to use MLT XML. This section describes
355 alternative forms the xml producer accepts.
357 First of all, the normal form is more of a linear format with producers
358 and playlists defined prior to their usage in a multitrack. The producer
359 also accepts a hierarchical format with producers as children of tracks
360 or playlist entries and with playlists as children of tracks:
369 <property name="resource">clip1.dv</property>
378 Obviously, this example is meant to demonstrate hierarchy and not effective
379 use of playlist or multitrack!
381 Secondly, as part of error handling, the producer is forgiving if you fail
382 to supply <tractor>, <track>, and <entry> where one can be understood.
383 This affords an abbreviated syntax that is less verbose and perhaps less
384 intimidating for a human to read and understand. One can simplify the
391 <property name="resource">clip1.dv</property>
397 Yes, filters and transitions can be added to the above example after the
398 closing multitrack tag (</multitrack>) because it is still enclosed within
401 If you specify in and out on a producer and it has been enclosed within
402 an <entry> or <playlist>, then the edit points apply to the playlist
403 entry and not to the producer itself. This facilitates re-use of media:
406 <producer id="clip1" in="25" out="78">
407 <property name="resource">clip1.dv</property>
409 <entry producer="clip1" in="119" out="347"/>
412 In the above example, the producer attribute of the entry element is
413 a reference to the preceding producer. All references must follow the
414 definition. The edit points supplied on the producer above will not affect
415 the entry that references it below because the producer knows the clip is a
416 playlist entry and optimises this situation. The advantage is that one
417 does not need to determine every clip to be included ahead of time
418 and specify them outside the context of the mutlitrack timeline.
420 This form of authoring will be easier for many to visualise as a non-linear
421 editor's timeline. Here is a more complex example:
426 <producer id="foo" in="100" out="149">
427 <property name="resource">clip2.mpeg</property>
430 <entry producer="foo" in="10" out="59"/>
434 <producer id="bar" in="100" out="199">
435 <property name="resource">clip3.mpeg</property>
437 <entry out="99" producer="bar"/>
440 <filter mlt_service="greyscale" track="0"/>
441 <transition mlt_service="luma" in="25" out="49" a_track="0" b_track="1"/>
442 <transition mlt_service="luma" in="75" out="99" a_track="0" b_track="1">
443 <property name="reverse" value="1"/>
447 Did you notice something different in the last example? Properties can be
448 expressed using XML attributes on the element as well. However, only
449 non-service-specific properties are supported in this way. For example,
450 "mlt_service" is available to any producer, filter, or transition. However,
451 "resource" is actually service-specific. Notice the syntax of the last
452 property, on the last transition. The producer accepts property values using
453 the "value" attribute as well as using element text.
455 We have seen a few different ways of expressing property values. There are
456 a couple more for properties that can accept XML data. For example, the
457 GDK pixbuf producer with librsvg can handle embedded SVG, and the Pango
458 producer can handle embedded Pango markup. You can enclose the embedded XML
459 using a CDATA section:
461 <property name="resource"><![CDATA[ <svg>...</svg> ]]></property>
463 Please ensure the opening CDATA tag immediately follows the opening
464 property tag and that the section closing tag immediately precedes the
465 closing property tag.
467 However, the producer can also accept inline embedded XML:
469 <property name="resource">
474 Currently, there is no namespace handling so a conflict will occur only on
475 any embedded XML that contains an element named "property" because
476 the producer collects embedded XML until it reaches a closing property tag.
479 Entities and Parameterisation:
481 The MLT XML producer parser supports XML entities. An example:
483 <?xml version="1.0"?>
485 <!ENTITY msg "Hello world!">
488 <producer id="producer0">
489 <property name="mlt_service">pango</property>
490 <property name="text">&msg;</property>
494 If you are embedding another XML document into a property value not using
495 a CNODE section, then any DOCTYPE section must be relocated before any of
496 the xml elements to be well-formed. See demo/svg.mlt for an example.
498 Entities can be used to parameterise MLT XML! Using the above example, the
499 entity declared serves as the default value for &msg;. The entity content
500 can be overridden from the resource property supplied to the xml
501 producer. The syntax is the familiar, url-encoded query string used with
502 HTTP, e.g.: file?name=value&name=value...
504 There are a couple of rules of usage. The melted LOAD command and melt
505 command line tool require you to preface the URL with "xml:" because
506 the query string destroys the filename extension matching peformed by
507 the auto-loader. Also, melt looks for '=' to tokenise property settings.
508 Therefore, one uses ':' between name and value instead of '='. Finally,
509 since melt is run from the shell, one must enclose the URL within single
510 quotes to prevent shell filename expansion, or similar.
512 Needless to say, the ability to parameterise MLT XML compositions is
513 an extremely powerful tool. An example for you to play with is available in
514 demo/entity.mlt. Try overriding the name from melt:
515 melt 'xml:entity.mlt?name:Charlie'
517 Technically, the entity declaration is not needed in the head of the XML
518 document if you always supply the parameter. However, you run the risk
519 of unpredictable behviour without one. Therefore, it is safest and a best
520 practice to always supply an entity declaration. It is improves the
521 readability as one does not need to search for the entity references to
522 see what parameters are available.
527 If one finds the above hierarchical, abbreviated format intuitive,
528 start with a simple template and fill and extend as needed:
533 ...add a playlist for each track...
535 ...add filters and transitions...
538 By using a playlist for each track, it is easier to iteratively add new
539 clips and blank regions as you develop the project. You will not have to
540 use <track> or later add <playlist> when necessary.
542 A more advanced template that allows sequencing multitracks is:
548 ...add a playlist for each track...
550 ...add filters and transitions...
557 ...add a playlist for each track...
559 ...add filters and transitions...
563 If you end up making a collection of templates for various situations, then
564 consider using XML Entities to make the template more effective by moving
565 anything that should parameterised into an entity.
567 If you want to have a silent, black background for audio and video fades,
568 then make the top track simply <producer mlt_service="colour"/>. Then,
569 use composite and volume effects. See the "Fade from/to black/silence"
570 demo for an example (demo/mlt_fade_black).
572 If you apply the reverse=1 property to a transition like "luma," then
573 be careful because it also inherently swaps the roles of A and B tracks.
574 Therefore, you need to might need to swap the a_track and b_track values
575 if it did not turn out the way you expected. See the "Clock in and out"
576 for an example (demo/mlt_clock_in_and_out).