Write about CasparCG integration.

[nageru-docs] / video.rst

diff --git a/video.rst b/video.rst
index e1f1eb54aec8932bf7d2ab26436c3e52ed0cbdca..1244b0944b6b0a39c040a17520169f5634477310 100644 (file)
--- a/video.rst
+++ b/video.rst
@@ -32,15 +32,15 @@ and/or it has an alpha channel you want to use. Getting the format right makes
 for better efficiency; you not only save a conversion step on the CPU, but
 sometimes also on the GPU.
 
 for better efficiency; you not only save a conversion step on the CPU, but
 sometimes also on the GPU.
 

-Videos are loaded like this:
+Videos are loaded like this::

 
   local video = VideoInput.new("filename.mp4", Nageru.VIDEO_FORMAT_YCBCR)
 
 
   local video = VideoInput.new("filename.mp4", Nageru.VIDEO_FORMAT_YCBCR)
 

-or, for a network stream, perhaps:
+or, for a network stream, perhaps::

 
   local video = VideoInput.new("http://localhost/file.nut", Nageru.VIDEO_FORMAT_BGRA)
 
 
   local video = VideoInput.new("http://localhost/file.nut", Nageru.VIDEO_FORMAT_BGRA)
 

-It can then be added to any chain like this:
+It can then be added to any chain like this::

 
   local input = chain:add_video_input(video, false)
 
 
   local input = chain:add_video_input(video, false)
 


@@ -48,7 +48,7 @@ The second parameter specifies no deinterlacing. Note that interlaced video
 is currently not supported, not even with deinterlacing, so this parameter
 must always be false.
 
 is currently not supported, not even with deinterlacing, so this parameter
 must always be false.
 

-You can use the same video object to create many different video inputs:
+You can use the same video object to create many different video inputs::

   
   local input1 = chain1:add_video_input(video, false)
   local input2 = chain2:add_video_input(video, false)
   
   local input1 = chain1:add_video_input(video, false)
   local input2 = chain2:add_video_input(video, false)


@@ -61,13 +61,109 @@ moving the new file atomically into place, you could end up corrupting the file
 Nageru is playing from, causing it to automatically rewind before the end of
 the segment.
 
 Nageru is playing from, causing it to automatically rewind before the end of
 the segment.
 

+Videos are assigned an arbitrary signal number when loaded. Whenever you need
+to refer to this signal number (say, to get its width or height for display),
+you should use *video:get_signal_num()*. Like any other signal, videos have
+a width and height, an interlaced flag (currently always false), a frame rate
+(which can vary during playback) and has_signal/is_connected member functions.
+The former is always true, but the former will be false if the video isn't
+currently playing for whatever reason (e.g., the file is corrupted, or a network
+stream is broken and hasn't reconnected yet).
+

 
 Controlling video playback
 --------------------------
 
 
 Controlling video playback
 --------------------------
 

-TODO
+Themes have some programmatic control over video playback. In particular,
+if you want to make a video start from the beginning, you can do::
+
+  video:rewind()
+
+which will instantly make it start from the first frame again. This can be
+useful if you e.g. want the video to start when you're switching to it,
+or if you're not really using it to loop (e.g. as a transition marker).
+
+You can also change its rate, e.g. by::
+
+  video:change_rate(2.0)
+
+This will make it play at twice its usual speed. Your rate should not be
+negative nor exactly zero. You can set a rate to e.g. 1e-6 if you want to
+in practice stop the video; once you change it back to normal speed,
+the next frame will resume playing.
+

 
 Integration with CasparCG
 -------------------------
 
 
 Integration with CasparCG
 -------------------------
 

-TODO
+`CasparCG <http://casparcg.com/>`_ is an open-source broadcast graphics system,
+originally written by SVT, the Swedish public TV broadcaster. (In this
+context, “graphics” refers mostly to synthetically generated content,
+such as the score box in a sporting match.) With some coaxing, it is possible
+to integrate CasparCG with Nageru, so that Nageru does the mixing of the video
+sources and CasparCG generates graphics—CasparCG can also work as a standalone
+mixer indepedently of Nageru, but this will not be covered here.
+
+The most straightforward use of CasparCG is to use it to generate an overlay,
+which is then taken in as a video input in Nageru. To achieve this, the simplest
+solution is to send raw BGRA data over a UNIX domain socket [#rawvideo]_, which involves
+adding an FFmpeg output to your CasparCG configuration. This can either be done
+by modifying your casparcg.config to open up a socket in your home directory
+(you shouldn't use /tmp on a multi-user machine, or you may open up a security
+hole)::
+
+  <consumers>
+    <ffmpeg>
+      <device>1</device>
+      <path>unix:///home/user/caspar.sock</path>
+      <args>-c:v rawvideo -vf format=pix_fmts=bgra -f nut -listen 1</args>
+    </ffmpeg>
+    <system-audio></system-audio>
+  </consumers>
+
+or by setting it up on-the-fly through ACMP::
+
+  add 1 stream unix:///home/user/caspar.sock -vf format=pix_fmts=bgra -f nut -listen 1
+
+You can then use *unix:///home/user/caspar.sock* as a video input to Nageru on the
+same machine, and then use e.g. *OverlayEffect* to overlay it on your video chains.
+(Remember to set up the video as BGRA and not Y'CbCr, so that you get alpha.)
+
+CasparCG and Nageru does not run with synchronized clocks, so you will not get
+frame-perfect synchronization between graphics and overlay; however, this is normal
+even in a hardware chain, and most overlay graphics does not need to be timed
+to the input more than through a few frames. However, note that it is possible
+that Nageru lags behind CasparCG's graphics production after a while (typically
+on the order of hours) due to such clock skew; the easiest solution to this is
+just to use *change_rate(2.0)* or similar on the input, so that Nageru will consume
+CasparCG's frames as quickly as they come in without waiting any further.
+
+There's also one usability stumbling block: *CasparCG's FFmpeg
+streams are one-shot, and so are FFmpeg's UNIX domain sockets.* This means that,
+in practice, if Nageru ever disconnects from CasparCG for any reason, the socket
+is “used up”, and you will need to recreate it somehow (e.g., by restarting CasparCG).
+Also note that the old socket still lingers in place even after being useless,
+so you will _first_ need to remove it, and CasparCG does not do this for you.
+The simplest way to deal with this is probably to have a wrapper script of some
+sort that orchestrates Nageru, CasparCG and your client for you, so that everything
+is taken up and down in the right order; it may be cumbersome and require some
+tweaking for oyur specific case, but it's not a challenging problem per se.
+
+Nageru does not have functionality to work as a CasparCG client in itself,
+nor can your theme present a very detailed UI to do so. However, do note that
+since the theme is written in unrestricted Lua, so you can use e.g.
+`lua-http <https://github.com/daurnimator/lua-http>`_ to send signals
+to your backend (assuming it speaks HTTP) on e.g. transition changes.
+With some creativity, this allows you to at least bring some loose coupling
+between the two.
+
+In general, the integration between Nageru and CasparCG leaves a bit to be
+desired, and in general, the combination of CasparCG + Nageru will require
+a beefire machine than Nageru alone. However, it also provides a much richer
+environment for graphics, so for many use cases, it will be worth it.
+
+.. [#rawvideo] Good video codecs that support alpha are rare, so as long as CasparCG
+               and Nageru are running on the same machine, raw video is probably your
+               best bet. Even so, note that FFmpeg's muxers are not really made for
+               such large amounts of data that raw HD video produces, so there will
+               be some performance overhead on both sides of the socket.