Use the new libavformat functionality for marking keyframes.

[nageru] / README

Nageru is a live video mixer, based around the standard M/E workflow.


Features:

 - High performance on modest hardware (720p60 with two input streams
   on my Thinkpad X240[1]); almost all pixel processing is done on the GPU.

 - High output quality; Lanczos3 scaling, subpixel precision everywhere,
   white balance adjustment, mix of 16- and 32-bit floating point
   for intermediate calculations, dithered output.

 - Proper sound support: Syncing of multiple unrelated sources through
   high-quality resampling, freely selectable input, cue out for headphones,
   dynamic range compression, simple EQ (low-cut), level meters conforming
   to EBU R128.

 - Theme engine encapsulating the design demands of each individual
   event; Lua code is responsible for setting up the pixel processing
   pipelines, running transitions etc., so that the visual look is
   consistent between operators.

[1] For reference, that is: Core i7 4600U (dualcore 2.10GHz, clocks down
to 800 MHz after 30 seconds due to thermal constraints), Intel HD Graphics
4400 (ie., without the extra L4 cache from Iris Pro), single-channel DDR3 RAM
(so 12.8 GB/sec theoretical memory bandwidth, shared between CPU and GPU).


Nageru is in beta stage. It currently needs:

 - An Intel processor with Intel Quick Sync, or otherwise some hardware
   H.264 encoder exposed through VA-API. Note that you can use VA-API over
   DRM instead of X11, to use a non-Intel GPU for rendering but still use
   Quick Sync (by giving e.g. “--va-display /dev/dri/renderD128”).

 - Two or more Blackmagic USB3 or PCI cards, either HDMI or SDI.
   The PCI cards need Blackmagic's own drivers installed. The USB3 cards
   are driven through the “bmusb” driver embedded in bmusb/, using libusb-1.0.
   You want a recent kernel and libusb-1.0; see below.

 - Movit, my GPU-based video filter library (https://movit.sesse.net).
   You will need at least version 1.3.1.

 - Qt 5.5 or newer for the GUI.

 - libmicrohttpd for the embedded web server.

 - x264 for encoding high-quality video suitable for streaming to end users
   (but see below).

 - ffmpeg for muxing, and for encoding audio.

 - Working OpenGL; Movit works with almost any modern OpenGL implementation.
   Nageru has been tested with Intel on Mesa (you want 11.2 or newer, due
   to critical stability bugfixes), and with NVIDIA's proprietary drivers.
   AMD's proprietary drivers (fglrx) are known not to work due to driver bugs;
   I am in contact with AMD to try to get this resolved.

 - libzita-resampler, for resampling sound sources so that they are in sync
   between sources, and also for oversampling for the peak meter.

 - Lua, for driving the theme engine.


If on Debian stretch or something similar, you can install everything you need
with:

  git submodule update --init
  apt install qtbase5-dev qt5-default pkg-config libmicrohttpd-dev \
    libusb-1.0-0-dev liblua5.2-dev libzita-resampler-dev libva-dev \
    libavcodec-dev libavformat-dev libswscale-dev libavresample-dev \
    libmovit-dev libegl1-mesa-dev libasound2-dev libx264-dev

Exceptions as of June 2016:

  - You need libusb 1.0.21 or newer (not yet released; you will need to
    use git) if you want zerocopy USB. Zerocopy USB helps not only for
    performance, but also for stability.

  - You want kernel 4.6.0 or newer to get zerocopy USB. Older kernels
    also lack some workarounds for LPM (link power management) and bandwidth
    allocation issues with USB3.

  - There is a critical bug fix with x264 speed control in
    928bd9d5def4f0ca5071ea176a11b816a01e6495, pushed to git mid-June 2016.

  - Nageru depends on an avformat API for marking block boundaries in the
    muxed byte stream that didn't enter ffmpeg before
    4e7a9212820a56bc731c09b2f11ae1422d070837, pushed to git late June 2016.


The patches/ directory contains a patch that helps zita-resampler performance.
It is meant for upstream, but was not in at the time Nageru was released.
It is taken to be by Steinar H. Gunderson <sesse@google.com> (ie., my ex-work
email), and under the same license as zita-resampler itself.

To start it, just hook up your equipment, type “make” and then “./nageru”.
It is strongly recommended to have the rights to run at real-time priority;
it will make the USB3 threads do so, which will make them a lot more stable.
(A reasonable hack for testing is probably just to run it as root using sudo,
although you might not want to do that in production.) Note also that if you
are running a desktop compositor, it will steal significant amounts of GPU
performance. The same goes for PulseAudio.

Nageru will open a HTTP server at port 9095, where you can extract a live
H264+PCM signal in nut mux (e.g. http://127.0.0.1:9095/stream.nut).
It is probably too high bitrate (~25 Mbit/sec depending on content) to send to
users, but you can easily send it around in your internal network and then
transcode it in e.g. VLC. A copy of the stream (separately muxed) will also
be saved live to local disk.

If you have a fast CPU (typically a quadcore desktop; most laptops will spend
most of their CPU on running Nageru itself), you can use x264 for the outgoing
stream instead of Quick Sync; it is much better quality for the same bitrate,
and also has proper bitrate controls. Simple add --http-x264-video on the
command line. (You may also need to add something like "--x264-preset veryfast",
since the default "medium" preset might be too CPU-intensive, but YMMV.)
The stream saved to disk will still be the Quick Sync-encoded stream, as it is
typically higher bitrate and thus also higher quality. Note that if you add
".metacube" at the end of the URL (e.g. "http://127.0.0.1:9095/stream.ts.metacube"),
you will get a stream suitable for streaming through the Cubemap video reflector
(cubemap.sesse.net). A typical example would be:

  ./nageru --http-x264-video --x264-preset veryfast --x264-tune film \
      --http-mux mp4 --http-audio-codec libfdk_aac --http-audio-bitrate 128

If you are comfortable with using all your remaining CPU power on the machine
for x264, try --x264-speedcontrol, which will try to adjust the preset
dynamically for maximum quality, at the expense of somewhat higher delay.

See --help for more information on options in general.

The name “Nageru” is a play on the Japanese verb 投げる (nageru), which means
to throw or cast. (I also later learned that it could mean to face defeat or
give up, but that's not the intended meaning.)


Nageru's home page is at https://nageru.sesse.net/, where you can also find
contact information and link to the latest version.


Legalese: TL;DR: Everything is GPLv3-or-newer compatible, and see
Intel's copyright license at h264encode.h.


Nageru is Copyright (C) 2015 Steinar H. Gunderson <steinar+nageru@gunderson.no>.
Portions Copyright (C) 2003 Rune Holm.
Portions Copyright (C) 2010-2015 Fons Adriaensen <fons@linuxaudio.org>.
Portions Copyright (C) 2012-2015 Fons Adriaensen <fons@linuxaudio.org>.
Portions Copyright (C) 2008-2015 Fons Adriaensen <fons@linuxaudio.org>.
Portions Copyright (c) 2007-2013 Intel Corporation. All Rights Reserved.


This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.


Portions of h264encode.h and h264encode.cpp:

Copyright (c) 2007-2013 Intel Corporation. All Rights Reserved.

Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sub license, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice (including the
next paragraph) shall be included in all copies or substantial portions
of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT.
IN NO EVENT SHALL PRECISION INSIGHT AND/OR ITS SUPPLIERS BE LIABLE FOR
ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


All files in decklink/:

Copyright (c) 2009 Blackmagic Design
Copyright (c) 2015 Blackmagic Design

Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:

The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.