1 // The x264 speed control tries to encode video at maximum possible quality
2 // without skipping frames (at the expense of higher encoding latency and
3 // less even output rates, although VBV is still respected). It does this
4 // by continuously (every frame) changing the x264 quality settings such that
5 // it uses maximum amount of CPU, but no more.
7 // Speed control works by maintaining a queue of frames, with the confusing
8 // nomenclature “full” meaning that there are no queues in the frame.
9 // (Conversely, if the queue is “empty” and a new frame comes in, we need to
10 // drop that frame.) It tries to keep the buffer 3/4 “full” by using a table
11 // of measured relative speeds for the different presets, and choosing one that it
12 // thinks will return the buffer to that state over time. However, since
13 // different frames take different times to encode regardless of preset, it
14 // also tries to maintain a running average of how long the typical frame will
15 // take to encode at the fastest preset (the so-called “complexity”), by dividing
16 // the actual time by the relative time for the preset used.
18 // Frame timings is a complex topic in its own sright, since usually, multiple
19 // frames are encoded in parallel. X264SpeedControl only supports the timing
20 // method that the original patch calls “alternate timing”; one simply measures
21 // the time the last x264_encoder_encode() call took. (The other alternative given
22 // is to measure the time between successive x264_encoder_encode() calls.)
23 // Unless using the zerocopy presets (which activate slice threading), the function
24 // actually returns not when the given frame is done encoding, but when one a few
25 // frames back is done encoding. So it doesn't actually measure the time of any
26 // given one frame, but it measures something correlated to it, at least as long as
27 // you are near 100% CPU utilization (ie., the encoded frame doesn't linger in the
28 // buffers already when x264_encoder_encode() is called).
30 // The code has a long history; it was originally part of Avail Media's x264
31 // branch, used in their encoder appliances, and then a snapshot of that was
32 // released. (Given that x264 is licensed under GPLv2 or newer, this means that
33 // we can also treat the patch as GPLv2 or newer if we want, which we do.
34 // As far as I know, it is copyright Avail Media, although no specific copyright
35 // notice was posted on the patch.)
37 // From there, it was incorporated in OBE's x264 tree (x264-obe) and some bugs
38 // were fixed. I started working on it for the purposes of Nageru, fixing various
39 // issues, adding VFR support and redoing the timings entirely based on more
40 // modern presets (the patch was made before several important x264 features,
41 // such as weighted P-frames). Finally, I took it out of x264 and put it into
42 // Nageru (it does not actually use any hooks into the codec itself), so that
43 // one does not need to patch x264 to use it in Nageru. It still could do with
44 // some cleanup, but it's much, much better than just using a static preset.
56 #include "x264_dynamic.h"
58 class X264SpeedControl {
60 // x264: Encoding object we are using; must be opened. Assumed to be
61 // set to the "faster" preset, and with 16 reference frames.
62 // f_speed: Relative encoding speed, usually 1.0.
63 // i_buffer_size: Number of frames in the buffer.
64 // f_buffer_init: Relative fullness of buffer at start
65 // (0.0 = assumed to be <i_buffer_size> frames in buffer,
66 // 1.0 = no frames in buffer)
67 X264SpeedControl(x264_t *x264, float f_speed, int i_buffer_size, float f_buffer_init);
70 // You need to call before_frame() immediately before each call to
71 // x264_encoder_encode(), and after_frame() immediately after.
73 // new_buffer_fill: Buffer fullness, in microseconds (_not_ a relative
74 // number, unlike f_buffer_init in the constructor).
75 // new_buffer_size: If > 0, new number of frames in the buffer,
76 // ie. the buffer size has changed. (It is harmless to set this
77 // even if the buffer hasn't actually changed.)
78 // f_uspf: If > 0, new microseconds per frame, ie. the frame rate has
79 // changed. (Of course, with VFR, it can be impossible to truly know
80 // the frame rate of the coming frames, but it is a reasonable
81 // assumption that the next second or so is likely to be the same
82 // frame rate as the last frame.)
83 void before_frame(float new_buffer_fill, int new_buffer_size, float f_uspf);
86 // x264 seemingly has an issue where x264_encoder_reconfig() is not reflected
87 // immediately in x264_encoder_parameters(). Since speed control keeps calling
88 // those two all the time, any changes you make outside X264SpeedControl
89 // could be overridden. Thus, to make changes to encoder parameters, you should
90 // instead set a function here, which will be called every time parameters
92 void set_config_override_function(std::function<void(x264_param_t *)> override_func)
94 this->override_func = override_func;
98 void set_buffer_size(int new_buffer_size);
99 int dither_preset(float f);
100 void apply_preset(int new_preset);
109 // all times that are not std::chrono::* are in usec
110 std::chrono::steady_clock::time_point timestamp; // when was speedcontrol last invoked
111 std::chrono::steady_clock::duration cpu_time_last_frame{std::chrono::seconds{0}}; // time spent encoding the previous frame
112 int64_t buffer_size; // assumed application-side buffer of frames to be streamed (measured in microseconds),
113 int64_t buffer_fill; // where full = we don't have to hurry
114 int64_t compensation_period; // how quickly we try to return to the target buffer fullness
115 float uspf; // microseconds per frame
116 int preset = -1; // which setting was used in the previous frame
117 float cplx_num = 3e3; // rolling average of estimated spf for preset #0. FIXME estimate initial complexity
126 int64_t min_buffer, max_buffer;
131 std::function<void(x264_param_t *)> override_func = nullptr;
134 Histogram metric_x264_speedcontrol_preset_used_frames;
135 std::atomic<double> metric_x264_speedcontrol_buffer_available_seconds{0.0};
136 std::atomic<double> metric_x264_speedcontrol_buffer_size_seconds{0.0};
137 std::atomic<int64_t> metric_x264_speedcontrol_idle_frames{0};
138 std::atomic<int64_t> metric_x264_speedcontrol_late_frames{0};