1 #ifndef _X264_SPEED_CONTROL_H
2 #define _X264_SPEED_CONTROL_H 1
4 // The x264 speed control tries to encode video at maximum possible quality
5 // without skipping frames (at the expense of higher encoding latency and
6 // less even output rates, although VBV is still respected). It does this
7 // by continuously (every frame) changing the x264 quality settings such that
8 // it uses maximum amount of CPU, but no more.
10 // Speed control works by maintaining a queue of frames, with the confusing
11 // nomenclature “full” meaning that there are no queues in the frame.
12 // (Conversely, if the queue is “empty” and a new frame comes in, we need to
13 // drop that frame.) It tries to keep the buffer 3/4 “full” by using a table
14 // of measured relative speeds for the different presets, and choosing one that it
15 // thinks will return the buffer to that state over time. However, since
16 // different frames take different times to encode regardless of preset, it
17 // also tries to maintain a running average of how long the typical frame will
18 // take to encode at the fastest preset (the so-called “complexity”), by dividing
19 // the actual time by the relative time for the preset used.
21 // Frame timings is a complex topic in its own sright, since usually, multiple
22 // frames are encoded in parallel. X264SpeedControl only supports the timing
23 // method that the original patch calls “alternate timing”; one simply measures
24 // the time the last x264_encoder_encode() call took. (The other alternative given
25 // is to measure the time between successive x264_encoder_encode() calls.)
26 // Unless using the zerocopy presets (which activate slice threading), the function
27 // actually returns not when the given frame is done encoding, but when one a few
28 // frames back is done encoding. So it doesn't actually measure the time of any
29 // given one frame, but it measures something correlated to it, at least as long as
30 // you are near 100% CPU utilization (ie., the encoded frame doesn't linger in the
31 // buffers already when x264_encoder_encode() is called).
33 // The code has a long history; it was originally part of Avail Media's x264
34 // branch, used in their encoder appliances, and then a snapshot of that was
35 // released. (Given that x264 is licensed under GPLv2 or newer, this means that
36 // we can also treat the patch as GPLv2 or newer if we want, which we do.
37 // As far as I know, it is copyright Avail Media, although no specific copyright
38 // notice was posted on the patch.)
40 // From there, it was incorporated in OBE's x264 tree (x264-obe) and some bugs
41 // were fixed. I started working on it for the purposes of Nageru, fixing various
42 // issues, adding VFR support and redoing the timings entirely based on more
43 // modern presets (the patch was made before several important x264 features,
44 // such as weighted P-frames). Finally, I took it out of x264 and put it into
45 // Nageru (it does not actually use any hooks into the codec itself), so that
46 // one does not need to patch x264 to use it in Nageru. It still could do with
47 // some cleanup, but it's much, much better than just using a static preset.
58 #include "shared/metrics.h"
59 #include "x264_dynamic.h"
61 class X264SpeedControl {
63 // x264: Encoding object we are using; must be opened. Assumed to be
64 // set to the "faster" preset, and with 16 reference frames.
65 // f_speed: Relative encoding speed, usually 1.0.
66 // i_buffer_size: Number of frames in the buffer.
67 // f_buffer_init: Relative fullness of buffer at start
68 // (0.0 = assumed to be <i_buffer_size> frames in buffer,
69 // 1.0 = no frames in buffer)
70 X264SpeedControl(x264_t *x264, float f_speed, int i_buffer_size, float f_buffer_init);
73 // You need to call before_frame() immediately before each call to
74 // x264_encoder_encode(), and after_frame() immediately after.
76 // new_buffer_fill: Buffer fullness, in microseconds (_not_ a relative
77 // number, unlike f_buffer_init in the constructor).
78 // new_buffer_size: If > 0, new number of frames in the buffer,
79 // ie. the buffer size has changed. (It is harmless to set this
80 // even if the buffer hasn't actually changed.)
81 // f_uspf: If > 0, new microseconds per frame, ie. the frame rate has
82 // changed. (Of course, with VFR, it can be impossible to truly know
83 // the frame rate of the coming frames, but it is a reasonable
84 // assumption that the next second or so is likely to be the same
85 // frame rate as the last frame.)
86 void before_frame(float new_buffer_fill, int new_buffer_size, float f_uspf);
89 // x264 seemingly has an issue where x264_encoder_reconfig() is not reflected
90 // immediately in x264_encoder_parameters(). Since speed control keeps calling
91 // those two all the time, any changes you make outside X264SpeedControl
92 // could be overridden. Thus, to make changes to encoder parameters, you should
93 // instead set a function here, which will be called every time parameters
95 void set_config_override_function(std::function<void(x264_param_t *)> override_func)
97 this->override_func = override_func;
101 void set_buffer_size(int new_buffer_size);
102 int dither_preset(float f);
103 void apply_preset(int new_preset);
112 // all times that are not std::chrono::* are in usec
113 std::chrono::steady_clock::time_point timestamp; // when was speedcontrol last invoked
114 std::chrono::steady_clock::duration cpu_time_last_frame{std::chrono::seconds{0}}; // time spent encoding the previous frame
115 int64_t buffer_size; // assumed application-side buffer of frames to be streamed (measured in microseconds),
116 int64_t buffer_fill; // where full = we don't have to hurry
117 int64_t compensation_period; // how quickly we try to return to the target buffer fullness
118 float uspf; // microseconds per frame
119 int preset = -1; // which setting was used in the previous frame
120 float cplx_num = 3e3; // rolling average of estimated spf for preset #0. FIXME estimate initial complexity
129 int64_t min_buffer, max_buffer;
134 std::function<void(x264_param_t *)> override_func = nullptr;
137 Histogram metric_x264_speedcontrol_preset_used_frames;
138 std::atomic<double> metric_x264_speedcontrol_buffer_available_seconds{0.0};
139 std::atomic<double> metric_x264_speedcontrol_buffer_size_seconds{0.0};
140 std::atomic<int64_t> metric_x264_speedcontrol_idle_frames{0};
141 std::atomic<int64_t> metric_x264_speedcontrol_late_frames{0};
144 #endif // !defined(_X264_SPEED_CONTROL_H)