The original plan here was to let the client hang until we had
some headers to send (ie., first send an empty header and then
0 bytes data, and then send the client back from SENDING_DATA
to SENDING_HEADER as we got data).
However, as time went by, we started inserting stuff in the middle of the
headers ourselves, resulting in us sending pretty much a junk header.
Worse, this would be sent on to other relays, corrupting the version of
the Metacube stream.
Simply return 503 Not Available now instead if the stream is still
starting up; it's pretty much as good, and has fewer edge cases
to worry about.
Update the VLC Metacube patch to apply against current VLC master.
This version is identical to the one that was sent to the VLC mailing list
for upstream inclusion (but not accepted), so it also contains some other
cleanups.
SO_MAX_PACING_RATE is the newfangled socket option from Eric Dumazet,
used with the new fq packet scheduler in Linux. It allows you to set
a max rate for the socket (presumably a stricter upper bound than the
RTT-based estimate from the kernel), delivering pacing without having
to resort to the relatively complex mark setup. It seems to enter
the Linux kernel in 3.13 at the earliest; not unlikely even later.
In time, fwmark will be deprecated, but the implementation of TCP pacing in
Linux is still a bit shaky (especially with not-always-filling applications
like streaming), so fwmark will stay the primary solution for now.
Philipp Kern [Sun, 8 Sep 2013 21:10:55 +0000 (23:10 +0200)]
Makefile: Implement sysconfdir and localstatedir.
It is a bit odd to use $(PREFIX) for some, but not all directories. Make
this a useable middle ground akin autoconf, which allows to pass in
different paths for /var (--localstatedir) and /etc (--sysconfdir).
Seemingly holding queued_data_mutex over add_data_raw(), which does writev(),
could be slow on systems where /tmp is not on tmpfs, causing the queued_data_mutex
to be held for so long (up to a second has been observed) that the input thread
couldn't keep up.
To fix this, we move queued_data_mutex into a per-stream variable (not sure
if it's ideal, but it was the simplest way to avoid ugliness), and then hold it
for as short as possible in process_queued_data().
While we're at it, document that queued_data_last_starting_point has the same
locking rules as queued_data.
Compared to the old version, this fixes many small deficiencies:
- The header is smaller; 16 instead of 24 bytes. With TS, Metacube actually
yields noticeable overhead (13% or so), so reducing it by 1/3 is good.
- The sync no longer overlaps with itself; both starting and ending with
the same sounds bad if we should ever drop bytes in the middle.
In fact, it no longer has any repeated characters.
- The header is checksummed, to avoid cases where a corrupted header
could cause us to pull in gigabytes of data. (The data is not.)
vlc-metacube.diff has been updated, although it includes another patch
(part of the WebM patch set) since that's what my VLC works from at the moment.
Support the new METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START Metacube flag.
This is created by new versions of the VLC Metacube patches;
it is designed for the needs of WebM, where browsers refuse to accept a
stream that does not start with a keyframe. Thus, we can have blocks that
are completely valid (and we wouldn't want to start joining blocks, as the
GOPs can become very large), but that we do not want to start a stream with.
For this, we introduce a new state called WAITING_FOR_KEYFRAME that all
new clients (except those fetching from the start of the backlog) go through;
they stay there until the next keyframe-marked block (anything that's not
marked METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START, so old Metacube streams
are still valid) comes along, at which point they go into the regular
SENDING_DATA.
The new version of the VLC Metacube patch is not included yet.
(Hopefully) fix an assert failure in httpinput.cpp.
If we get the Metacube sync marker but not the right of the header,
we can set has_metacube_header, and then return. If the connection
drops right there, the flag won't be correctly reset, and if the
next connection starts with something that's not a Metacube block,
we will try to read junk data as Metacube.
Never send more than IOV_MAX (1024) iovecs in pwritev.
This fixes issues with “pwritev: Invalid argument” showing up
under rare circumstances. Apparently the glibc wrapper workaround
described in writev(2) does not work, or does not work for pwritev().
Catch up on any lost data before serializing, so increasing the backlog will not inject zero bytes into the data of (too) backlogged clients if we increase the backlog size.
Refer to streams internally mostly by an index, not the stream_id.
Inputs no longer know about stream_id, only the stream_index.
This is faster (we used maybe 5% of our time in find_stream),
and will make it easier to add non-HTTP outputs in the future.
Within (HTTP) clients, stream_id also no longer exists, but is replaced
by "url", which really works exactly the same way. url is mostly used during
re-exec, however, so it's less important than it used to be.
Add Metacube headers in add_data_deferred(), not add_data().
This fixes a problem where Metacube blocks are not deterministically added
for each add_data() call (because they would only be added for each call to
process_queued_data()), causing the different servers to go out of sync.
Also moved most of this logic to Stream, where it seems more appropriate
(even though the mutexes are a bit odd now).
Reinstate the new signal handling; revert the revert.
The fix for high CPU usage was not doing wakeup() all the time for
new data; not only can seemingly sending the signal be expensive,
but we sent it while still holding the mutex the awakening thread
needed, so it was extra-bad.
We're seemingly down to a reasonable amount of CPU usage again now,
but still with good non-racing semantics.
Rewrite the entire internal signal handling/wakeup.
We now solve things in a much less racy way, sending SIGUSR1 internally,
which is blocked except in poll/epoll. This should be more stable,
require no more fixed-length wakeups (no more wakeup every 20 ms in
each server), and upset Valgrind less.
However, it's also complex to get right, so it might introduce new bugs.
projects / cubemap / log