This would only happen if a HTTP input that wasn't fully setup yet
was no longer in use after an reload. This would normally manifest itself as
a “close(): Bad file descriptor”, but could also end up closing an arbitrary
descriptor, causing various sorts of havoc.
Seemingly open() needs to take a pathname only, not a full filename.
This made us _always_ go into the mkstemp() path, which was of course
not the intention.
This has been fully obsoleted by the fq qdisc, which is easier to set up,
more scalable and does not require root privileges. Removing it also removes
a significant chunk of code, which is good.
The original plan here was to let the client hang until we had
some headers to send (ie., first send an empty header and then
0 bytes data, and then send the client back from SENDING_DATA
to SENDING_HEADER as we got data).
However, as time went by, we started inserting stuff in the middle of the
headers ourselves, resulting in us sending pretty much a junk header.
Worse, this would be sent on to other relays, corrupting the version of
the Metacube stream.
Simply return 503 Not Available now instead if the stream is still
starting up; it's pretty much as good, and has fewer edge cases
to worry about.
Update the VLC Metacube patch to apply against current VLC master.
This version is identical to the one that was sent to the VLC mailing list
for upstream inclusion (but not accepted), so it also contains some other
cleanups.
SO_MAX_PACING_RATE is the newfangled socket option from Eric Dumazet,
used with the new fq packet scheduler in Linux. It allows you to set
a max rate for the socket (presumably a stricter upper bound than the
RTT-based estimate from the kernel), delivering pacing without having
to resort to the relatively complex mark setup. It seems to enter
the Linux kernel in 3.13 at the earliest; not unlikely even later.
In time, fwmark will be deprecated, but the implementation of TCP pacing in
Linux is still a bit shaky (especially with not-always-filling applications
like streaming), so fwmark will stay the primary solution for now.
Philipp Kern [Sun, 8 Sep 2013 21:10:55 +0000 (23:10 +0200)]
Makefile: Implement sysconfdir and localstatedir.
It is a bit odd to use $(PREFIX) for some, but not all directories. Make
this a useable middle ground akin autoconf, which allows to pass in
different paths for /var (--localstatedir) and /etc (--sysconfdir).
Seemingly holding queued_data_mutex over add_data_raw(), which does writev(),
could be slow on systems where /tmp is not on tmpfs, causing the queued_data_mutex
to be held for so long (up to a second has been observed) that the input thread
couldn't keep up.
To fix this, we move queued_data_mutex into a per-stream variable (not sure
if it's ideal, but it was the simplest way to avoid ugliness), and then hold it
for as short as possible in process_queued_data().
While we're at it, document that queued_data_last_starting_point has the same
locking rules as queued_data.
Compared to the old version, this fixes many small deficiencies:
- The header is smaller; 16 instead of 24 bytes. With TS, Metacube actually
yields noticeable overhead (13% or so), so reducing it by 1/3 is good.
- The sync no longer overlaps with itself; both starting and ending with
the same sounds bad if we should ever drop bytes in the middle.
In fact, it no longer has any repeated characters.
- The header is checksummed, to avoid cases where a corrupted header
could cause us to pull in gigabytes of data. (The data is not.)
vlc-metacube.diff has been updated, although it includes another patch
(part of the WebM patch set) since that's what my VLC works from at the moment.
Support the new METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START Metacube flag.
This is created by new versions of the VLC Metacube patches;
it is designed for the needs of WebM, where browsers refuse to accept a
stream that does not start with a keyframe. Thus, we can have blocks that
are completely valid (and we wouldn't want to start joining blocks, as the
GOPs can become very large), but that we do not want to start a stream with.
For this, we introduce a new state called WAITING_FOR_KEYFRAME that all
new clients (except those fetching from the start of the backlog) go through;
they stay there until the next keyframe-marked block (anything that's not
marked METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START, so old Metacube streams
are still valid) comes along, at which point they go into the regular
SENDING_DATA.
The new version of the VLC Metacube patch is not included yet.
(Hopefully) fix an assert failure in httpinput.cpp.
If we get the Metacube sync marker but not the right of the header,
we can set has_metacube_header, and then return. If the connection
drops right there, the flag won't be correctly reset, and if the
next connection starts with something that's not a Metacube block,
we will try to read junk data as Metacube.
projects / cubemap / log