]> git.sesse.net Git - cubemap/log
cubemap
10 years agoRemove the Metacube VLC patch; it is now upstream.
Steinar H. Gunderson [Sat, 15 Mar 2014 01:08:04 +0000 (02:08 +0100)]
Remove the Metacube VLC patch; it is now upstream.

10 years agoCompile with large file support, especially since pwritev() on 32-bit x86 seems to...
Steinar H. Gunderson [Thu, 6 Feb 2014 23:42:40 +0000 (00:42 +0100)]
Compile with large file support, especially since pwritev() on 32-bit x86 seems to be broken without.

10 years agoBump version number to 1.0.4-pre.
Steinar H. Gunderson [Thu, 6 Feb 2014 21:40:07 +0000 (22:40 +0100)]
Bump version number to 1.0.4-pre.

10 years agoRelease Cubemap 1.0.3. 1.0.3
Steinar H. Gunderson [Thu, 6 Feb 2014 21:37:28 +0000 (22:37 +0100)]
Release Cubemap 1.0.3.

10 years agoFix compilation error with newer glibc; broken after IWYU. origin/HEAD origin/master
Steinar H. Gunderson [Thu, 6 Feb 2014 21:35:04 +0000 (22:35 +0100)]
Fix compilation error with newer glibc; broken after IWYU.

10 years agoBump version number to 1.0.3-pre.
Steinar H. Gunderson [Mon, 3 Feb 2014 23:47:49 +0000 (00:47 +0100)]
Bump version number to 1.0.3-pre.

10 years agoRelease Cubemap 1.0.2. 1.0.2
Steinar H. Gunderson [Mon, 3 Feb 2014 23:39:57 +0000 (00:39 +0100)]
Release Cubemap 1.0.2.

10 years agoThe SO_MAX_PACING_RATE patches are now released in 3.13.
Steinar H. Gunderson [Mon, 3 Feb 2014 23:37:49 +0000 (00:37 +0100)]
The SO_MAX_PACING_RATE patches are now released in 3.13.

10 years agoFix handling of streams with no data.
Steinar H. Gunderson [Sun, 2 Feb 2014 23:25:41 +0000 (00:25 +0100)]
Fix handling of streams with no data.

The original plan here was to let the client hang until we had
some headers to send (ie., first send an empty header and then
0 bytes data, and then send the client back from SENDING_DATA
to SENDING_HEADER as we got data).

However, as time went by, we started inserting stuff in the middle of the
headers ourselves, resulting in us sending pretty much a junk header.
Worse, this would be sent on to other relays, corrupting the version of
the Metacube stream.

Simply return 503 Not Available now instead if the stream is still
starting up; it's pretty much as good, and has fewer edge cases
to worry about.

10 years agoFix a crash bug on reload.
Steinar H. Gunderson [Sun, 2 Feb 2014 21:36:43 +0000 (22:36 +0100)]
Fix a crash bug on reload.

This could happen in the case where we reloaded while still waiting for
the request from a given client.

10 years agoRemove some legacy from older versions that nobody uses anymore.
Steinar H. Gunderson [Sat, 1 Feb 2014 21:58:11 +0000 (22:58 +0100)]
Remove some legacy from older versions that nobody uses anymore.

10 years agoMinor deduplication.
Steinar H. Gunderson [Sun, 1 Dec 2013 01:41:15 +0000 (02:41 +0100)]
Minor deduplication.

10 years agoFix some duplicated IP address parsing code.
Steinar H. Gunderson [Sun, 1 Dec 2013 00:27:50 +0000 (01:27 +0100)]
Fix some duplicated IP address parsing code.

10 years agoMove some config stuff into an anonymous namespace.
Steinar H. Gunderson [Sun, 1 Dec 2013 00:21:21 +0000 (01:21 +0100)]
Move some config stuff into an anonymous namespace.

10 years agoFix some signed/unsigned comparison warnings (pacing_rate should be uint32_t everywhe...
Steinar H. Gunderson [Sun, 1 Dec 2013 00:18:41 +0000 (01:18 +0100)]
Fix some signed/unsigned comparison warnings (pacing_rate should be uint32_t everywhere, not just in UDPStream).

10 years agoAdd a listen statement to listen on only specific IP addresses, in addition to the...
Steinar H. Gunderson [Sun, 1 Dec 2013 00:16:41 +0000 (01:16 +0100)]
Add a listen statement to listen on only specific IP addresses, in addition to the port statement.

11 years agoWhen checking to see if we want to warn about pacing rate, use the correct default...
Steinar H. Gunderson [Thu, 17 Oct 2013 12:43:34 +0000 (14:43 +0200)]
When checking to see if we want to warn about pacing rate, use the correct default (~0U, not 0).

11 years agoBump version number to 1.0.2-pre.
Steinar H. Gunderson [Tue, 1 Oct 2013 22:19:59 +0000 (00:19 +0200)]
Bump version number to 1.0.2-pre.

11 years agoUpdate the VLC Metacube patch to apply against current VLC master.
Steinar H. Gunderson [Tue, 1 Oct 2013 22:19:10 +0000 (00:19 +0200)]
Update the VLC Metacube patch to apply against current VLC master.

This version is identical to the one that was sent to the VLC mailing list
for upstream inclusion (but not accepted), so it also contains some other
cleanups.

11 years agoSupport SO_MAX_PACING_RATE.
Steinar H. Gunderson [Sat, 28 Sep 2013 22:37:43 +0000 (00:37 +0200)]
Support SO_MAX_PACING_RATE.

SO_MAX_PACING_RATE is the newfangled socket option from Eric Dumazet,
used with the new fq packet scheduler in Linux. It allows you to set
a max rate for the socket (presumably a stricter upper bound than the
RTT-based estimate from the kernel), delivering pacing without having
to resort to the relatively complex mark setup. It seems to enter
the Linux kernel in 3.13 at the earliest; not unlikely even later.

In time, fwmark will be deprecated, but the implementation of TCP pacing in
Linux is still a bit shaky (especially with not-always-filling applications
like streaming), so fwmark will stay the primary solution for now.

11 years agoRelease Cubemap 1.0.1. 1.0.1
Steinar H. Gunderson [Thu, 19 Sep 2013 17:41:42 +0000 (19:41 +0200)]
Release Cubemap 1.0.1.

11 years agoAdd NEWS file for upcoming release.
Steinar H. Gunderson [Thu, 19 Sep 2013 17:41:33 +0000 (19:41 +0200)]
Add NEWS file for upcoming release.

11 years agoFix compilation on 32-bit systems.
Steinar H. Gunderson [Thu, 19 Sep 2013 09:34:54 +0000 (11:34 +0200)]
Fix compilation on 32-bit systems.

11 years agoFix capitalization in systemd service unit.
Steinar H. Gunderson [Tue, 10 Sep 2013 20:45:34 +0000 (22:45 +0200)]
Fix capitalization in systemd service unit.

11 years agoUpdate the man page with our non-alphaness as well.
Steinar H. Gunderson [Sun, 8 Sep 2013 22:27:41 +0000 (00:27 +0200)]
Update the man page with our non-alphaness as well.

11 years agoUpdate README; we are not in alpha stage anymore.
Steinar H. Gunderson [Sun, 8 Sep 2013 22:23:02 +0000 (00:23 +0200)]
Update README; we are not in alpha stage anymore.

11 years agoAdd systemd service unit.
Philipp Kern [Sun, 8 Sep 2013 21:10:56 +0000 (23:10 +0200)]
Add systemd service unit.

11 years agoMakefile: Implement sysconfdir and localstatedir.
Philipp Kern [Sun, 8 Sep 2013 21:10:55 +0000 (23:10 +0200)]
Makefile: Implement sysconfdir and localstatedir.

It is a bit odd to use $(PREFIX) for some, but not all directories. Make
this a useable middle ground akin autoconf, which allows to pass in
different paths for /var (--localstatedir) and /etc (--sysconfdir).

11 years agoUpdate version number to 1.0.1-pre.
Steinar H. Gunderson [Sun, 8 Sep 2013 16:48:52 +0000 (18:48 +0200)]
Update version number to 1.0.1-pre.

11 years agoUse the new-in-3.11 O_TMPFILE flag when available.
Steinar H. Gunderson [Sun, 8 Sep 2013 16:47:41 +0000 (18:47 +0200)]
Use the new-in-3.11 O_TMPFILE flag when available.

Also, we don't need to mess around with umask in the fallback case,
since 0600 is default for mkstemp() anyway.

11 years agoReduce contention of queued_data_mutex.
Steinar H. Gunderson [Sun, 8 Sep 2013 16:18:14 +0000 (18:18 +0200)]
Reduce contention of queued_data_mutex.

Seemingly holding queued_data_mutex over add_data_raw(), which does writev(),
could be slow on systems where /tmp is not on tmpfs, causing the queued_data_mutex
to be held for so long (up to a second has been observed) that the input thread
couldn't keep up.

To fix this, we move queued_data_mutex into a per-stream variable (not sure
if it's ideal, but it was the simplest way to avoid ugliness), and then hold it
for as short as possible in process_queued_data().

While we're at it, document that queued_data_last_starting_point has the same
locking rules as queued_data.

11 years agoRelease Cubemap 1.0.0. 1.0.0
Steinar H. Gunderson [Sat, 24 Aug 2013 15:55:21 +0000 (17:55 +0200)]
Release Cubemap 1.0.0.

11 years agoInclude a Munin plugin that was missing in the archive.
Steinar H. Gunderson [Sat, 24 Aug 2013 15:58:02 +0000 (17:58 +0200)]
Include a Munin plugin that was missing in the archive.

11 years agoFix an issue where rewind-to-HTTP-header would not properly work.
Steinar H. Gunderson [Sun, 18 Aug 2013 11:46:22 +0000 (13:46 +0200)]
Fix an issue where rewind-to-HTTP-header would not properly work.

11 years agoRe-run include-what-you-use.
Steinar H. Gunderson [Sun, 18 Aug 2013 11:03:40 +0000 (13:03 +0200)]
Re-run include-what-you-use.

11 years agoMunin plugins should be symlinked, not created directly in /etc.
Steinar H. Gunderson [Sun, 18 Aug 2013 09:53:20 +0000 (11:53 +0200)]
Munin plugins should be symlinked, not created directly in /etc.

11 years agoMake the install target install the Munin plugins.
Steinar H. Gunderson [Sun, 18 Aug 2013 09:52:21 +0000 (11:52 +0200)]
Make the install target install the Munin plugins.

11 years agoTweak the MutexLock implementation slightly, so as to confuse Coverity less.
Steinar H. Gunderson [Sun, 18 Aug 2013 08:55:43 +0000 (10:55 +0200)]
Tweak the MutexLock implementation slightly, so as to confuse Coverity less.

11 years agoCompute the checksum on outgoing Metacube packets as well.
Steinar H. Gunderson [Sun, 18 Aug 2013 00:54:54 +0000 (02:54 +0200)]
Compute the checksum on outgoing Metacube packets as well.

11 years agoUse memcpy instead of incurring potential unaligned reads when reading the Metacube...
Steinar H. Gunderson [Sun, 18 Aug 2013 00:48:13 +0000 (02:48 +0200)]
Use memcpy instead of incurring potential unaligned reads when reading the Metacube header.

This would probably be a portability problem for many other platforms than x86.

11 years agoSwitch to a new version of the Metacube protocol.
Steinar H. Gunderson [Sun, 18 Aug 2013 00:37:16 +0000 (02:37 +0200)]
Switch to a new version of the Metacube protocol.

Compared to the old version, this fixes many small deficiencies:

 - The header is smaller; 16 instead of 24 bytes. With TS, Metacube actually
   yields noticeable overhead (13% or so), so reducing it by 1/3 is good.
 - The sync no longer overlaps with itself; both starting and ending with
   the same sounds bad if we should ever drop bytes in the middle.
   In fact, it no longer has any repeated characters.
 - The header is checksummed, to avoid cases where a corrupted header
   could cause us to pull in gigabytes of data. (The data is not.)

vlc-metacube.diff has been updated, although it includes another patch
(part of the WebM patch set) since that's what my VLC works from at the moment.

11 years agoMake the HTTP inputs time out after 30 seconds of no activity.
Steinar H. Gunderson [Sat, 17 Aug 2013 13:09:03 +0000 (15:09 +0200)]
Make the HTTP inputs time out after 30 seconds of no activity.

This is useful for the case where the remote machine is gone,
but TCP doesn't pick it up, and thus you can't easily restart
the streaming.

11 years agoCorrect my e-mail address in the man page (cubemap is not written by my Debian hat).
Steinar H. Gunderson [Sat, 17 Aug 2013 09:01:21 +0000 (11:01 +0200)]
Correct my e-mail address in the man page (cubemap is not written by my Debian hat).

11 years agoAdd a manual page for cubemap.
Philipp Kern [Fri, 16 Aug 2013 22:14:24 +0000 (00:14 +0200)]
Add a manual page for cubemap.

11 years agoStore the stream header in the inputs, not just the streams.
Steinar H. Gunderson [Fri, 16 Aug 2013 18:33:57 +0000 (20:33 +0200)]
Store the stream header in the inputs, not just the streams.

This avoids an issue where a stream would get no stream header
when it reused an existing input.

11 years agoWhen adding new streams that are copies of old streams, copy the HTTP header.
Steinar H. Gunderson [Fri, 16 Aug 2013 15:18:56 +0000 (17:18 +0200)]
When adding new streams that are copies of old streams, copy the HTTP header.

This was already fixed for UDP inputs, but not HTTP inputs. Also, we probably
do not do the right thing here for the stream header.

11 years agoMake an option to properly delete streams.
Steinar H. Gunderson [Fri, 16 Aug 2013 17:11:03 +0000 (19:11 +0200)]
Make an option to properly delete streams.

11 years agoDo not crash on invalid src= URLs.
Steinar H. Gunderson [Fri, 16 Aug 2013 15:43:53 +0000 (17:43 +0200)]
Do not crash on invalid src= URLs.

11 years agoDefault PREFIX should be /usr/local, not /usr.
Steinar H. Gunderson [Fri, 16 Aug 2013 12:14:29 +0000 (14:14 +0200)]
Default PREFIX should be /usr/local, not /usr.

11 years agoMakefile: accept LDFLAGS for linking
Philipp Kern [Fri, 16 Aug 2013 09:57:17 +0000 (11:57 +0200)]
Makefile: accept LDFLAGS for linking

11 years agoFix the vertical label of the cubemap_input Munin plugin.
Steinar H. Gunderson [Fri, 16 Aug 2013 11:39:39 +0000 (13:39 +0200)]
Fix the vertical label of the cubemap_input Munin plugin.

11 years agoProvide an install target in the Makefile.
Steinar H. Gunderson [Fri, 16 Aug 2013 09:44:51 +0000 (11:44 +0200)]
Provide an install target in the Makefile.

Based on a patch from Philipp Kern.

11 years agoAdded some Munin plugins.
Steinar H. Gunderson [Fri, 16 Aug 2013 09:29:54 +0000 (11:29 +0200)]
Added some Munin plugins.

11 years agoStore and log connection time for inputs.
Steinar H. Gunderson [Fri, 16 Aug 2013 00:09:34 +0000 (02:09 +0200)]
Store and log connection time for inputs.

11 years agoTwo minor cleanups in the stats code.
Steinar H. Gunderson [Fri, 16 Aug 2013 00:07:19 +0000 (02:07 +0200)]
Two minor cleanups in the stats code.

11 years agoAdd preliminary support for input stream statistics.
Steinar H. Gunderson [Thu, 15 Aug 2013 23:37:43 +0000 (01:37 +0200)]
Add preliminary support for input stream statistics.

11 years agoFix broken umask (we had the bits inverted).
Steinar H. Gunderson [Thu, 15 Aug 2013 17:40:08 +0000 (19:40 +0200)]
Fix broken umask (we had the bits inverted).

11 years agoMake the Metacube block size warning a bit less aggressive, as WebM seems to generate...
Steinar H. Gunderson [Wed, 14 Aug 2013 23:02:06 +0000 (01:02 +0200)]
Make the Metacube block size warning a bit less aggressive, as WebM seems to generate blocks of up to 70-80 kB sometimes.

11 years agoSupport the new METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START Metacube flag.
Steinar H. Gunderson [Wed, 14 Aug 2013 21:09:52 +0000 (23:09 +0200)]
Support the new METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START Metacube flag.

This is created by new versions of the VLC Metacube patches;
it is designed for the needs of WebM, where browsers refuse to accept a
stream that does not start with a keyframe. Thus, we can have blocks that
are completely valid (and we wouldn't want to start joining blocks, as the
GOPs can become very large), but that we do not want to start a stream with.

For this, we introduce a new state called WAITING_FOR_KEYFRAME that all
new clients (except those fetching from the start of the backlog) go through;
they stay there until the next keyframe-marked block (anything that's not
marked METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START, so old Metacube streams
are still valid) comes along, at which point they go into the regular
SENDING_DATA.

The new version of the VLC Metacube patch is not included yet.

11 years agoWhen warning about unusually long Metacube blocks, use decimal instead of hex.
Steinar H. Gunderson [Wed, 14 Aug 2013 19:34:24 +0000 (21:34 +0200)]
When warning about unusually long Metacube blocks, use decimal instead of hex.

11 years agoWhen reading the HTTP input, skip the \r\n\r\n. Fixes the problem of 4 non-Metacube...
Steinar H. Gunderson [Tue, 13 Aug 2013 14:08:01 +0000 (16:08 +0200)]
When reading the HTTP input, skip the \r\n\r\n. Fixes the problem of 4 non-Metacube bytes being skipped.

11 years agoWarn on large Metacube blocks.
Steinar H. Gunderson [Tue, 13 Aug 2013 09:42:41 +0000 (11:42 +0200)]
Warn on large Metacube blocks.

11 years agoAdd an assert, to hopefully catch a bug.
Steinar H. Gunderson [Tue, 13 Aug 2013 09:13:38 +0000 (11:13 +0200)]
Add an assert, to hopefully catch a bug.

11 years agoSet num_servers to 1 in the sample config, as that is the more typical situation.
Steinar H. Gunderson [Mon, 12 Aug 2013 21:37:17 +0000 (23:37 +0200)]
Set num_servers to 1 in the sample config, as that is the more typical situation.

11 years ago(Hopefully) fix an assert failure in httpinput.cpp.
Steinar H. Gunderson [Thu, 6 Jun 2013 00:21:46 +0000 (02:21 +0200)]
(Hopefully) fix an assert failure in httpinput.cpp.

If we get the Metacube sync marker but not the right of the header,
we can set has_metacube_header, and then return. If the connection
drops right there, the flag won't be correctly reset, and if the
next connection starts with something that's not a Metacube block,
we will try to read junk data as Metacube.

Thankfully, the assert caught this case.

11 years agoFix a possible corruption/crash when name lookups fail, due to incorrect error checki...
Steinar H. Gunderson [Mon, 20 May 2013 13:47:41 +0000 (15:47 +0200)]
Fix a possible corruption/crash when name lookups fail, due to incorrect error checking of the getaddrinfo() result.

11 years agoFix a possible infinite loop in HTTPInput.
Steinar H. Gunderson [Wed, 15 May 2013 20:26:31 +0000 (22:26 +0200)]
Fix a possible infinite loop in HTTPInput.

11 years agoFix a potential double-free.
Steinar H. Gunderson [Wed, 15 May 2013 20:24:54 +0000 (22:24 +0200)]
Fix a potential double-free.

11 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Wed, 8 May 2013 18:45:46 +0000 (20:45 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

11 years agoUnbreak access logging.
Steinar H. Gunderson [Wed, 8 May 2013 18:45:27 +0000 (20:45 +0200)]
Unbreak access logging.

We forgot to wake up the logger thread. 

11 years agoAdd a hack for starting at the _start_ of backlog. Useful for fetching samples of...
Steinar H. Gunderson [Sun, 28 Apr 2013 16:48:24 +0000 (18:48 +0200)]
Add a hack for starting at the _start_ of backlog. Useful for fetching samples of streams crashing VLC after-the-back.

11 years agoNever send more than IOV_MAX (1024) iovecs in pwritev.
Steinar H. Gunderson [Sat, 27 Apr 2013 11:54:52 +0000 (13:54 +0200)]
Never send more than IOV_MAX (1024) iovecs in pwritev.

This fixes issues with “pwritev: Invalid argument” showing up
under rare circumstances. Apparently the glibc wrapper workaround
described in writev(2) does not work, or does not work for pwritev().

11 years agoFix a minor error message.
Steinar H. Gunderson [Wed, 24 Apr 2013 22:18:53 +0000 (00:18 +0200)]
Fix a minor error message.

11 years agoEnable TCP_CORK if available.
Steinar H. Gunderson [Mon, 22 Apr 2013 21:24:59 +0000 (23:24 +0200)]
Enable TCP_CORK if available.

11 years agoSupport UDP packets larger than 4 kB.
Steinar H. Gunderson [Sun, 21 Apr 2013 22:52:44 +0000 (00:52 +0200)]
Support UDP packets larger than 4 kB.

11 years agoTimestamp the error log.
Steinar H. Gunderson [Sun, 21 Apr 2013 19:48:00 +0000 (21:48 +0200)]
Timestamp the error log.

11 years agoCatch up on any lost data before serializing, so increasing the backlog will not...
Steinar H. Gunderson [Sun, 21 Apr 2013 19:12:39 +0000 (21:12 +0200)]
Catch up on any lost data before serializing, so increasing the backlog will not inject zero bytes into the data of (too) backlogged clients if we increase the backlog size.

11 years agoBump version number to 0.2; it's about time.
Steinar H. Gunderson [Sun, 21 Apr 2013 14:54:52 +0000 (16:54 +0200)]
Bump version number to 0.2; it's about time.

11 years agoSmall error message fix.
Steinar H. Gunderson [Sun, 21 Apr 2013 12:00:27 +0000 (14:00 +0200)]
Small error message fix.

11 years agoRun include-what-you-use.
Steinar H. Gunderson [Sun, 21 Apr 2013 11:57:00 +0000 (13:57 +0200)]
Run include-what-you-use.

11 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Sun, 21 Apr 2013 11:56:03 +0000 (13:56 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

11 years agoAdd support for UDP outputs.
Steinar H. Gunderson [Sun, 21 Apr 2013 11:55:23 +0000 (13:55 +0200)]
Add support for UDP outputs.

11 years agoFix a Valgrind hit that could cause an undefined set of signals to be blocked.
Steinar H. Gunderson [Sun, 21 Apr 2013 10:48:27 +0000 (12:48 +0200)]
Fix a Valgrind hit that could cause an undefined set of signals to be blocked.

11 years agoMinor cleanup.
Steinar H. Gunderson [Sun, 21 Apr 2013 10:31:01 +0000 (12:31 +0200)]
Minor cleanup.

11 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Sun, 21 Apr 2013 00:26:45 +0000 (02:26 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

11 years agoFix a (harmless) close() warning that would come if we deleted a HTTP input that...
Steinar H. Gunderson [Sun, 21 Apr 2013 00:26:35 +0000 (02:26 +0200)]
Fix a (harmless) close() warning that would come if we deleted a HTTP input that was not connected.

11 years agoRun include-what-you-use.
Steinar H. Gunderson [Sat, 20 Apr 2013 23:55:57 +0000 (01:55 +0200)]
Run include-what-you-use.

11 years agoRefer to streams internally mostly by an index, not the stream_id.
Steinar H. Gunderson [Sat, 20 Apr 2013 23:51:52 +0000 (01:51 +0200)]
Refer to streams internally mostly by an index, not the stream_id.

Inputs no longer know about stream_id, only the stream_index.
This is faster (we used maybe 5% of our time in find_stream),
and will make it easier to add non-HTTP outputs in the future.

Within (HTTP) clients, stream_id also no longer exists, but is replaced
by "url", which really works exactly the same way. url is mostly used during
re-exec, however, so it's less important than it used to be.

11 years agoFix merge snafu in last commit. (Oops.)
Steinar H. Gunderson [Sat, 20 Apr 2013 14:41:18 +0000 (16:41 +0200)]
Fix merge snafu in last commit. (Oops.)

11 years agoSmall optimization to Metacube output.
Steinar H. Gunderson [Sat, 20 Apr 2013 14:39:20 +0000 (16:39 +0200)]
Small optimization to Metacube output.

11 years agoMove iovecs around instead of having single data buffers. Hopefully a tad more effici...
Steinar H. Gunderson [Sat, 20 Apr 2013 14:30:16 +0000 (16:30 +0200)]
Move iovecs around instead of having single data buffers. Hopefully a tad more efficient, although it does mean somewhat more malloc-ing.

11 years agoAdd Metacube headers in add_data_deferred(), not add_data().
Steinar H. Gunderson [Sat, 20 Apr 2013 13:56:57 +0000 (15:56 +0200)]
Add Metacube headers in add_data_deferred(), not add_data().

This fixes a problem where Metacube blocks are not deterministically added
for each add_data() call (because they would only be added for each call to
process_queued_data()), causing the different servers to go out of sync.

Also moved most of this logic to Stream, where it seems more appropriate
(even though the mutexes are a bit odd now).

11 years agoWhen unwrapping in set_backlog_size, do not add a spurious Metacube header.
Steinar H. Gunderson [Sat, 20 Apr 2013 12:25:24 +0000 (14:25 +0200)]
When unwrapping in set_backlog_size, do not add a spurious Metacube header.

11 years agoRevert "Fix a bug where Metacube streams would become inconsistent between different...
Steinar H. Gunderson [Sat, 20 Apr 2013 12:12:41 +0000 (14:12 +0200)]
Revert "Fix a bug where Metacube streams would become inconsistent between different servers, since the first one did not get the right encoding set."

Doesn't actually fix the issue.

This reverts commit 5605c88a27fcf64ea8f6fd6cbb52dada73856ee0.

11 years agoFix a bug where Metacube streams would become inconsistent between different servers...
Steinar H. Gunderson [Sat, 20 Apr 2013 12:00:02 +0000 (14:00 +0200)]
Fix a bug where Metacube streams would become inconsistent between different servers, since the first one did not get the right encoding set.

11 years agoReinstate the new signal handling; revert the revert.
Steinar H. Gunderson [Sat, 20 Apr 2013 00:19:22 +0000 (02:19 +0200)]
Reinstate the new signal handling; revert the revert.

The fix for high CPU usage was not doing wakeup() all the time for
new data; not only can seemingly sending the signal be expensive,
but we sent it while still holding the mutex the awakening thread
needed, so it was extra-bad.

We're seemingly down to a reasonable amount of CPU usage again now,
but still with good non-racing semantics.

11 years agoRevert "Rewrite the entire internal signal handling/wakeup."
Steinar H. Gunderson [Fri, 19 Apr 2013 23:21:15 +0000 (01:21 +0200)]
Revert "Rewrite the entire internal signal handling/wakeup."

Seemingly this had very bad effects on CPU usage. Will need to
investigate later.

This reverts commit 3fd8650ccf3da3960a946d8ac9abc305aec399ce.

11 years agoFix broken linking.
Steinar H. Gunderson [Fri, 19 Apr 2013 22:59:54 +0000 (00:59 +0200)]
Fix broken linking.

11 years agoCompile and link with -pthread, not -lpthread.
Steinar H. Gunderson [Fri, 19 Apr 2013 21:44:23 +0000 (23:44 +0200)]
Compile and link with -pthread, not -lpthread.

11 years agoRewrite the entire internal signal handling/wakeup.
Steinar H. Gunderson [Fri, 19 Apr 2013 21:39:23 +0000 (23:39 +0200)]
Rewrite the entire internal signal handling/wakeup.

We now solve things in a much less racy way, sending SIGUSR1 internally,
which is blocked except in poll/epoll. This should be more stable,
require no more fixed-length wakeups (no more wakeup every 20 ms in
each server), and upset Valgrind less.

However, it's also complex to get right, so it might introduce new bugs.