cubemap
7 years agoRevert "Rewrite the entire internal signal handling/wakeup."
Steinar H. Gunderson [Fri, 19 Apr 2013 23:21:15 +0000 (01:21 +0200)]
Revert "Rewrite the entire internal signal handling/wakeup."

Seemingly this had very bad effects on CPU usage. Will need to
investigate later.

This reverts commit 3fd8650ccf3da3960a946d8ac9abc305aec399ce.

7 years agoFix broken linking.
Steinar H. Gunderson [Fri, 19 Apr 2013 22:59:54 +0000 (00:59 +0200)]
Fix broken linking.

7 years agoCompile and link with -pthread, not -lpthread.
Steinar H. Gunderson [Fri, 19 Apr 2013 21:44:23 +0000 (23:44 +0200)]
Compile and link with -pthread, not -lpthread.

7 years agoRewrite the entire internal signal handling/wakeup.
Steinar H. Gunderson [Fri, 19 Apr 2013 21:39:23 +0000 (23:39 +0200)]
Rewrite the entire internal signal handling/wakeup.

We now solve things in a much less racy way, sending SIGUSR1 internally,
which is blocked except in poll/epoll. This should be more stable,
require no more fixed-length wakeups (no more wakeup every 20 ms in
each server), and upset Valgrind less.

However, it's also complex to get right, so it might introduce new bugs.

7 years agoRemove unneeded default Client constructor.
Steinar H. Gunderson [Fri, 19 Apr 2013 19:08:08 +0000 (21:08 +0200)]
Remove unneeded default Client constructor.

7 years agoFix 100% CPU usage when there is no access_log.
Steinar H. Gunderson [Fri, 19 Apr 2013 18:48:35 +0000 (20:48 +0200)]
Fix 100% CPU usage when there is no access_log.

7 years agoDo not crash on re-exec if we do not have an access log.
Steinar H. Gunderson [Fri, 19 Apr 2013 17:57:15 +0000 (19:57 +0200)]
Do not crash on re-exec if we do not have an access log.

7 years agoFix some issues with the last stats.cpp fix.
Steinar H. Gunderson [Fri, 19 Apr 2013 17:40:38 +0000 (19:40 +0200)]
Fix some issues with the last stats.cpp fix.

7 years agoFix more missing log_perror() calls.
Steinar H. Gunderson [Fri, 19 Apr 2013 17:28:22 +0000 (19:28 +0200)]
Fix more missing log_perror() calls.

There's nothing we can do about them, but it's good to report.

Found by Coverity Scan.

7 years agoAdd a missing log_perror() on close() failure (cosmetic issue).
Steinar H. Gunderson [Fri, 19 Apr 2013 17:25:59 +0000 (19:25 +0200)]
Add a missing log_perror() on close() failure (cosmetic issue).

Found by Coverity Scan.

7 years agoSet umask security before calling mkstemp().
Steinar H. Gunderson [Fri, 19 Apr 2013 17:23:24 +0000 (19:23 +0200)]
Set umask security before calling mkstemp().

Found by Coverity Scan.

7 years agoIf make_tempfile() returns -1 in set_backlog_size, exit properly.
Steinar H. Gunderson [Fri, 19 Apr 2013 17:13:45 +0000 (19:13 +0200)]
If make_tempfile() returns -1 in set_backlog_size, exit properly.

Found by Coverity Scan.

7 years agoNew run of include-what-you-use.
Steinar H. Gunderson [Thu, 18 Apr 2013 23:39:53 +0000 (01:39 +0200)]
New run of include-what-you-use.

7 years agoSet Connection: close in outgoing HTTP headers.
Steinar H. Gunderson [Thu, 18 Apr 2013 23:38:36 +0000 (01:38 +0200)]
Set Connection: close in outgoing HTTP headers.

7 years agoWhen logging losses, log loss percentage, too.
Steinar H. Gunderson [Thu, 18 Apr 2013 23:33:33 +0000 (01:33 +0200)]
When logging losses, log loss percentage, too.

7 years agoShow IPv4 clients by their IPv4 address instead of with a v4-mapped address.
Steinar H. Gunderson [Thu, 18 Apr 2013 23:27:24 +0000 (01:27 +0200)]
Show IPv4 clients by their IPv4 address instead of with a v4-mapped address.

7 years agoSend backlog file descriptors around instead of going through the protobuf. Much...
Steinar H. Gunderson [Thu, 18 Apr 2013 22:41:19 +0000 (00:41 +0200)]
Send backlog file descriptors around instead of going through the protobuf. Much faster restart with big backlog sizes, and does not hit the 64MB protobuf limit.

7 years agoFix an issue where new UDP streams would be without HTTP headers.
Steinar H. Gunderson [Thu, 18 Apr 2013 19:28:10 +0000 (21:28 +0200)]
Fix an issue where new UDP streams would be without HTTP headers.

7 years agoDo not bother making a METACUBE_FLAGS_HEADER block if it is empty.
Steinar H. Gunderson [Wed, 17 Apr 2013 23:10:14 +0000 (01:10 +0200)]
Do not bother making a METACUBE_FLAGS_HEADER block if it is empty.

7 years agoWhen closing unused inputs, remember to take down their sockets.
Steinar H. Gunderson [Wed, 17 Apr 2013 23:07:41 +0000 (01:07 +0200)]
When closing unused inputs, remember to take down their sockets.

7 years agoSupport Metacube _output_. Required splitting HTTP headers from stream headers, which...
Steinar H. Gunderson [Wed, 17 Apr 2013 23:02:28 +0000 (01:02 +0200)]
Support Metacube _output_. Required splitting HTTP headers from stream headers, which was a good thing anyway, but a bit painful in upgrades.

7 years agoDeduplicate inputs.
Steinar H. Gunderson [Wed, 17 Apr 2013 21:36:46 +0000 (23:36 +0200)]
Deduplicate inputs.

7 years agoDo not segfault on unknown options.
Steinar H. Gunderson [Wed, 17 Apr 2013 18:14:59 +0000 (20:14 +0200)]
Do not segfault on unknown options.

7 years agoFix a leak in the stats thread.
Steinar H. Gunderson [Tue, 16 Apr 2013 20:39:46 +0000 (22:39 +0200)]
Fix a leak in the stats thread.

7 years agoFix another minor leak.
Steinar H. Gunderson [Tue, 16 Apr 2013 20:38:20 +0000 (22:38 +0200)]
Fix another minor leak.

7 years agoStop leaking the mark pools.
Steinar H. Gunderson [Tue, 16 Apr 2013 20:38:09 +0000 (22:38 +0200)]
Stop leaking the mark pools.

7 years agoFix a memory leak in Server (streams were leaked).
Steinar H. Gunderson [Tue, 16 Apr 2013 20:35:06 +0000 (22:35 +0200)]
Fix a memory leak in Server (streams were leaked).

7 years agoFix a small memory leak in HTTPInput.
Steinar H. Gunderson [Tue, 16 Apr 2013 20:33:11 +0000 (22:33 +0200)]
Fix a small memory leak in HTTPInput.

7 years agoFix a Valgrind hit.
Steinar H. Gunderson [Tue, 16 Apr 2013 20:30:47 +0000 (22:30 +0200)]
Fix a Valgrind hit.

7 years agoFix a Valgrind hit.
Steinar H. Gunderson [Tue, 16 Apr 2013 20:30:30 +0000 (22:30 +0200)]
Fix a Valgrind hit.

7 years agoLog all finished accesses to an access log.
Steinar H. Gunderson [Tue, 16 Apr 2013 20:29:39 +0000 (22:29 +0200)]
Log all finished accesses to an access log.

7 years agoLess confusing error message when realpath() fails.
Steinar H. Gunderson [Tue, 16 Apr 2013 19:47:56 +0000 (21:47 +0200)]
Less confusing error message when realpath() fails.

7 years agoClearer warnings in the cubemap.config.sample file about daemonizing and paths.
Steinar H. Gunderson [Tue, 16 Apr 2013 18:08:06 +0000 (20:08 +0200)]
Clearer warnings in the cubemap.config.sample file about daemonizing and paths.

7 years agoRun --config-test with the canonicalized paths.
Steinar H. Gunderson [Tue, 16 Apr 2013 18:06:51 +0000 (20:06 +0200)]
Run --config-test with the canonicalized paths.

7 years agoSupport daemonizing.
Steinar H. Gunderson [Tue, 16 Apr 2013 18:01:26 +0000 (20:01 +0200)]
Support daemonizing.

7 years agoCanonicalize paths, so that we do not need to worry about cwd on re-exec.
Steinar H. Gunderson [Tue, 16 Apr 2013 17:57:51 +0000 (19:57 +0200)]
Canonicalize paths, so that we do not need to worry about cwd on re-exec.

7 years agoShut down cleanly on SIGINT (Ctrl-C), for easier Valgrind runs.
Steinar H. Gunderson [Mon, 15 Apr 2013 23:31:24 +0000 (01:31 +0200)]
Shut down cleanly on SIGINT (Ctrl-C), for easier Valgrind runs.

7 years agoDrop NO_LEVEL; it does not look very pretty now.
Steinar H. Gunderson [Mon, 15 Apr 2013 23:15:55 +0000 (01:15 +0200)]
Drop NO_LEVEL; it does not look very pretty now.

7 years agoLog when we have a successful HTTPInput request.
Steinar H. Gunderson [Mon, 15 Apr 2013 23:09:20 +0000 (01:09 +0200)]
Log when we have a successful HTTPInput request.

7 years agoIdentify UDPInput error messages by the stream, too.
Steinar H. Gunderson [Mon, 15 Apr 2013 23:08:01 +0000 (01:08 +0200)]
Identify UDPInput error messages by the stream, too.

7 years agoIdentify HTTPInput error messages by the stream.
Steinar H. Gunderson [Mon, 15 Apr 2013 23:07:37 +0000 (01:07 +0200)]
Identify HTTPInput error messages by the stream.

7 years agoMake things line up a bit better.
Steinar H. Gunderson [Mon, 15 Apr 2013 23:04:53 +0000 (01:04 +0200)]
Make things line up a bit better.

7 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Mon, 15 Apr 2013 22:41:10 +0000 (00:41 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

7 years agoCorrect use of strerror_r(); we always need to use the return value.
Steinar H. Gunderson [Mon, 15 Apr 2013 22:41:04 +0000 (00:41 +0200)]
Correct use of strerror_r(); we always need to use the return value.

7 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Mon, 15 Apr 2013 22:39:36 +0000 (00:39 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

7 years agoStore socket and fwmark in the stats file.
Steinar H. Gunderson [Mon, 15 Apr 2013 22:39:17 +0000 (00:39 +0200)]
Store socket and fwmark in the stats file.

7 years agoLog IP address instead of file descriptor.
Steinar H. Gunderson [Mon, 15 Apr 2013 22:37:31 +0000 (00:37 +0200)]
Log IP address instead of file descriptor.

7 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Mon, 15 Apr 2013 22:33:34 +0000 (00:33 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

7 years agoReplace all perror() calls with our own log calls.
Steinar H. Gunderson [Mon, 15 Apr 2013 22:32:52 +0000 (00:32 +0200)]
Replace all perror() calls with our own log calls.

7 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Mon, 15 Apr 2013 21:27:58 +0000 (23:27 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

7 years agoDocument the new logging options in the example config.
Steinar H. Gunderson [Mon, 15 Apr 2013 21:26:56 +0000 (23:26 +0200)]
Document the new logging options in the example config.

7 years agoImplement much better logging, with support for both files and syslog.
Steinar H. Gunderson [Mon, 15 Apr 2013 21:24:38 +0000 (23:24 +0200)]
Implement much better logging, with support for both files and syslog.

7 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Mon, 15 Apr 2013 10:20:37 +0000 (12:20 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

7 years agoFix a signed/unsigned warning.
Steinar H. Gunderson [Mon, 15 Apr 2013 08:37:18 +0000 (10:37 +0200)]
Fix a signed/unsigned warning.

7 years agoStore the client pointer directly in the epoll structure, instead of doing a map...
Steinar H. Gunderson [Mon, 15 Apr 2013 08:36:33 +0000 (10:36 +0200)]
Store the client pointer directly in the epoll structure, instead of doing a map lookup every time. Saves some CPU.

7 years agoSmall microoptimization in do_work().
Steinar H. Gunderson [Sun, 14 Apr 2013 21:41:22 +0000 (23:41 +0200)]
Small microoptimization in do_work().

7 years agoWe now have benchmark results.
Steinar H. Gunderson [Sun, 14 Apr 2013 21:24:33 +0000 (23:24 +0200)]
We now have benchmark results.

7 years agoIgnore SIGPIPE, so we do not die when a client shuts down at the wrong time.
Steinar H. Gunderson [Sun, 14 Apr 2013 18:54:25 +0000 (20:54 +0200)]
Ignore SIGPIPE, so we do not die when a client shuts down at the wrong time.

7 years agoMake read_tempfile() much faster, and make it close the file also on error.
Steinar H. Gunderson [Sun, 14 Apr 2013 18:21:25 +0000 (20:21 +0200)]
Make read_tempfile() much faster, and make it close the file also on error.

7 years agoDo not start the servers until all the deserialized clients have been put back. Fixes...
Steinar H. Gunderson [Sun, 14 Apr 2013 18:10:49 +0000 (20:10 +0200)]
Do not start the servers until all the deserialized clients have been put back. Fixes an assert failure on load, and makes for faster loads.

7 years agoSignal thread stop through a pipe; fixes issues where the statistics thread would...
Steinar H. Gunderson [Sun, 14 Apr 2013 17:53:48 +0000 (19:53 +0200)]
Signal thread stop through a pipe; fixes issues where the statistics thread would wait one round on shutdown.

7 years agoRemove an include IWYU should not have added.
Steinar H. Gunderson [Sun, 14 Apr 2013 16:49:05 +0000 (18:49 +0200)]
Remove an include IWYU should not have added.

7 years agoMove SIGHUP handling as soon as possible, so that repeated SIGHUPing is less likely...
Steinar H. Gunderson [Sun, 14 Apr 2013 16:48:02 +0000 (18:48 +0200)]
Move SIGHUP handling as soon as possible, so that repeated SIGHUPing is less likely to kill us.

7 years agoLog bytes sent and lost better.
Steinar H. Gunderson [Sun, 14 Apr 2013 16:13:19 +0000 (18:13 +0200)]
Log bytes sent and lost better.

7 years agoFix a crash on re-exec if a client was not in SENDING_DATA.
Steinar H. Gunderson [Sun, 14 Apr 2013 16:13:08 +0000 (18:13 +0200)]
Fix a crash on re-exec if a client was not in SENDING_DATA.

7 years agoRename bytes_sent to stream_pos, because that is what it really is.
Steinar H. Gunderson [Sun, 14 Apr 2013 00:30:47 +0000 (02:30 +0200)]
Rename bytes_sent to stream_pos, because that is what it really is.

7 years agoMake more usable documentation.
Steinar H. Gunderson [Sat, 13 Apr 2013 21:03:49 +0000 (23:03 +0200)]
Make more usable documentation.

7 years agoMerge branch 'master' of ssh://git.sesse.net/srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Sat, 13 Apr 2013 20:53:23 +0000 (22:53 +0200)]
Merge branch 'master' of ssh://git.sesse.net/srv/git.sesse.net/www/cubemap

7 years agoWhen HTTPInput disconnects, also clear the header.
Steinar H. Gunderson [Sat, 13 Apr 2013 20:53:02 +0000 (22:53 +0200)]
When HTTPInput disconnects, also clear the header.

7 years agoAnother include-what-you-use pass.
Steinar H. Gunderson [Sat, 13 Apr 2013 20:48:38 +0000 (22:48 +0200)]
Another include-what-you-use pass.

7 years agoMake backlog_size changeable across HUPs.
Steinar H. Gunderson [Sat, 13 Apr 2013 20:47:24 +0000 (22:47 +0200)]
Make backlog_size changeable across HUPs.

7 years agoMove Server:add_data() into Stream, where it more logically belongs.
Steinar H. Gunderson [Sat, 13 Apr 2013 20:35:45 +0000 (22:35 +0200)]
Move Server:add_data() into Stream, where it more logically belongs.

7 years agoRun include-what-you-use.
Steinar H. Gunderson [Sat, 13 Apr 2013 20:13:56 +0000 (22:13 +0200)]
Run include-what-you-use.

7 years agoFix incorrect struct/class in forward declares (found by Clang).
Steinar H. Gunderson [Sat, 13 Apr 2013 20:12:53 +0000 (22:12 +0200)]
Fix incorrect struct/class in forward declares (found by Clang).

7 years agoMove Client and Stream into their own files.
Steinar H. Gunderson [Sat, 13 Apr 2013 20:02:18 +0000 (22:02 +0200)]
Move Client and Stream into their own files.

7 years agoSupport configurable BACKLOG_SIZE (per-stream). No support for changing across restar...
Steinar H. Gunderson [Sat, 13 Apr 2013 19:53:45 +0000 (21:53 +0200)]
Support configurable BACKLOG_SIZE (per-stream). No support for changing across restarts yet.

7 years agoFix a socket leak in HTTPInput.
Steinar H. Gunderson [Sat, 13 Apr 2013 19:50:13 +0000 (21:50 +0200)]
Fix a socket leak in HTTPInput.

7 years agoFix a spurious warning.
Steinar H. Gunderson [Sat, 13 Apr 2013 17:06:48 +0000 (19:06 +0200)]
Fix a spurious warning.

7 years agoSend data using sendfile() instead of write(). Seems to be something like 30% lower...
Steinar H. Gunderson [Sat, 13 Apr 2013 17:04:27 +0000 (19:04 +0200)]
Send data using sendfile() instead of write(). Seems to be something like 30% lower CPU usage.

7 years agoFix a bug introduced with the move of read_tempfile() to util.cpp.
Steinar H. Gunderson [Sat, 13 Apr 2013 17:03:54 +0000 (19:03 +0200)]
Fix a bug introduced with the move of read_tempfile() to util.cpp.

7 years agoMove the “read the whole file” logic into a new file.
Steinar H. Gunderson [Sat, 13 Apr 2013 16:47:58 +0000 (18:47 +0200)]
Move the “read the whole file” logic into a new file.

7 years agoEven more missing unistd.h includes.
Steinar H. Gunderson [Fri, 12 Apr 2013 23:16:09 +0000 (01:16 +0200)]
Even more missing unistd.h includes.

7 years agoEven more missing unistd.h includes.
Steinar H. Gunderson [Fri, 12 Apr 2013 23:15:57 +0000 (01:15 +0200)]
Even more missing unistd.h includes.

7 years agoThread has virtual member functions, so it should have a virtual destructor.
Steinar H. Gunderson [Fri, 12 Apr 2013 23:15:24 +0000 (01:15 +0200)]
Thread has virtual member functions, so it should have a virtual destructor.

7 years agoFix a missing #include.
Steinar H. Gunderson [Fri, 12 Apr 2013 23:14:32 +0000 (01:14 +0200)]
Fix a missing #include.

7 years agoWhen re-execing, try with --test-config first, and revert if it fails.
Steinar H. Gunderson [Fri, 12 Apr 2013 22:24:36 +0000 (00:24 +0200)]
When re-execing, try with --test-config first, and revert if it fails.

7 years agoImplement --test-config.
Steinar H. Gunderson [Fri, 12 Apr 2013 21:07:01 +0000 (23:07 +0200)]
Implement --test-config.

7 years agoSince we just broke upgrade compatibility, kill some older stuff in the state protos.
Steinar H. Gunderson [Fri, 12 Apr 2013 21:04:50 +0000 (23:04 +0200)]
Since we just broke upgrade compatibility, kill some older stuff in the state protos.

7 years agoSwitch option parsing to getopt.
Steinar H. Gunderson [Fri, 12 Apr 2013 21:04:01 +0000 (23:04 +0200)]
Switch option parsing to getopt.

7 years agoMove some serialization logic into ServerPool, where it belongs.
Steinar H. Gunderson [Fri, 12 Apr 2013 20:31:05 +0000 (22:31 +0200)]
Move some serialization logic into ServerPool, where it belongs.

7 years agoSplit configuration parsing out cleanly from initialization.
Steinar H. Gunderson [Fri, 12 Apr 2013 20:09:24 +0000 (22:09 +0200)]
Split configuration parsing out cleanly from initialization.

This mostly is required for -configtest later, but also generally splits things out
in cleaner and nicer ways than before.

7 years agoFix UDPInput serialization.
Steinar H. Gunderson [Fri, 12 Apr 2013 20:08:09 +0000 (22:08 +0200)]
Fix UDPInput serialization.

7 years agoAdd rudimentary README.
Steinar H. Gunderson [Thu, 11 Apr 2013 23:56:12 +0000 (01:56 +0200)]
Add rudimentary README.

7 years agoSplit config parsing out of parse.h.
Steinar H. Gunderson [Thu, 11 Apr 2013 23:52:29 +0000 (01:52 +0200)]
Split config parsing out of parse.h.

7 years agoSupport UDP input. Also fix some issues with socket closing.
Steinar H. Gunderson [Thu, 11 Apr 2013 21:34:03 +0000 (23:34 +0200)]
Support UDP input. Also fix some issues with socket closing.

7 years agoWhen create_input() fails, give an error message instead of crashing.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:46:10 +0000 (21:46 +0200)]
When create_input() fails, give an error message instead of crashing.

7 years agoDo not keep pending data across HTTP connections.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:36:55 +0000 (21:36 +0200)]
Do not keep pending data across HTTP connections.

7 years agoMore support for multiple input types.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:31:11 +0000 (21:31 +0200)]
More support for multiple input types.

7 years agoMove version identification into a common place.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:08:14 +0000 (21:08 +0200)]
Move version identification into a common place.

7 years agoMake Input a bit more generic, to pave the way for UDP.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:06:03 +0000 (21:06 +0200)]
Make Input a bit more generic, to pave the way for UDP.