cubemap
8 years agoDrop NO_LEVEL; it does not look very pretty now.
Steinar H. Gunderson [Mon, 15 Apr 2013 23:15:55 +0000 (01:15 +0200)]
Drop NO_LEVEL; it does not look very pretty now.

8 years agoLog when we have a successful HTTPInput request.
Steinar H. Gunderson [Mon, 15 Apr 2013 23:09:20 +0000 (01:09 +0200)]
Log when we have a successful HTTPInput request.

8 years agoIdentify UDPInput error messages by the stream, too.
Steinar H. Gunderson [Mon, 15 Apr 2013 23:08:01 +0000 (01:08 +0200)]
Identify UDPInput error messages by the stream, too.

8 years agoIdentify HTTPInput error messages by the stream.
Steinar H. Gunderson [Mon, 15 Apr 2013 23:07:37 +0000 (01:07 +0200)]
Identify HTTPInput error messages by the stream.

8 years agoMake things line up a bit better.
Steinar H. Gunderson [Mon, 15 Apr 2013 23:04:53 +0000 (01:04 +0200)]
Make things line up a bit better.

8 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Mon, 15 Apr 2013 22:41:10 +0000 (00:41 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

8 years agoCorrect use of strerror_r(); we always need to use the return value.
Steinar H. Gunderson [Mon, 15 Apr 2013 22:41:04 +0000 (00:41 +0200)]
Correct use of strerror_r(); we always need to use the return value.

8 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Mon, 15 Apr 2013 22:39:36 +0000 (00:39 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

8 years agoStore socket and fwmark in the stats file.
Steinar H. Gunderson [Mon, 15 Apr 2013 22:39:17 +0000 (00:39 +0200)]
Store socket and fwmark in the stats file.

8 years agoLog IP address instead of file descriptor.
Steinar H. Gunderson [Mon, 15 Apr 2013 22:37:31 +0000 (00:37 +0200)]
Log IP address instead of file descriptor.

8 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Mon, 15 Apr 2013 22:33:34 +0000 (00:33 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

8 years agoReplace all perror() calls with our own log calls.
Steinar H. Gunderson [Mon, 15 Apr 2013 22:32:52 +0000 (00:32 +0200)]
Replace all perror() calls with our own log calls.

8 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Mon, 15 Apr 2013 21:27:58 +0000 (23:27 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

8 years agoDocument the new logging options in the example config.
Steinar H. Gunderson [Mon, 15 Apr 2013 21:26:56 +0000 (23:26 +0200)]
Document the new logging options in the example config.

8 years agoImplement much better logging, with support for both files and syslog.
Steinar H. Gunderson [Mon, 15 Apr 2013 21:24:38 +0000 (23:24 +0200)]
Implement much better logging, with support for both files and syslog.

8 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Mon, 15 Apr 2013 10:20:37 +0000 (12:20 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

8 years agoFix a signed/unsigned warning.
Steinar H. Gunderson [Mon, 15 Apr 2013 08:37:18 +0000 (10:37 +0200)]
Fix a signed/unsigned warning.

8 years agoStore the client pointer directly in the epoll structure, instead of doing a map...
Steinar H. Gunderson [Mon, 15 Apr 2013 08:36:33 +0000 (10:36 +0200)]
Store the client pointer directly in the epoll structure, instead of doing a map lookup every time. Saves some CPU.

8 years agoSmall microoptimization in do_work().
Steinar H. Gunderson [Sun, 14 Apr 2013 21:41:22 +0000 (23:41 +0200)]
Small microoptimization in do_work().

8 years agoWe now have benchmark results.
Steinar H. Gunderson [Sun, 14 Apr 2013 21:24:33 +0000 (23:24 +0200)]
We now have benchmark results.

8 years agoIgnore SIGPIPE, so we do not die when a client shuts down at the wrong time.
Steinar H. Gunderson [Sun, 14 Apr 2013 18:54:25 +0000 (20:54 +0200)]
Ignore SIGPIPE, so we do not die when a client shuts down at the wrong time.

8 years agoMake read_tempfile() much faster, and make it close the file also on error.
Steinar H. Gunderson [Sun, 14 Apr 2013 18:21:25 +0000 (20:21 +0200)]
Make read_tempfile() much faster, and make it close the file also on error.

8 years agoDo not start the servers until all the deserialized clients have been put back. Fixes...
Steinar H. Gunderson [Sun, 14 Apr 2013 18:10:49 +0000 (20:10 +0200)]
Do not start the servers until all the deserialized clients have been put back. Fixes an assert failure on load, and makes for faster loads.

8 years agoSignal thread stop through a pipe; fixes issues where the statistics thread would...
Steinar H. Gunderson [Sun, 14 Apr 2013 17:53:48 +0000 (19:53 +0200)]
Signal thread stop through a pipe; fixes issues where the statistics thread would wait one round on shutdown.

8 years agoRemove an include IWYU should not have added.
Steinar H. Gunderson [Sun, 14 Apr 2013 16:49:05 +0000 (18:49 +0200)]
Remove an include IWYU should not have added.

8 years agoMove SIGHUP handling as soon as possible, so that repeated SIGHUPing is less likely...
Steinar H. Gunderson [Sun, 14 Apr 2013 16:48:02 +0000 (18:48 +0200)]
Move SIGHUP handling as soon as possible, so that repeated SIGHUPing is less likely to kill us.

8 years agoLog bytes sent and lost better.
Steinar H. Gunderson [Sun, 14 Apr 2013 16:13:19 +0000 (18:13 +0200)]
Log bytes sent and lost better.

8 years agoFix a crash on re-exec if a client was not in SENDING_DATA.
Steinar H. Gunderson [Sun, 14 Apr 2013 16:13:08 +0000 (18:13 +0200)]
Fix a crash on re-exec if a client was not in SENDING_DATA.

8 years agoRename bytes_sent to stream_pos, because that is what it really is.
Steinar H. Gunderson [Sun, 14 Apr 2013 00:30:47 +0000 (02:30 +0200)]
Rename bytes_sent to stream_pos, because that is what it really is.

8 years agoMake more usable documentation.
Steinar H. Gunderson [Sat, 13 Apr 2013 21:03:49 +0000 (23:03 +0200)]
Make more usable documentation.

8 years agoMerge branch 'master' of ssh://git.sesse.net/srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Sat, 13 Apr 2013 20:53:23 +0000 (22:53 +0200)]
Merge branch 'master' of ssh://git.sesse.net/srv/git.sesse.net/www/cubemap

8 years agoWhen HTTPInput disconnects, also clear the header.
Steinar H. Gunderson [Sat, 13 Apr 2013 20:53:02 +0000 (22:53 +0200)]
When HTTPInput disconnects, also clear the header.

8 years agoAnother include-what-you-use pass.
Steinar H. Gunderson [Sat, 13 Apr 2013 20:48:38 +0000 (22:48 +0200)]
Another include-what-you-use pass.

8 years agoMake backlog_size changeable across HUPs.
Steinar H. Gunderson [Sat, 13 Apr 2013 20:47:24 +0000 (22:47 +0200)]
Make backlog_size changeable across HUPs.

8 years agoMove Server:add_data() into Stream, where it more logically belongs.
Steinar H. Gunderson [Sat, 13 Apr 2013 20:35:45 +0000 (22:35 +0200)]
Move Server:add_data() into Stream, where it more logically belongs.

8 years agoRun include-what-you-use.
Steinar H. Gunderson [Sat, 13 Apr 2013 20:13:56 +0000 (22:13 +0200)]
Run include-what-you-use.

8 years agoFix incorrect struct/class in forward declares (found by Clang).
Steinar H. Gunderson [Sat, 13 Apr 2013 20:12:53 +0000 (22:12 +0200)]
Fix incorrect struct/class in forward declares (found by Clang).

8 years agoMove Client and Stream into their own files.
Steinar H. Gunderson [Sat, 13 Apr 2013 20:02:18 +0000 (22:02 +0200)]
Move Client and Stream into their own files.

8 years agoSupport configurable BACKLOG_SIZE (per-stream). No support for changing across restar...
Steinar H. Gunderson [Sat, 13 Apr 2013 19:53:45 +0000 (21:53 +0200)]
Support configurable BACKLOG_SIZE (per-stream). No support for changing across restarts yet.

8 years agoFix a socket leak in HTTPInput.
Steinar H. Gunderson [Sat, 13 Apr 2013 19:50:13 +0000 (21:50 +0200)]
Fix a socket leak in HTTPInput.

8 years agoFix a spurious warning.
Steinar H. Gunderson [Sat, 13 Apr 2013 17:06:48 +0000 (19:06 +0200)]
Fix a spurious warning.

8 years agoSend data using sendfile() instead of write(). Seems to be something like 30% lower...
Steinar H. Gunderson [Sat, 13 Apr 2013 17:04:27 +0000 (19:04 +0200)]
Send data using sendfile() instead of write(). Seems to be something like 30% lower CPU usage.

8 years agoFix a bug introduced with the move of read_tempfile() to util.cpp.
Steinar H. Gunderson [Sat, 13 Apr 2013 17:03:54 +0000 (19:03 +0200)]
Fix a bug introduced with the move of read_tempfile() to util.cpp.

8 years agoMove the “read the whole file” logic into a new file.
Steinar H. Gunderson [Sat, 13 Apr 2013 16:47:58 +0000 (18:47 +0200)]
Move the “read the whole file” logic into a new file.

8 years agoEven more missing unistd.h includes.
Steinar H. Gunderson [Fri, 12 Apr 2013 23:16:09 +0000 (01:16 +0200)]
Even more missing unistd.h includes.

8 years agoEven more missing unistd.h includes.
Steinar H. Gunderson [Fri, 12 Apr 2013 23:15:57 +0000 (01:15 +0200)]
Even more missing unistd.h includes.

8 years agoThread has virtual member functions, so it should have a virtual destructor.
Steinar H. Gunderson [Fri, 12 Apr 2013 23:15:24 +0000 (01:15 +0200)]
Thread has virtual member functions, so it should have a virtual destructor.

8 years agoFix a missing #include.
Steinar H. Gunderson [Fri, 12 Apr 2013 23:14:32 +0000 (01:14 +0200)]
Fix a missing #include.

8 years agoWhen re-execing, try with --test-config first, and revert if it fails.
Steinar H. Gunderson [Fri, 12 Apr 2013 22:24:36 +0000 (00:24 +0200)]
When re-execing, try with --test-config first, and revert if it fails.

8 years agoImplement --test-config.
Steinar H. Gunderson [Fri, 12 Apr 2013 21:07:01 +0000 (23:07 +0200)]
Implement --test-config.

8 years agoSince we just broke upgrade compatibility, kill some older stuff in the state protos.
Steinar H. Gunderson [Fri, 12 Apr 2013 21:04:50 +0000 (23:04 +0200)]
Since we just broke upgrade compatibility, kill some older stuff in the state protos.

8 years agoSwitch option parsing to getopt.
Steinar H. Gunderson [Fri, 12 Apr 2013 21:04:01 +0000 (23:04 +0200)]
Switch option parsing to getopt.

8 years agoMove some serialization logic into ServerPool, where it belongs.
Steinar H. Gunderson [Fri, 12 Apr 2013 20:31:05 +0000 (22:31 +0200)]
Move some serialization logic into ServerPool, where it belongs.

8 years agoSplit configuration parsing out cleanly from initialization.
Steinar H. Gunderson [Fri, 12 Apr 2013 20:09:24 +0000 (22:09 +0200)]
Split configuration parsing out cleanly from initialization.

This mostly is required for -configtest later, but also generally splits things out
in cleaner and nicer ways than before.

8 years agoFix UDPInput serialization.
Steinar H. Gunderson [Fri, 12 Apr 2013 20:08:09 +0000 (22:08 +0200)]
Fix UDPInput serialization.

8 years agoAdd rudimentary README.
Steinar H. Gunderson [Thu, 11 Apr 2013 23:56:12 +0000 (01:56 +0200)]
Add rudimentary README.

8 years agoSplit config parsing out of parse.h.
Steinar H. Gunderson [Thu, 11 Apr 2013 23:52:29 +0000 (01:52 +0200)]
Split config parsing out of parse.h.

8 years agoSupport UDP input. Also fix some issues with socket closing.
Steinar H. Gunderson [Thu, 11 Apr 2013 21:34:03 +0000 (23:34 +0200)]
Support UDP input. Also fix some issues with socket closing.

8 years agoWhen create_input() fails, give an error message instead of crashing.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:46:10 +0000 (21:46 +0200)]
When create_input() fails, give an error message instead of crashing.

8 years agoDo not keep pending data across HTTP connections.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:36:55 +0000 (21:36 +0200)]
Do not keep pending data across HTTP connections.

8 years agoMore support for multiple input types.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:31:11 +0000 (21:31 +0200)]
More support for multiple input types.

8 years agoMove version identification into a common place.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:08:14 +0000 (21:08 +0200)]
Move version identification into a common place.

8 years agoMake Input a bit more generic, to pave the way for UDP.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:06:03 +0000 (21:06 +0200)]
Make Input a bit more generic, to pave the way for UDP.

8 years agoSupport multiple listening sockets. Actually mostly because it makes the code somewha...
Steinar H. Gunderson [Wed, 10 Apr 2013 23:30:23 +0000 (01:30 +0200)]
Support multiple listening sockets. Actually mostly because it makes the code somewhat more regular.

8 years agoMove stopping into ServerPool.
Steinar H. Gunderson [Wed, 10 Apr 2013 22:24:33 +0000 (00:24 +0200)]
Move stopping into ServerPool.

8 years agoFactor serializing into its own function. Again, less stuff in main().
Steinar H. Gunderson [Wed, 10 Apr 2013 22:21:09 +0000 (00:21 +0200)]
Factor serializing into its own function. Again, less stuff in main().

8 years agoMove stream creation out of main().
Steinar H. Gunderson [Wed, 10 Apr 2013 22:12:42 +0000 (00:12 +0200)]
Move stream creation out of main().

8 years agoSome small refactoring of main().
Steinar H. Gunderson [Wed, 10 Apr 2013 22:06:29 +0000 (00:06 +0200)]
Some small refactoring of main().

8 years agoFactor all the common thread starting/stopping into a common Thread class.
Steinar H. Gunderson [Wed, 10 Apr 2013 21:55:24 +0000 (23:55 +0200)]
Factor all the common thread starting/stopping into a common Thread class.

8 years agoMinor nit.
Steinar H. Gunderson [Wed, 10 Apr 2013 21:45:25 +0000 (23:45 +0200)]
Minor nit.

8 years agoWrap the acceptor into the same thread logic as everything else.
Steinar H. Gunderson [Wed, 10 Apr 2013 21:45:20 +0000 (23:45 +0200)]
Wrap the acceptor into the same thread logic as everything else.

8 years agoMerge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Wed, 10 Apr 2013 21:36:59 +0000 (23:36 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap

8 years agoFactor statistics writing into its own class and file.
Steinar H. Gunderson [Wed, 10 Apr 2013 21:36:56 +0000 (23:36 +0200)]
Factor statistics writing into its own class and file.

8 years agoSmall refactoring in main.cpp.
Steinar H. Gunderson [Wed, 10 Apr 2013 21:35:08 +0000 (23:35 +0200)]
Small refactoring in main.cpp.

8 years agoFactor statistics writing into its own class and file.
Steinar H. Gunderson [Wed, 10 Apr 2013 21:11:00 +0000 (23:11 +0200)]
Factor statistics writing into its own class and file.

8 years agoSeparate out the acceptor stuff into its own file.
Steinar H. Gunderson [Wed, 10 Apr 2013 20:40:00 +0000 (22:40 +0200)]
Separate out the acceptor stuff into its own file.

8 years agoAdd support for setting fwmarks per-connection. Useful for shaping individual streams...
Steinar H. Gunderson [Wed, 10 Apr 2013 19:10:00 +0000 (21:10 +0200)]
Add support for setting fwmarks per-connection. Useful for shaping individual streams (for doing TCP pacing).

8 years agoRemove some obsolete #defines.
Steinar H. Gunderson [Tue, 9 Apr 2013 23:11:22 +0000 (01:11 +0200)]
Remove some obsolete #defines.

8 years agoExplicitly SIGHUP threads to kill them out of syscalls when we want to join them...
Steinar H. Gunderson [Tue, 9 Apr 2013 20:37:55 +0000 (22:37 +0200)]
Explicitly SIGHUP threads to kill them out of syscalls when we want to join them. Also fixes some bugs related to EINTR handling, since they became very obvious now.

8 years agoMeasure how long the serialize/exec/deserialize process takes.
Steinar H. Gunderson [Tue, 9 Apr 2013 20:29:25 +0000 (22:29 +0200)]
Measure how long the serialize/exec/deserialize process takes.

8 years agoSupport writing a stats file listing the number of clients currently connected.
Steinar H. Gunderson [Tue, 9 Apr 2013 20:17:16 +0000 (22:17 +0200)]
Support writing a stats file listing the number of clients currently connected.

8 years agoRename cubemap.cpp to main.cpp. Aaaa....
Steinar H. Gunderson [Tue, 9 Apr 2013 18:49:01 +0000 (20:49 +0200)]
Rename cubemap.cpp to main.cpp. Aaaa....

8 years agoParse the HTTP header (more) properly, and send the headers on to any connecting...
Steinar H. Gunderson [Mon, 8 Apr 2013 23:35:03 +0000 (01:35 +0200)]
Parse the HTTP header (more) properly, and send the headers on to any connecting clients.

8 years agoDeserialize/serialize inputs. Woo, totally glitch-free restarts!
Steinar H. Gunderson [Mon, 8 Apr 2013 23:11:06 +0000 (01:11 +0200)]
Deserialize/serialize inputs. Woo, totally glitch-free restarts!

8 years agoFix a few issues in the to_process() handling.
Steinar H. Gunderson [Mon, 8 Apr 2013 22:48:57 +0000 (00:48 +0200)]
Fix a few issues in the to_process() handling.

8 years agoAdd URL parsing.
Steinar H. Gunderson [Mon, 8 Apr 2013 22:24:31 +0000 (00:24 +0200)]
Add URL parsing.

8 years agoWrite our own HTTP client instead of using curl. Not finished yet (missing URL parsin...
Steinar H. Gunderson [Mon, 8 Apr 2013 22:16:41 +0000 (00:16 +0200)]
Write our own HTTP client instead of using curl. Not finished yet (missing URL parsing, for one).

8 years agoMake most operations on Server deferred, so that we a) do not get bugs with epoll...
Steinar H. Gunderson [Mon, 8 Apr 2013 18:28:03 +0000 (20:28 +0200)]
Make most operations on Server deferred, so that we a) do not get bugs with epoll going out-of-sync with the client state, and b) do not get performance issues stemming from Input not managing to push data often enough to the servers.

8 years agoMake the acceptor thread stop nicely, so that it does not try to mess up the server...
Steinar H. Gunderson [Mon, 8 Apr 2013 18:27:03 +0000 (20:27 +0200)]
Make the acceptor thread stop nicely, so that it does not try to mess up the server while we are shutting down  the server.

8 years agoMake the list of sleeping clients be per-stream instead of global, so we do not wake...
Steinar H. Gunderson [Sun, 7 Apr 2013 22:21:40 +0000 (00:21 +0200)]
Make the list of sleeping clients be per-stream instead of global, so we do not wake up tons of clients for no good reason.

8 years agoIf accept() fails, do not take down the entire thing; it could be a transient error...
Steinar H. Gunderson [Sun, 7 Apr 2013 22:20:51 +0000 (00:20 +0200)]
If accept() fails, do not take down the entire thing; it could be a transient error like “too many open files”. Instead, sleep 0.1 seconds and try again.

8 years agoFix so clients would actually be woken up from the worker thread, not the input threa...
Steinar H. Gunderson [Sun, 7 Apr 2013 21:26:37 +0000 (23:26 +0200)]
Fix so clients would actually be woken up from the worker thread, not the input thread. (Gah!)

8 years agoFix an error where clients in asleep state would hang after a HUP.
Steinar H. Gunderson [Sun, 7 Apr 2013 21:20:16 +0000 (23:20 +0200)]
Fix an error where clients in asleep state would hang after a HUP.

8 years agoMake edge-triggering more consistent.
Steinar H. Gunderson [Sun, 7 Apr 2013 21:13:43 +0000 (23:13 +0200)]
Make edge-triggering more consistent.

8 years agoRemove some leftover code.
Steinar H. Gunderson [Sun, 7 Apr 2013 20:48:13 +0000 (22:48 +0200)]
Remove some leftover code.

8 years agoFix an issue where clients would be put twice into sleeping_clients (probably since...
Steinar H. Gunderson [Sun, 7 Apr 2013 20:06:24 +0000 (22:06 +0200)]
Fix an issue where clients would be put twice into sleeping_clients (probably since the EPOLLOUT trigger would go off once unexpectedly), causing a chain reaction where we did tons of empty write() calls.

8 years agoSome fixes to issues that could cause client hangs.
Steinar H. Gunderson [Sun, 7 Apr 2013 19:10:20 +0000 (21:10 +0200)]
Some fixes to issues that could cause client hangs.

8 years agoStop doing find_stream() every time we send out data; it takes some CPU.
Steinar H. Gunderson [Sun, 7 Apr 2013 19:06:16 +0000 (21:06 +0200)]
Stop doing find_stream() every time we send out data; it takes some CPU.

8 years agoChange from level-triggered to edge-triggered epoll mode. More than halves CPU usage.
Steinar H. Gunderson [Sun, 7 Apr 2013 18:49:54 +0000 (20:49 +0200)]
Change from level-triggered to edge-triggered epoll mode. More than halves CPU usage.

8 years agoHandle streams coming and going from the configuration file across restarts.
Steinar H. Gunderson [Sun, 7 Apr 2013 17:03:42 +0000 (19:03 +0200)]
Handle streams coming and going from the configuration file across restarts.