]>
git.sesse.net Git - cubemap/log
Steinar H. Gunderson [Sat, 13 Apr 2013 20:12:53 +0000 (22:12 +0200)]
Fix incorrect struct/class in forward declares (found by Clang).
Steinar H. Gunderson [Sat, 13 Apr 2013 20:02:18 +0000 (22:02 +0200)]
Move Client and Stream into their own files.
Steinar H. Gunderson [Sat, 13 Apr 2013 19:53:45 +0000 (21:53 +0200)]
Support configurable BACKLOG_SIZE (per-stream). No support for changing across restarts yet.
Steinar H. Gunderson [Sat, 13 Apr 2013 19:50:13 +0000 (21:50 +0200)]
Fix a socket leak in HTTPInput.
Steinar H. Gunderson [Sat, 13 Apr 2013 17:06:48 +0000 (19:06 +0200)]
Fix a spurious warning.
Steinar H. Gunderson [Sat, 13 Apr 2013 17:04:27 +0000 (19:04 +0200)]
Send data using sendfile() instead of write(). Seems to be something like 30% lower CPU usage.
Steinar H. Gunderson [Sat, 13 Apr 2013 17:03:54 +0000 (19:03 +0200)]
Fix a bug introduced with the move of read_tempfile() to util.cpp.
Steinar H. Gunderson [Sat, 13 Apr 2013 16:47:58 +0000 (18:47 +0200)]
Move the “read the whole file” logic into a new file.
Steinar H. Gunderson [Fri, 12 Apr 2013 23:16:09 +0000 (01:16 +0200)]
Even more missing unistd.h includes.
Steinar H. Gunderson [Fri, 12 Apr 2013 23:15:57 +0000 (01:15 +0200)]
Even more missing unistd.h includes.
Steinar H. Gunderson [Fri, 12 Apr 2013 23:15:24 +0000 (01:15 +0200)]
Thread has virtual member functions, so it should have a virtual destructor.
Steinar H. Gunderson [Fri, 12 Apr 2013 23:14:32 +0000 (01:14 +0200)]
Fix a missing #include.
Steinar H. Gunderson [Fri, 12 Apr 2013 22:24:36 +0000 (00:24 +0200)]
When re-execing, try with --test-config first, and revert if it fails.
Steinar H. Gunderson [Fri, 12 Apr 2013 21:07:01 +0000 (23:07 +0200)]
Implement --test-config.
Steinar H. Gunderson [Fri, 12 Apr 2013 21:04:50 +0000 (23:04 +0200)]
Since we just broke upgrade compatibility, kill some older stuff in the state protos.
Steinar H. Gunderson [Fri, 12 Apr 2013 21:04:01 +0000 (23:04 +0200)]
Switch option parsing to getopt.
Steinar H. Gunderson [Fri, 12 Apr 2013 20:31:05 +0000 (22:31 +0200)]
Move some serialization logic into ServerPool, where it belongs.
Steinar H. Gunderson [Fri, 12 Apr 2013 20:09:24 +0000 (22:09 +0200)]
Split configuration parsing out cleanly from initialization.
This mostly is required for -configtest later, but also generally splits things out
in cleaner and nicer ways than before.
Steinar H. Gunderson [Fri, 12 Apr 2013 20:08:09 +0000 (22:08 +0200)]
Fix UDPInput serialization.
Steinar H. Gunderson [Thu, 11 Apr 2013 23:56:12 +0000 (01:56 +0200)]
Add rudimentary README.
Steinar H. Gunderson [Thu, 11 Apr 2013 23:52:29 +0000 (01:52 +0200)]
Split config parsing out of parse.h.
Steinar H. Gunderson [Thu, 11 Apr 2013 21:34:03 +0000 (23:34 +0200)]
Support UDP input. Also fix some issues with socket closing.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:46:10 +0000 (21:46 +0200)]
When create_input() fails, give an error message instead of crashing.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:36:55 +0000 (21:36 +0200)]
Do not keep pending data across HTTP connections.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:31:11 +0000 (21:31 +0200)]
More support for multiple input types.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:08:14 +0000 (21:08 +0200)]
Move version identification into a common place.
Steinar H. Gunderson [Thu, 11 Apr 2013 19:06:03 +0000 (21:06 +0200)]
Make Input a bit more generic, to pave the way for UDP.
Steinar H. Gunderson [Wed, 10 Apr 2013 23:30:23 +0000 (01:30 +0200)]
Support multiple listening sockets. Actually mostly because it makes the code somewhat more regular.
Steinar H. Gunderson [Wed, 10 Apr 2013 22:24:33 +0000 (00:24 +0200)]
Move stopping into ServerPool.
Steinar H. Gunderson [Wed, 10 Apr 2013 22:21:09 +0000 (00:21 +0200)]
Factor serializing into its own function. Again, less stuff in main().
Steinar H. Gunderson [Wed, 10 Apr 2013 22:12:42 +0000 (00:12 +0200)]
Move stream creation out of main().
Steinar H. Gunderson [Wed, 10 Apr 2013 22:06:29 +0000 (00:06 +0200)]
Some small refactoring of main().
Steinar H. Gunderson [Wed, 10 Apr 2013 21:55:24 +0000 (23:55 +0200)]
Factor all the common thread starting/stopping into a common Thread class.
Steinar H. Gunderson [Wed, 10 Apr 2013 21:45:25 +0000 (23:45 +0200)]
Minor nit.
Steinar H. Gunderson [Wed, 10 Apr 2013 21:45:20 +0000 (23:45 +0200)]
Wrap the acceptor into the same thread logic as everything else.
Steinar H. Gunderson [Wed, 10 Apr 2013 21:36:59 +0000 (23:36 +0200)]
Merge branch 'master' of /srv/git.sesse.net/www/cubemap
Steinar H. Gunderson [Wed, 10 Apr 2013 21:36:56 +0000 (23:36 +0200)]
Factor statistics writing into its own class and file.
Steinar H. Gunderson [Wed, 10 Apr 2013 21:35:08 +0000 (23:35 +0200)]
Small refactoring in main.cpp.
Steinar H. Gunderson [Wed, 10 Apr 2013 21:11:00 +0000 (23:11 +0200)]
Factor statistics writing into its own class and file.
Steinar H. Gunderson [Wed, 10 Apr 2013 20:40:00 +0000 (22:40 +0200)]
Separate out the acceptor stuff into its own file.
Steinar H. Gunderson [Wed, 10 Apr 2013 19:10:00 +0000 (21:10 +0200)]
Add support for setting fwmarks per-connection. Useful for shaping individual streams (for doing TCP pacing).
Steinar H. Gunderson [Tue, 9 Apr 2013 23:11:22 +0000 (01:11 +0200)]
Remove some obsolete #defines.
Steinar H. Gunderson [Tue, 9 Apr 2013 20:37:55 +0000 (22:37 +0200)]
Explicitly SIGHUP threads to kill them out of syscalls when we want to join them. Also fixes some bugs related to EINTR handling, since they became very obvious now.
Steinar H. Gunderson [Tue, 9 Apr 2013 20:29:25 +0000 (22:29 +0200)]
Measure how long the serialize/exec/deserialize process takes.
Steinar H. Gunderson [Tue, 9 Apr 2013 20:17:16 +0000 (22:17 +0200)]
Support writing a stats file listing the number of clients currently connected.
Steinar H. Gunderson [Tue, 9 Apr 2013 18:49:01 +0000 (20:49 +0200)]
Rename cubemap.cpp to main.cpp. Aaaa....
Steinar H. Gunderson [Mon, 8 Apr 2013 23:35:03 +0000 (01:35 +0200)]
Parse the HTTP header (more) properly, and send the headers on to any connecting clients.
Steinar H. Gunderson [Mon, 8 Apr 2013 23:11:06 +0000 (01:11 +0200)]
Deserialize/serialize inputs. Woo, totally glitch-free restarts!
Steinar H. Gunderson [Mon, 8 Apr 2013 22:48:57 +0000 (00:48 +0200)]
Fix a few issues in the to_process() handling.
Steinar H. Gunderson [Mon, 8 Apr 2013 22:24:31 +0000 (00:24 +0200)]
Add URL parsing.
Steinar H. Gunderson [Mon, 8 Apr 2013 22:16:41 +0000 (00:16 +0200)]
Write our own HTTP client instead of using curl. Not finished yet (missing URL parsing, for one).
Steinar H. Gunderson [Mon, 8 Apr 2013 18:28:03 +0000 (20:28 +0200)]
Make most operations on Server deferred, so that we a) do not get bugs with epoll going out-of-sync with the client state, and b) do not get performance issues stemming from Input not managing to push data often enough to the servers.
Steinar H. Gunderson [Mon, 8 Apr 2013 18:27:03 +0000 (20:27 +0200)]
Make the acceptor thread stop nicely, so that it does not try to mess up the server while we are shutting down the server.
Steinar H. Gunderson [Sun, 7 Apr 2013 22:21:40 +0000 (00:21 +0200)]
Make the list of sleeping clients be per-stream instead of global, so we do not wake up tons of clients for no good reason.
Steinar H. Gunderson [Sun, 7 Apr 2013 22:20:51 +0000 (00:20 +0200)]
If accept() fails, do not take down the entire thing; it could be a transient error like “too many open files”. Instead, sleep 0.1 seconds and try again.
Steinar H. Gunderson [Sun, 7 Apr 2013 21:26:37 +0000 (23:26 +0200)]
Fix so clients would actually be woken up from the worker thread, not the input thread. (Gah!)
Steinar H. Gunderson [Sun, 7 Apr 2013 21:20:16 +0000 (23:20 +0200)]
Fix an error where clients in asleep state would hang after a HUP.
Steinar H. Gunderson [Sun, 7 Apr 2013 21:13:43 +0000 (23:13 +0200)]
Make edge-triggering more consistent.
Steinar H. Gunderson [Sun, 7 Apr 2013 20:48:13 +0000 (22:48 +0200)]
Remove some leftover code.
Steinar H. Gunderson [Sun, 7 Apr 2013 20:06:24 +0000 (22:06 +0200)]
Fix an issue where clients would be put twice into sleeping_clients (probably since the EPOLLOUT trigger would go off once unexpectedly), causing a chain reaction where we did tons of empty write() calls.
Steinar H. Gunderson [Sun, 7 Apr 2013 19:10:20 +0000 (21:10 +0200)]
Some fixes to issues that could cause client hangs.
Steinar H. Gunderson [Sun, 7 Apr 2013 19:06:16 +0000 (21:06 +0200)]
Stop doing find_stream() every time we send out data; it takes some CPU.
Steinar H. Gunderson [Sun, 7 Apr 2013 18:49:54 +0000 (20:49 +0200)]
Change from level-triggered to edge-triggered epoll mode. More than halves CPU usage.
Steinar H. Gunderson [Sun, 7 Apr 2013 17:03:42 +0000 (19:03 +0200)]
Handle streams coming and going from the configuration file across restarts.
Steinar H. Gunderson [Sun, 7 Apr 2013 16:51:27 +0000 (18:51 +0200)]
EINTR-protect a bunch of calls, and warn on close() failing.
Steinar H. Gunderson [Sun, 7 Apr 2013 15:46:32 +0000 (17:46 +0200)]
Handle EINTR in epoll_wait().
Steinar H. Gunderson [Sun, 7 Apr 2013 15:42:23 +0000 (17:42 +0200)]
Support parsing streams from config file. Also support multiple streams (includes parsing HTTP), with HTTP error messages and all.
Steinar H. Gunderson [Sun, 7 Apr 2013 15:11:55 +0000 (17:11 +0200)]
When a transfer ends, output the URL.
Steinar H. Gunderson [Sun, 7 Apr 2013 15:02:37 +0000 (17:02 +0200)]
Factor out some config parsing into its own function.
Steinar H. Gunderson [Sun, 7 Apr 2013 14:11:28 +0000 (16:11 +0200)]
Add a sample configuration file.
Steinar H. Gunderson [Sun, 7 Apr 2013 14:10:39 +0000 (16:10 +0200)]
Fetch num_servers from the config file.
Steinar H. Gunderson [Sun, 7 Apr 2013 14:01:03 +0000 (16:01 +0200)]
Move the logic of load-balancing etc. into ServerPool, which frees external users from knowing anything about NUM_SERVERS.
Steinar H. Gunderson [Sun, 7 Apr 2013 13:43:36 +0000 (15:43 +0200)]
Reopen the port if it changes between runs.
Steinar H. Gunderson [Sun, 7 Apr 2013 13:39:54 +0000 (15:39 +0200)]
Take the port from the configuration file.
Steinar H. Gunderson [Sun, 7 Apr 2013 13:33:37 +0000 (15:33 +0200)]
Parse a config file. Not used for anything yet.
Steinar H. Gunderson [Sun, 7 Apr 2013 11:10:03 +0000 (13:10 +0200)]
Support deserialization of most state (curl input is not really good yet).
Steinar H. Gunderson [Sun, 7 Apr 2013 00:22:49 +0000 (02:22 +0200)]
Remember to turn off has_metacube_header once we have consumed a block.
Steinar H. Gunderson [Sat, 6 Apr 2013 22:51:43 +0000 (00:51 +0200)]
Make Input honor should_stop even when it is not connected to an encoder.
Steinar H. Gunderson [Sat, 6 Apr 2013 22:51:12 +0000 (00:51 +0200)]
When we get a header, retroactively send it out to all clients that have received no data yet. Useful if the clients connect before the encoder does.
Steinar H. Gunderson [Sat, 6 Apr 2013 22:14:30 +0000 (00:14 +0200)]
Makefile tweaks to deal with the protobuf dependency.
Steinar H. Gunderson [Sat, 6 Apr 2013 22:09:07 +0000 (00:09 +0200)]
Reconnect when the input goes away.
Steinar H. Gunderson [Sat, 6 Apr 2013 22:07:40 +0000 (00:07 +0200)]
Serialize and die on SIGHUP (no deserialization yet). Required a whole bunch of Valgrind fixes to actually work.
Steinar H. Gunderson [Sat, 6 Apr 2013 21:08:34 +0000 (23:08 +0200)]
Fix broken Client constructor.
Steinar H. Gunderson [Sat, 6 Apr 2013 20:08:14 +0000 (22:08 +0200)]
Start working on serialization.
Steinar H. Gunderson [Sat, 6 Apr 2013 19:34:48 +0000 (21:34 +0200)]
Make a useful constructor for Client.
Steinar H. Gunderson [Sat, 6 Apr 2013 19:11:12 +0000 (21:11 +0200)]
Fix the broken SO_REUSEADDR.
Steinar H. Gunderson [Sat, 6 Apr 2013 17:02:43 +0000 (19:02 +0200)]
Make Server stoppable.
Steinar H. Gunderson [Sat, 6 Apr 2013 16:39:10 +0000 (18:39 +0200)]
Merge remote-tracking branch 'origin/master'
Steinar H. Gunderson [Sat, 6 Apr 2013 16:36:14 +0000 (18:36 +0200)]
Rename client_request to request, and clear it when we are done with it.
Steinar H. Gunderson [Sat, 6 Apr 2013 16:36:14 +0000 (18:36 +0200)]
Rename client_request to request, and clear it when we are done with it.
Steinar H. Gunderson [Sat, 6 Apr 2013 16:18:35 +0000 (18:18 +0200)]
Remove a no longer relevant TODO.
Steinar H. Gunderson [Sat, 6 Apr 2013 16:17:12 +0000 (18:17 +0200)]
When closing a client, make sure it is not left in sleeping_clients.
Steinar H. Gunderson [Sat, 6 Apr 2013 16:13:55 +0000 (18:13 +0200)]
Implement writev(). Working reflector!
Steinar H. Gunderson [Sat, 6 Apr 2013 16:08:51 +0000 (18:08 +0200)]
Implement sleeping/waking clients.
Steinar H. Gunderson [Sat, 6 Apr 2013 16:00:54 +0000 (18:00 +0200)]
Implement add_data().
Steinar H. Gunderson [Sat, 6 Apr 2013 15:56:48 +0000 (17:56 +0200)]
Implement basic data sending support.
Steinar H. Gunderson [Sat, 6 Apr 2013 15:48:24 +0000 (17:48 +0200)]
Skip the Metacube header when adding data.
Steinar H. Gunderson [Sat, 6 Apr 2013 15:47:44 +0000 (17:47 +0200)]
Implement header sending.
Steinar H. Gunderson [Sat, 6 Apr 2013 14:58:54 +0000 (16:58 +0200)]
Implement epoll main loop in Server, and parse header.
Steinar H. Gunderson [Sat, 6 Apr 2013 14:55:16 +0000 (16:55 +0200)]
Set SO_REUSEADDR on the server socket.