Replace an assert with a small hack.

Instead of checking that connect times are monotonic, explicitly make them
so if they're not. This seems safest, in case the monotonic clock goes
backwards a small bit (e.g. when changing CPUs). We don't need it for the
serialized case since we explicitly sort those by time; the assert can stay.