Anything less seemingly triggers a bug in the ALSA PulseAudio
plugin when used against PipeWire (audio just never starts playing).
It would be nice to generally have less latency here, but with the
current design based on large resampling queues, this isn't going
to be what causes it.