Why does named log "error sending response"?
Sometimes on busy servers, named is unable to send a client response and will log messages similar to:
client 192.0.2.17#4218: error sending response: unexpected error
More recent versions of BIND have changed the error message slightly:
client 192.0.2.123#48516 (looking.this.up.com): error sending response: unset
And in future, some of the errors that the kernel/OS can return will also be called-out specifically, rather than bundled into the generic 'unset' logged message. The two that are included in this new format are EWOULDBLOCK and EAGAIN, both reported as 'would block', since this is the same outcome:
client 192.0.2.123#29472 (looking.this.up.com): error sending response: would block
The other two kernel error codes that contribute to named logging 'error sending response' without an explicit message are ECONNREFUSED and too many repeated EINTR errors (we loop and retry a few times on EINTR).
These errors indicate that there is a bottleneck or barrier somewhere in the outbound path and that when named has tried to send, the kernel was unable to accept the outbound packet. Typically it will be seen on a busy server, particularly one which is sending large outbound responses. Note that query responses are usually larger than the original inbound client queries.
The underlying cause may vary from installation to installation. Some things to check:
- Are your links/NICs configured for full duplex?
- If you're connected to a switch/router/load-balancer, can it handle the outbound packet rate as well as the inbound?
- Is there a firewall deployed - and is it able to handle the outbound packet rate?
- Are you using ipfilters/iptables - are they able to handle the outbound packet rate, even if the packets should simply pass through?
- Do you have inbound buffer overflows (this can sometimes cause problems for outbound traffic due to the way the kernel is handling resources)?
- Is the OS running out of mbufs or sk_buffs?
But since the outbound traffic rate is likely to be 'bursty', particularly on a recursive server, it may also be useful to tune your outbound buffer sizes. On linux kernels (which often by default have small write buffers), the kernel parameters to tune are net.core.wmem_default and net.core.wmem_max. You might also need to increase net.core.rmem_default and net.core.rmem_max if you're finding that the server cannot handle the inbound packet rate too.