What does "isc_socket_create: fcntl/reserved: Too many open files" mean?
  • 23 Jun 2021
  • 1 Minute to read
  • Contributors
  • Dark
    Light
  • PDF

What does "isc_socket_create: fcntl/reserved: Too many open files" mean?

  • Dark
    Light
  • PDF

Article Summary

Question:

What do these named errors signify - and how should they be investigated?

10-May-2011 15:41:13.815 general: error: isc_socket_create:
fcntl/reserved: Too many open files
10-May-2011 15:41:13.815 general: error: isc_socket_create:
fcntl/reserved: Too many open files
10-May-2011 15:41:13.815 general: error: isc_socket_create:
fcntl/reserved: Too many open files
10-May-2011 15:41:13.815 general: error: isc_socket_create:
fcntl/reserved: Too many open files

Answer:

These errors would typically be seen on a server which is providing recursive resolution to clients and are usually indicating that the per-process (sometimes per system - but that would be less usual) limit on the number of open file descriptors (of which sockets are one example) has been reached.

You can check the per-system use of sockets using:

netstat -an

You can (approximately) check named's use of sockets for DNS protocol activity using:

netstat -an|grep ":53"

And more accurately (if you have lsof), and logged in with sufficient access:

lsof -p <pid of named>

The first setting to check is your system limits on open files per process - how to do this may vary between operating systems.

Also check the named startup logs - depending on the BIND version and the system configuration, named's own limit on open sockets should be logged and occasionally warnings will be seen when the system limits further constrict the value that named would prefer to use.

Also looking at the logfiles - is there a pattern of other errors being reported when these errors are seen?

When these errors are being logged, check the output from rndc status - in particular the number of recursive clients. If there's a build-up of recursive clients at the same time, this might explain why the limit on open files has also been reached. This could be due to a normal peak in client activity, or it might be indicative of some other problem.

If there is a build-up of recursive clients, then we recommend using rndc recursing to obtain the details of waiting clients and the queries that they have made.

If there are no resolution problems being reported and these errors are infrequent and transient, then it's likely that you're hitting a resource bottleneck periodically. We'd recommend further monitoring and investigation to understand what is happening when the errors are seen in case the situation deteriorates, but this wouldn't be an urgent matter.