DHCPv4 Server Performance

Background

In the DHCPv4 server there are two areas of the code that can limit the server's performance: saving lease information to non-volatile storage and manipulating a lease entry within the in-memory data structures. This article discusses some of the issues involved in these bottlenecks and some of the options for improving the performance.

In our testing we found including either a lease storage option or an in-memory structure option to have a noticeable performance impact, but if both were included the impact was quite significant.

Due to the wide range of hardware available, we can't provide specific performance numbers for any combination. The best advice we can give in general is to try out various options to find a set that meets your performance and budget requirements and for which the data loss risk is acceptable.

Note that the DHCPv6 server uses different code for handling these areas and may have different issues.

Lease Storage

In order to work properly after restarts, a DHCP server must be able to determine which leases it has issued and when those leases will expire. The DHCPv4 server accomplishes this by writing an entry to the lease file whenever it updates a lease and reading the entries in from the lease file whenever it restarts. Because the entry is written out before the lease is transmitted to the client, the server can be assured that it will not hand out a duplicate lease as long as the lease file is not damaged.

To minimize the time the server spends on the lease file, it only appends entries to the end of the lease file and the lease file is allowed to contain multiple entires for a single lease. As the server simply appends entries as they are updated, the last entry for a given lease is always the current one. Periodically the server will write out all of the entries to the lease file; this will generally shrink the lease file as any leases with multiple entries will be reduced to a single entry.

The bottleneck is then that the server has to synchronously write an entry to the disk file on most lease updates.  While the OS waits for the disk to spin and the write to complete the server is not doing anything else.

Use non-disk non-volatile memory

This is generally a good solution as it avoids other complications and can simply speed up the "disk" performance. The downside is that the storage solution tends to be more expensive, though as prices have fallen over the years that is less of an issue.  

Delayed acknowledgements

In this option the server will attempt to "batch" writes to the disk. The server will either wait for a small time period or a set number of acknowledgements to accumulate before trying to write them to disk and then perform a single fsync() to flush them all at the end. This doesn't require any additional hardware but it can introduce a small delay in responding to a client. In addition, when failover or DDNS are enabled their updates make this option less useful. The feature needs to be enabled at compile time:

--enable-delayed-ack

And then the following options may be used to control how long the server delays (please see the man page for more description of these options):

delayed-ack [count]

max-ack-delay [microseconds]

We have tested this in-house but have not received much feedback from people using it in the field and so can't be sure how well it truly works in real-life situations.  

Cache threshold

In this feature the server attempts to avoid unnecessary disk writes. If a client attempts to renew a lease within a short time of getting the lease, the server responds with the exact same lease information and the remaining time for the original lease. Because the server hasn't changed anything it does not need to write the lease to the disk file. For example, if the client receives a lease with a 1 hour (3600 seconds) lifetime and then attempts to renew in only 10 seconds, the server would respond with a lease with a lifetime of 3590 seconds.

This feature is enabled by default with the default value being 25%.

dhcp-cache-threshold [percentage]

This is particularly useful if you have a small number of clients that renew very frequently, either due to bugs in their code or external events (such as a smartphone trying to determine its current subnet).

Disable fsync

Part of the reason for the performance issues with writing to disk is ensuring that the data has actually been written to disk. The server attempts to verify this by calling fsync() after it writes an entry to the lease file. Depending on your OS and hardware, this may not be necessary and we now provide an option to turn off this call to fsync(). We don't recommend you use this feature without a thorough understanding of how your OS and hardware work and the risks involved. If an entry is not written out to disk and the system is halted, the server may issue that lease to a different client and cause problems on your network.  

dont-use-fsync [flag]

Use RAM-based storage

One option we have heard of people using is to create a RAM disk and to put the lease file on that disk. This certainly should speed things up a bit, but has the possibility of losing the lease file and we don't recommend it. See Using RAMdisks and other similar volatile storage for the leases file for a longer discussion of this option.

In-memory structures

When the DHCP server was first written, the number of leases involved in a typical installation was somewhat smaller than it is today and the in-memory data structures reflect that. Several of the key data structures use a singly linked list to hold the leases, and a large amount of time is spent traversing those lists to either add or remove a lease as it moves either from one structure to another (for example from free to active or active to free) or within one structure (for example from active to active due to a lease renew). As these structures are sorted linear lists, the processing time grows as the lease pools get larger.

In 4.3.3 we have added a compile time option to process the lists in a binary fashion instead of needing to walk them in a linear fashion. As with all of our code, we have tested this feature out and found it useful. However, we have chosen to require you to select it via a compile time option, which allows our users to test it out in their environments and report back to us in case there are cases we did not consider in our testing while still having the fallback of the previous code.

--enable-binary-leases