AXFR-style IXFR explained

Sometimes (and usually unexpectedly), a zone update that would normally be incremental (IXFR), instead takes place as a full zone transfer (AXFR) and you will see the sending server logging something similar to:

01-Aug-2018 17:56:12.607 xfer-out: client 192.0.2.25#47497 (ns2.myzone.example.com): transfer of 'myzone.example.com/IN': AXFR-style IXFR started (serial 412563719)
01-Aug-2018 17:56:12.610 xfer-out: client 192.0.2.25#47497 (ns2.myzone.example.com): transfer of 'myzone.example.com/IN': AXFR-style IXFR ended

When a secondary server requests an incremental zone update from a primary, the primary will provide an IXFR when it can, but sometimes it's not able to, and sends a full AXFR instead.

In order to provide an IXFR, the authoritative server needs to have a record of all updates that have taken place to the zone between its version of the zone as identified by the current serial number in the SOA RR, and the version that the requesting secondary says that it has.

The record of updates or differences is held in a .jnl (journal) file associated with the zone. This file is maintained as the zone is incrementally updated, via one of:

• dynamic updates (nsupdate) applied directly to the zone
• incremental updates (IXFRs) received
• manual zone file maintenance and zone reload in a configuration where the ixfr-from-differences option is applicable to the zone.
• manual zone file maintenance and an inbound AXFR in a configuration where the ixfr-from-differences option is applicable to the zone.

The zone being served is held by named in its working memory, and periodically written out to disk as a backup. It is the in-memory version of the zone that is being updated as the .jnl file is being maintained. Therefore in order to ensure integrity of the zone content, in case of unexpected interruptions before the full zone can be written to disk, the .jnl file for a zone on a particular server will, at a minimum, always contain the full record of update transactions need to replay/reload the zone when named is restarted on that server.

The .jnl file may however contain more transactions than are minimally necessary to ensure the integrity of the zone content; the usual reason why this is desirable is to ensure that the server is capable of fulfilling all IXFR requests from other servers, even if the local instance of the zone no longer needs all of the transactions in the .jnl file to recover itself in an emergency.

A discussion on optimising .jnl file sizes can be found here:
Choosing the right value for max-journal-size

The most commonly-encountered reason for an authoritative server being unable to provide the requested IXFR, is that the .jnl file no longer holds all of the individual transactions needed by the requesting secondary server. This can happen if:

• The requesting secondary server has not updated for some time and is a long way behind the authoritative server. In that case a full zone transfer (AXFR) might well be the most efficient method for updating it.
• The authoritative server has itself received an unusually high rate of updates recently and has subsequently pruned its .jnl file following the routine backup of the full zone to disk. Again, with a high volume of incremental changes, it is likely that the full zone transfer will be more efficient than a large incremental zone transfer.

There is one other situation, rarely encountered, in which an authoritative server is unable to fulfill an IXFR request.

In this article previously, incrementatal zone updates were referred to as transactions. When a zone is being updated dynamically, a single update transaction can consist of a series of RR and RRset adds and deletes. If the update transaction cannot complete in entirety, then it is reverted. If it is successful, then the starting and ending zone serial numbers are recorded so that in a zone recovery situation, or when responding to IXFR requests, only complete transactions are applied/sent.

The IXFR protocol allows (it is entirely optional) a server that receives several transaction deltas A,B,C,D,E to combine these into a single delta ABCDE. This is called delta compression; BIND does not support this (ISC engineers decided that it was a bad idea because of the problem that will shortly be explained below), but some other DNS implementations do.

Consider an environment where there are several layers of authoritative servers handling zone propagation (typically this will be where there is one ultimate primary updating several distribution primaries, who in their turn are providing incremental updates to consumer-facing authoritative servers). It's typical in this type of environment to have secondaries that can update from multiple primaries. It's also possible to run these servers on a diverse range of DNS software (typically this is a design decision to increase resilience to failures that affect just one implementation).

In this situation, it can happen that a secondary that has received updates A,B requests an IXFR from another server that has updates A, BC, D, E in its journal file. (During its zone update history, transactions B and C have been compressed into transaction BC that this server holds.)

The secondaries' IXFR request cannot be fulfilled, because although the authoritative server is updated as far as E, it does not have a separate transaction in its journal file from which to provide update C.