DHCPv6 "not our server identifier", discarding renews

Question:

Why am I seeing lots of "Discarding Renew ... not our server identifier" messages in logs after a server hardware (or OS) upgrade?

Example:

Renew message from fe80::250:56ff:fe32:202 port 546, transaction ID 0x17D67300
Discarding Renew from fe80::250:56ff:fe32:202; not our server identifier (CLIENTID 00:01:00:01:1a:01:2d:7c:00:50:56:32:02:02, SERVERID 00:01:00:01:19:fd:aa:20:00:50:56:2f:de:8a, server DUID 00:01:00:01:1b:10:e6:be:00:50:56:2f:de:8a)
Renew message from fe80::250:56ff:fe32:202 port 546, transaction ID 0x17D67300
Discarding Renew from fe80::250:56ff:fe32:202; not our server identifier (CLIENTID 00:01:00:01:1a:01:2d:7c:00:50:56:32:02:02, SERVERID 00:01:00:01:19:fd:aa:20:00:50:56:2f:de:8a, server DUID 00:01:00:01:1b:10:e6:be:00:50:56:2f:de:8a)
Rebind message from fe80::250:56ff:fe32:202 port 546, transaction ID 0xF2709A00
Reply NA: address 2000::37a to client with duid 00:01:00:01:1a:01:2d:7c:00:50:56:32:02:02 iaid = 1446117890 valid for 120 seconds
Sending Reply to fe80::250:56ff:fe32:202 port 546

Answer:

The DHCPv6 server DUID by default is "LLT": link layer address and time. While the address should stay the same barring any hardware changes, the four-byte timestamp changes every second. Note from the sample above:

   SERVERID 00:01:00:01:**19:fd:aa:20** :00:50:56:2f:de:8a
server DUID 00:01:00:01::00:50:56:2f:de:8a

The server DUID is built the first time the server runs and is then stored in the lease file, so if you delete your lease file you get a new one. If you keep your lease file intact, you keep using the old one. Above we see the client presenting a "SERVERID" associated with its previous lease, while the server DUID differs in the four-byte timestamp and the six-byte MAC (link layer) address is the same.

So most likely here, the server has lost its lease file. This is a good example of why the lease file should be maintained in permanent storage, e.g., a hard drive. If your OS deleted it in an upgrade, that could be a bug in its upgrade procedure.

Without the lease file, every existing lease will fail to renew, even if the client-provided SERVERID does happen to match the server DUID.

In due time the client will give up on its renewal attempts and will solicit a new lease. The problem will go away eventually, but likely at the cost of most clients having changed IP addresses.