This is due to a kernel bug where the fact that a socket is marked non-blocking is ignored. It is reported that setting xfrm_larval_drop to 1 helps but this may have negative side effects.
For more information, see:
https://bugzilla.redhat.com/show_bug.cgi?id=427629
and
https://lkml.org/lkml/2007/12/4/260
xfrm_larval_drop can be set to 1 by the following procedure:
echo "1" > proc/sys/net/core/xfrm_larval_drop