It looks like ipv6 is not configured correctly in libvirt. Restarting libvirt/guests at least made it stop to work completely (it fails immediately instead of hanging), so at least it's fast again... I suppose I should figure out how to fix the ipv6 configuration but I'm lazy :)