Provisioned nodes end up with two default routes causing intermittent connectivity.

When provisioning nodes through the ICN scripts the newly created nodes boot up and run /var/lib/cloud/scripts/per-boot/set_provider_network.sh which adds the PROVIDER_NETWORK_GATEWAY as a default route to allow internet connectivity. However, on the Pod 20 servers there is already a default route that is connected to the BS_DHCP_INTERFACE so you end up with two default routes. I often have intermittent Internet connectivity in these pods because of this.  Removing the route connected to the PROVIDER_NETWORK_GATEWAY fixes this for me and the Internet connectivity is much more stable.

root@pod20-node5:/var/lib/cloud/scripts/per-boot# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.10.200.1 0.0.0.0 UG 0 0 0 eno1
default _gateway 0.0.0.0 UG 0 0 0 enp24s0f1
10.10.200.0 0.0.0.0 255.255.255.0 U 0 0 0 eno1
10.10.200.0 0.0.0.0 255.255.255.0 U 0 0 0 eno2
172.22.0.0 0.0.0.0 255.255.255.0 U 0 0 0 enp24s0f0
172.31.1.0 0.0.0.0 255.255.255.0 U 0 0 0 enp24s0f1

root@pod20-node5:/var/lib/cloud/scripts/per-boot# route del default gw _gateway enp24s0f1