How to ensure you are using Private Networking with your Kubernetes cluster on Digital Ocean.
While learning how to launch a Kubernetes cluster I read a blog post where as part of their instructions for setting up a Kubernetes cluster on Digital Ocean they enabled Private Networking on their droplets. They then did nothing to actually tell Kubernetes to use private networking. It then took me a bit of reading to figure out how to fix that, so hopefully this post makes that problem easier to search for.
I did this before Digital Ocean introduced their own Kubernetes service so the information isn't necessarily relevant any more in regards to running a Kubernetes cluster on Digital Ocean, but it will still teach you something about configuring Kubernetes and about what enabling Private Networking does in Digital Ocean.
The information here should still be relevant. The information about which adaptor you reference might not reflect how your platform of choice works but the ideas should be the same.
Using Private Networking means that your droplets/instances/VMs are able to communicate via the providers internal infrastructure rather than over the internet. The result is that communication is both faster and generally more secure.
A default Digital Ocean droplet has a publicly accessibly IP address via eth0. If you enable Private Networking, your droplet will now have an eth1. This is the adaptor through which your droplet can communicate directly with other droplets through the data center's internal communication infrastructure rather than through the public internet.
If you just run kubeadm init with default parameters it will detect eth0 and use that for everything. This will work just fine on Digital Ocean and if you pass the IP address assigned to that droplet to your worker nodes they will connect to the master just fine. If you want to use Private Networking you need to give it some specific instructions. Here's the command I ran using Ansible - note the references to eth0 and eth1:
kubeadm init --apiserver-advertise-address {{ ansible_eth1.ipv4.address }} --apiserver-cert-extra-sans {{ ansible_eth0.ipv4.address }}
The key takeaway here is: --apiserver-advertise-address should be set to the private network's IP address. This configures the cluster to listen for worker connections via private networking. Additionally, running kubeadm token create --print-join-command will now create a join command that uses that same IP address. Pass this to your worker nodes and you're good to go!
The --apiserver-cert-extra-sans parameter should be set to the public network's IP address. This allows you to expose your services to the public internet.
In my last blog post I explained how to run a multi-node Kubernetes cluster in Vagrant and that knowledge ended up being directly transferable to this problem.