Experiment Nebula Mesh - Part 1

Saturday, July 16, 2022 · 5 minutes · 917 words

Is it possible to use the public network, namely Internet, to make 2 machines communicate securely ? And if possible something easier to install and configure than OpenVpn ?

A few months ago, I heard about Nebula from Defined and wanted to test it : one single binary, some certificates and as secure as it can be, that sounded promising !

So here we are trying to simulate a mesh network between 2 virtual machines using Vagrant on my Ubuntu Linux distribution 20.04 with Virtualbox already installed.

Vagrant

Vagrant from Hashicorp is the tool I use to manage virtual machines.

1sudo apt install -y vagrant

I configure Vagrant to have 2 virtual machines with Debian 11, named respectively boxA and boxB.

To make it simple, I use my host network interface to link the 2 virtual machines (so eth0 in my case) and use the available DHCP of my local network.

 1cat <<EOF > Vagrantfile
 2Vagrant.configure("2") do |config|
 3    config.vm.define "boxA" do |boxA|
 4        boxA.vm.box = "generic/debian11"
 5        boxA.vm.network "public_network", bridge: "eth0"
 6    end
 7    config.vm.define "boxB" do |boxB|
 8        boxB.vm.box = "generic/debian11"
 9        boxB.vm.network "public_network", bridge: "eth0"
10    end
11end
12EOF

Now I can launch the 2 machines with a single command :

1vagrant up

Nebula

Installing nebula is as simple as downloading the binaries from github. I choose nebula-linux-amd64.tar.gz to comply to my configuration.

1wget https://github.com/slackhq/nebula/releases/download/v1.6.0/nebula-linux-amd64.tar.gz

After downloading, I uncompress the archive :

tar xf nebula-linux-amd64.tar.gz

I now have 2 binaries : nebula and nebula-cert.

  • nebula will be copied on the 2 virtual machines. This is the main binary who will establish the bridge between the machines.
  • nebula-cert will be used to generate all needed certificates :
1./nebula-cert ca -name "ACME, Inc"
2./nebula-cert sign -name "boxA" -ip "192.168.168.100/24"
3./nebula-cert sign -name "boxB" -ip "192.168.168.200/24"

I choose 192.168.168.* as my mesh network IP addresses : .100 for boxA and .200 for boxB.

The results of the commands are this list of files :

1ca.crt
2ca.key
3boxA.crt
4boxA.key
5boxB.crt
6boxB.key

To make it more simple to copy the files to the virtual machines, I create a folder per machine and copy the needed files :

1mkdir boxA boxB
2mv boxA.* boxA && mv boxB.* boxB

Now I want to generate a Nebula configuration file, that will be common for the 2 machines. I need to get their IP addresses on my local network (those assigned by my DHCP server). I know that vagrant will use the eth1 interface inside the machines, so I use some magic shell script :

First on boxA :

1boxAip=$(vagrant ssh boxA --no-tty -c "ip address show eth1| grep 'inet ' | sed -e 's/^.*inet //' -e 's/\/.*$//'" | tr -d '\r')

Then on boxB :

1boxBip=$(vagrant ssh boxB --no-tty -c "ip address show eth1| grep 'inet ' | sed -e 's/^.*inet //' -e 's/\/.*$//'" | tr -d '\r')

The magic is that $boxAip and $boxBip variables now contain the wanted IPs. Let use them inside the Nebula config in the static_host_map section :

 1cat <<EOF > config.yml
 2pki:
 3  ca: /etc/nebula/ca.crt
 4  cert: /etc/nebula/host.crt
 5  key: /etc/nebula/host.key
 6static_host_map:
 7    "192.168.168.100": ["$boxAip:4242"]
 8    "192.168.168.200": ["$boxBip:4242"]
 9lighthouse:
10  am_lighthouse: false
11listen:
12  host: 0.0.0.0
13  port: 4242
14punchy:
15  punch: true
16tun:
17  disabled: false
18  dev: nebula1
19  drop_local_broadcast: false
20  drop_multicast: false
21  tx_queue: 500
22  mtu: 1300
23  routes:
24  unsafe_routes:
25logging:
26  level: info
27  format: text
28firewall:
29  conntrack:
30    tcp_timeout: 12m
31    udp_timeout: 3m
32    default_timeout: 10m
33    max_connections: 100000
34  outbound:
35    - port: any
36      proto: any
37      host: any
38  inbound:
39    - port: any
40      proto: any
41      host: any
42EOF

It’s time to configure the machines. First, let send all the files to each machine :

1vagrant upload ./boxA /tmp/ boxA
2vagrant upload ./boxB /tmp/ boxB
3vagrant upload ca.crt /tmp/ boxA
4vagrant upload ca.crt /tmp/ boxB
5vagrant upload config.yml /tmp/ boxA
6vagrant upload config.yml /tmp/ boxB
7vagrant upload nebula /tmp/ boxA
8vagrant upload nebula /tmp/ boxB

Now I ssh in each machine and continue the configuration :

First, boxA :

1vagrant ssh boxA

Inside boxA, I move the files in their respective location and finally launch Nebula in the background before quitting boxA :

 1sudo mkdir /etc/nebula
 2sudo mv /tmp/config.yml /etc/nebula/config.yml
 3sudo mv /tmp/ca.crt /etc/nebula/ca.crt
 4sudo mv /tmp/boxA.crt /etc/nebula/host.crt
 5sudo mv /tmp/boxA.key /etc/nebula/host.key
 6sudo mkdir /opt/nebula 
 7sudo mv /tmp/nebula /opt/nebula/
 8chmod +x /opt/nebula/nebula
 9cd /opt/nebula
10sudo ./nebula -config /etc/nebula/config.yml &
11exit

Let’s configure boxB,

1vagrant ssh boxB

Then I move all files and launch Nebula :

 1sudo mkdir /etc/nebula
 2sudo mv /tmp/config.yml /etc/nebula/config.yml
 3sudo mv /tmp/ca.crt /etc/nebula/ca.crt
 4sudo mv /tmp/boxB.crt /etc/nebula/host.crt
 5sudo mv /tmp/boxB.key /etc/nebula/host.key
 6sudo mkdir /opt/nebula 
 7sudo mv /tmp/nebula /opt/nebula/
 8chmod +x /opt/nebula/nebula
 9cd /opt/nebula
10sudo ./nebula -config /etc/nebula/config.yml &
11exit

Ok, at this moment, I have 2 virtual machines, with a Nebula network between them. How am I going to verify that it is really working ?

Well if I can ping boxB from boxA, it will mean that all is ok, so let’s found out :

1vagrant ssh boxA -c "ping 192.168.168.200"

Hourra ! It is working. Of course, I can do it on the other side :

1vagrant ssh boxB -c "ping 192.168.168.100"

And that’s all folks for part 1 !

Next, I’ll try to use this configuration to execute a MySQL server on one machine, and access it from a MySQL client on the other machine. And to make it a little more difficult, I’ll use Podman as the container engine to execute this client and server.

Technical Linux