Host a full virtual infrastructure behind a single public IP on Kimsufi [Part Two]

Disclaimer: this goes at a reasonably fast pace. As someone who has or is considering a Kimsufi server, I assume you are technically competent and understand the fundamentals of virtualisation and basic configuration of a firewall such as OPNSense. Naturally, perform these steps at your own risk.

So, let’s get down to business. First off, you’ll need to provision your new server at Kimsufi, and Choose Proxmox 4.X as your operating system. This could take up to an hour to run through the install, but you’ll be sent an email once it’s done with your root account credentials.

SSH onto the box and edit /etc/network/interfaces – it’s important to do this via SSH as the Proxmox Web Interface can and will overwrite any changes. Don’t, after this point, make any changes via the Web UI.

Leave VMBR0 and VMBR1 alone, we’ll be creating two new bridged network interfaces. VMBR2 will become our WAN network for the purposes of our OPNSense VM to connect to the outside world, and VMBR3 will be our local network. Change the addresses as appropriate to fit your own requirements. The comments below pretty much explain the purpose of each command. Essentially, NAT port 22 and 9006 back to the host, pass all other traffic through to our OPNSense VM, which we’ll assign 10.11.110.2 as its WAN IP.

auto vmbr2
iface vmbr2 inet static
address 10.11.110.1
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0
post-up echo 1 > /proc/sys/net/ipv4/ip_forward

#ALLOW THIS INTERFACE TO NAT VIA VMBR0
post-up iptables -t nat -A POSTROUTING -s ‘10.11.110.0/24’ -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s ‘10.11.110.0/24’ -o vmbr0 -j MASQUERADE

#PRESERVE ACCESS TO SSH
#post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp –dport 22 -j DNAT –to 10.11.110.1:22
#post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp –dport 22 -j DNAT –to 10.11.110.1:22

#PRESERVE ACCESS TO HYPERVISOR WEB UI
#post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp –dport 8006 -j DNAT –to 10.11.110.1:8006
#post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp –dport 8006 -j DNAT –to 10.11.110.1:8006

#FORWARDING ALL OTHER TRAFFIC TO VIRTUAL ROUTER
#post-up iptables -t nat -A PREROUTING -i vmbr0 -j DNAT –to-destination 10.11.110.2
#post-down iptables -t nat -D PREROUTING -i vmbr0 -j DNAT –to-destination 10.11.110.2

auto vmbr3
iface vmbr3 inet static
address 10.11.111.1
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0

Save your changes (in Nano, it’s Ctrl + X, Y) and reboot the machine, when it comes up, you should notice nothing really different, but the new bridged interfaces will be available to use. You’ll have noticed the iptables rules are currently commented out, that’s for very good reason. The first time I tried this, I enabled the rules and the host stopped responding to ping – quite expected, but Kimsufi’s automated monitoring tools are not very happy with this at all, and rebooted the host, assuming it was down, a number of times before leaving it powered up into a recovery environment for me to ‘fix’. We’ll revisit these later.

Next up, we’ll create the Router VM. For this example, I’m using OPNSense, but PFSense will perform just as well. I’m not going to run through the install of the OS itself – you’re grown up enough to do this, but I will say that you need to add an additional network interface, and ensure both virtual NICs are of the type Virtio.

When you first run your setup, you’ll be asked to assign your WAN and LAN interfaces, set their IP addresses etc. – refer to your Network Devices, and ensure you assign vmbr2 to the WAN, vmbr3 to the LAN. Make sure you set a static IP for both the WAN and LAN interfaces, according to our earlier setup. WAN would be 10.11. 110.2 and LAN would be 10.11.111.2. You can enable DHCP and once this is completed, you will be able to further configure the firewall to your requirements.

Your best bet to further configure the firewall is to open a reverse SSH tunnel to the VM, so you can access it over http. Your alternative route involves setting up a client OS on the hypervisor attached to VMBR2, so we’ll stick to the simple approach.

This command – substitute {host} for your Kimsufi Server’s IP will open your local port 9000 and forward to the Web UI for OPNSense:
ssh -N -p 22 root@{host} -L 127.0.0.1:9000:10.11.111.1:80

Browse to http://127.0.0.1:9000 assuming you’ve opened a tunnel as above, and you’ll be invited to run through an initial wizard – whilst it would seem like repetition, please go through this as it creates a number of automated Firewall rules for you. Once you’re done, ensure you’ve unticked both ‘Block Private Networks’ and ‘Block bogon networks’ as our WAN interface technically sits on a local IP range.

The only other rule you’ll need to setup is for Kimsufi’s benefit, we’ll enable ICMP (Ping) on the WAN interface so the automated tools know we’re ok. Once this is done, we’re ready to apply the changes to the IPTables rules we added to our /etc/network/interfaces file, so go ahead – SSH back in, uncomment the IPTables lines, save and reboot. You should now have a working OPNSense Firewall that can reach the outside world, open up ports and forward them internally, and even host a VPN server, which we’ll cover in a future post.

Where can this go next?

The traditional limitation of Kimsufi has now been overcome, you can expose internal services running on VMs to the outside world, you could use an Nginx reverse proxy to expose multiple websites via port 80, you can even host docker containers via an appropriate platform. The possibilities now, pretty much are endless.

Coming in Part Three – setting up a site to site link with another Kimsufi server. Check back soon.

No Comments

Post a reply

Copyright © James Coleman-Powell, 2016