Host a full virtual infrastructure behind a single public IP on Kimsufi [Part Two]

Disclaimer: this goes at a reasonably fast pace. As someone who has or is considering a Kimsufi server, I assume you are technically competent and understand the fundamentals of virtualisation and basic configuration of a firewall such as OPNSense. Naturally, perform these steps at your own risk.

So, let’s get down to business. First off, you’ll need to provision your new server at Kimsufi, and Choose Proxmox 4.X as your operating system. This could take up to an hour to run through the install, but you’ll be sent an email once it’s done with your root account credentials.

SSH onto the box and edit /etc/network/interfaces – it’s important to do this via SSH as the Proxmox Web Interface can and will overwrite any changes. Don’t, after this point, make any changes via the Web UI.

Leave VMBR0 and VMBR1 alone, we’ll be creating two new bridged network interfaces. VMBR2 will become our WAN network for the purposes of our OPNSense VM to connect to the outside world, and VMBR3 will be our local network. Change the addresses as appropriate to fit your own requirements. The comments below pretty much explain the purpose of each command. Essentially, NAT port 22 and 9006 back to the host, pass all other traffic through to our OPNSense VM, which we’ll assign 10.11.110.2 as its WAN IP.

auto vmbr2
iface vmbr2 inet static
address 10.11.110.1
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0
post-up echo 1 > /proc/sys/net/ipv4/ip_forward

#ALLOW THIS INTERFACE TO NAT VIA VMBR0
post-up iptables -t nat -A POSTROUTING -s ‘10.11.110.0/24’ -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s ‘10.11.110.0/24’ -o vmbr0 -j MASQUERADE

#PRESERVE ACCESS TO SSH
#post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp –dport 22 -j DNAT –to 10.11.110.1:22
#post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp –dport 22 -j DNAT –to 10.11.110.1:22

#PRESERVE ACCESS TO HYPERVISOR WEB UI
#post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp –dport 8006 -j DNAT –to 10.11.110.1:8006
#post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp –dport 8006 -j DNAT –to 10.11.110.1:8006

#FORWARDING ALL OTHER TRAFFIC TO VIRTUAL ROUTER
#post-up iptables -t nat -A PREROUTING -i vmbr0 -j DNAT –to-destination 10.11.110.2
#post-down iptables -t nat -D PREROUTING -i vmbr0 -j DNAT –to-destination 10.11.110.2

auto vmbr3
iface vmbr3 inet static
address 10.11.111.1
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0

Save your changes (in Nano, it’s Ctrl + X, Y) and reboot the machine, when it comes up, you should notice nothing really different, but the new bridged interfaces will be available to use. You’ll have noticed the iptables rules are currently commented out, that’s for very good reason. The first time I tried this, I enabled the rules and the host stopped responding to ping – quite expected, but Kimsufi’s automated monitoring tools are not very happy with this at all, and rebooted the host, assuming it was down, a number of times before leaving it powered up into a recovery environment for me to ‘fix’. We’ll revisit these later.

Next up, we’ll create the Router VM. For this example, I’m using OPNSense, but PFSense will perform just as well. I’m not going to run through the install of the OS itself – you’re grown up enough to do this, but I will say that you need to add an additional network interface, and ensure both virtual NICs are of the type Virtio.

When you first run your setup, you’ll be asked to assign your WAN and LAN interfaces, set their IP addresses etc. – refer to your Network Devices, and ensure you assign vmbr2 to the WAN, vmbr3 to the LAN. Make sure you set a static IP for both the WAN and LAN interfaces, according to our earlier setup. WAN would be 10.11. 110.2 and LAN would be 10.11.111.2. You can enable DHCP and once this is completed, you will be able to further configure the firewall to your requirements.

Your best bet to further configure the firewall is to open a reverse SSH tunnel to the VM, so you can access it over http. Your alternative route involves setting up a client OS on the hypervisor attached to VMBR2, so we’ll stick to the simple approach.

This command – substitute {host} for your Kimsufi Server’s IP will open your local port 9000 and forward to the Web UI for OPNSense:
ssh -N -p 22 root@{host} -L 127.0.0.1:9000:10.11.111.1:80

Browse to http://127.0.0.1:9000 assuming you’ve opened a tunnel as above, and you’ll be invited to run through an initial wizard – whilst it would seem like repetition, please go through this as it creates a number of automated Firewall rules for you. Once you’re done, ensure you’ve unticked both ‘Block Private Networks’ and ‘Block bogon networks’ as our WAN interface technically sits on a local IP range.

The only other rule you’ll need to setup is for Kimsufi’s benefit, we’ll enable ICMP (Ping) on the WAN interface so the automated tools know we’re ok. Once this is done, we’re ready to apply the changes to the IPTables rules we added to our /etc/network/interfaces file, so go ahead – SSH back in, uncomment the IPTables lines, save and reboot. You should now have a working OPNSense Firewall that can reach the outside world, open up ports and forward them internally, and even host a VPN server, which we’ll cover in a future post.

Where can this go next?

The traditional limitation of Kimsufi has now been overcome, you can expose internal services running on VMs to the outside world, you could use an Nginx reverse proxy to expose multiple websites via port 80, you can even host docker containers via an appropriate platform. The possibilities now, pretty much are endless.

Coming in Part Three – setting up a site to site link with another Kimsufi server. Check back soon.

Host a full virtual infrastructure behind a single public IP on Kimsufi [Part One]

Kimsufi is an inexpensive dedicated server platform that makes server infrastructure, in ‘proper’ datacentres, available to the masses – I wouldn’t recommend putting something into production on this platform owing to there being little to no support available and the hardware being of questionable specification. The types of machine on offer look to be ex-corporate desktops coming out of their refresh cycle, powerful by desktop standards, but many generations away from the current iterations.

I currently have 2 running Kimsufi servers, one hosted in Montreal, Canada with the other in Roubaix, France – I wanted to play with geographic replication and resiliency, taking lots of theoretical knowledge picked up in my day job and in my studies towards Microsoft certification.

One of the bigger considerations / limitations of the Kimsufi platform is that you are provided with a single public IPv4 address, which is associated with the machine itself, so in our case, the Proxmox hypervisor. I’d assume it’s to put you off using it for ‘proper’ uses, but also to upsell you into their SoYouStart servers instead, much better placed for production roles.

This makes it quite difficult to expose underlying virtual machines and their associated services as it is dedicated to the management of the host. The most widely discussed means of working around this involved using IPTABLES to forward traffic using NAT from the network interface to a given virtual machine hosted on an internal bridged network – whilst a perfectly valid approach, it proved inflexible to my needs, requiring lots of command line tweaks, and it’s less user friendly than a traditional management UI in a firewall such as PFSense or OPNSense – from my research nobody had put forward a better way, so I researched the capabilities of IPTABLES, and after a few times of knocking my machine off the network and having to wipe and start again, I managed to get something working that I could use.

I’ll soon be publishing a tutorial that is going to run you through the following:

  1. Configuring Proxmox virtual network interfaces and their subnets
  2. Installing a virtual router and passing all traffic through to this, whilst preserving direct access to the VM host
  3. Modifying the IPTable rules to pass all unsolicited traffic to the virtual router
  4. Using this virtual router to provide internet access and port forwarding to/from virtual machines hosted on the machine
  5. Establishing a site to site link over an encrypted VPN connection between an equivalent setup in the other geographic location
  6. Routing configuration to make the internal networks at each site available to the others

This setup makes it possible to play with a few interesting, more complex infrastructure concepts – Active Directory replication across sites, off-site backup targets, externally hosted services with access to local resources, the possibilities are endless. I’ll reiterate my previous warning – use this for production roles at your peril.

Part Two covering the basics, so steps 1 to 4, has now been published.

The road to Agile

agile_development_logo2Over the last five years I’ve been part of an IT team that’s gone through a period of expansion in line with the explosive growth of the business. Originally part of a team of two, between us we ran IT Support, and delivered various software development tasks to increase or improve the capabilities of the business. There was no concept of prioritisation, forward planning or even robust requirements gathering – we didn’t need it, we just got on with the task, and it worked. It was a symptom of a then small business, needing to adapt and react quickly to improve.

Since 2011, incredible growth of the business led to two relocations, and an IT team of 2 becoming a team of 13 with the overall headcount growing from 30 to over 250 in the same time. As the company changed, so did the management structure; two business principles became a board of directors and a senior team below this. The number of stakeholders, each with their own unique set of demands from IT increased exponentially.

My role changed from support, to developer, to team lead, to manager and ultimately, I’ve been charged with delivering change to bring our department in line with the expectations of a business of this size, providing better visibility, better prioritisation and better management of expectations. The real challenge is going to be delivering this whilst still supporting the business in maintaining it’s biggest competitive edge – continual improvement and quick reaction to changing needs.

The challenge has become putting forward the right blend of requirements gathering, rigorous specifications and test plans whilst still being able to quickly get tasks off the ground and delivered in a timely manner.

Researching the various Software Development methodologies led me to Agile, adopted successfully by thousands across the world, it was clear this was something we should look at. Full Agile / Scrum would require fundamental changes to the way we work and the way we interact with the business, and I don’t think they’d respond well to ‘shock and awe’. Getting a better understanding of User Stories and Acceptance Criteria showed me that this is where we’d get the greatest impact, whilst maintaining our strong working relationship with the stakeholders. User Stories, whilst simplistic in nature, were an incredibly easy way to convey the intent and overall goal of any task.

I once read that they key to Agile was making it work for you, there were no right or wrong, nor hard and fast rules – Agile is a collective ideal, and so I felt comfortable in not putting forward adopting the full workflow.

With Management briefed, and a new work request system currently under construction, only time will tell if this yields the results I’ve been tasked to achieve…

Copyright © James Coleman-Powell, 2016