If I am travelling & need to pull data from home, it would go as Rohtak -> s01.bom / s02.bom -»> me. I can access the storage server at home over IAXN (primary ISP) because it has public IP but not over Siti broadband (secondary ISP) because that is behind CGNAT.This adds firewall management more complex. Both my ISP lag on the IPv6 front & hence I have to put fallback support for IPv4. All this network was completely IPv6 between all servers except my home.My primary ISP irritatingly still caps at 1TB/month (though they reset the cap on request I cannot pump 500-600GB worth of data over it without looking at the cap) while my secondary ISP Siti has no caps but lower speeds (to keep cost low). I had Gitlab CI jobs to tweak OSPF cost on the virtual circuits based on traffic for the month.Fibre cut can bring down a link (as these are retail GPON connections with no path protection) and hence dynamic protocols had to support link switchover. Most of this network is stable except the most critical part i.e my home office where I sit.I wanted easy dynamic routing and hence I ran wireguard with the “table = off” command & simply ran routing on top of it with OSPF + iBGP + route reflectors with FRR.It’s extremely complicated to setup and maintain.While this setup works, it has a few issues: This also gave me basic features like running cameras are home which feeds into the Frigate instance in Mumbai for motion detection-based recording, monitoring these cameras & other device uptime using the uptime-kuma instance in Ashburn etc. This setup ensured private network connectivity with encryption so that I can have GitLab runners spread around based on available CPU load and those runners would speak to database/storage servers securely without having to deal with encryption on per project/app basis. I would have a home node here in Rohtak connected to two different servers in Mumbai over two different ISPs (via policy-based routing), those two Mumbai nodes would maintain the site-to-site VPNs with a few servers in Europe & those servers further connect to a few servers in the US. These were not mesh but rather in a linear topology. Originally these used to be on OpenVPN and later I moved to wireguard.

I am running a site-to-site VPN for a long time between various servers located far away from each other. If you are not planning to deploy it right away, you can skip the post after the “Configs and setup” section. This will be a long post documenting the concept of mesh VPN, the problem it is solving as well as a working demo.

A rather long title but the post is about self-hosted open-source mesh VPN with IPv6 support and works with nodes behind CGNAT!

Ivpn open source