-
Notifications
You must be signed in to change notification settings - Fork 974
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🐛 BUG: Fallback lighthouse setup? #1220
Comments
Hi @azukaar - I'm not totally sure I understand your issue. First, let's talk about your configuration:
In this configuration, you've defined two Lighthouse hosts, each with a single underlay IP address. The host will attempt to connect to 192.168.201.2 using a public IPv4 address and will attempt to connect to 192.168.201.1 using its private (LAN) IPv4 address. A couple notes:
Because of the final point, we do not recommend that hosts talk to Lighthouses over LAN / private IPs. This is because the packets arriving at the Lighthouse will have a source field pointing to the private IP address of the node, which is only routable to other hosts on the same LAN. When Lighthouses are instead configured to talk over the public Internet, the source field will contain the public IPv4 address of the host's network, giving hosts on other networks the best chance to handshake with it. Because hosts still self-report IP addresses assigned to them, the private IP address will still be available to the Lighthouse for nodes that are able to communicate directly over the LAN. With that in mind, can you clarify exactly what you're trying to achieve? |
@azukaar Yes, I think you've got it - relays cannot be used to connect to Lighthouses. If I understand your goals correctly, you have a main Lighthouse that operates on the Internet. You have a secondary Lighthouse that is on a LAN with some of your hosts. If you lose Internet connection, you wish for the LAN devices to use the local Lighthouse. You also wish for external nodes to access the Lighthouse directly as a peer (i.e. non-Lighthouse traffic.) One thing you could try is adding two IP addresses for your LAN Lighthouse to the rest of your nodes' configs: (a) a public and routable IP address/port for the LAN Lighthouse (i.e. setup port forwarding in your NAT, so it is also a Lighthouse on the Internet), (b) a local IP address for the LAN Lighthouse In this case, nodes external to the local Lighthouse will access it over the public Internet. I believe nodes local to it will also attempt to route this way, preferentially. However, if the Internet route to the Lighthouse is lost, they should try to failover to the private IPv4 address: https://nebula.defined.net/docs/config/preferred-ranges/#how-nebula-orders-underlay-ip-addresses-it-learns-about |
Thanks. Yes I thought about it, but it's not possible because of CGNAT. Also it's a bit of a waste of bandwith to relay connection to a node that is local to your network. Essentially the use case is, in a basic home server situation, I have
The phone uses the lighthouse to communicate with the home server (with relay). But when I'm home, it should be able to connect to the home server directly, without manual intervention Ideally, what I think what would be good would be
In that scenario then if the lighthouse is offline, you wouldn't use complete connectivity Something else that would be potentially useful is an option for fallback lighthouse
And basically this would cause a client to act as a lighthouse until it is able to connect to a lighthouse itself. With those 2 features, it would be possible to design much more reliable and reliant networks that are able to react to loss of connectivity better EDIT: How about an option that allows lighthouse to ping other lighthouses? Isn't it odd that lighthouse and clients are completely considered to be 2 separate entities? Not every clients are lighthouses, but a lighthouse is still a client, why doesn't it attempt to establish connectivity to the rest of the network? If I have a Nebula network with only 2 lighthouses, they should still be able to communicate together, but because they are lighthouses, they will ignore each others. |
This is how Nebula already behaves today.
Again, what you're asking for already exists in Nebula today. As long as Lighthouses list each other in their
I don't understand the fallback Lighthouse proposal. Maybe you can describe how you envision that working in more detail? |
What version of
nebula
are you using? (nebula -version
)1.8.2
What operating system are you using?
Describe the Bug
When using 2 lighthouse, with a static_host_map, the selected IP in this map seems to override every other routing logic, including the relay. I have a setup where I have 2 lighthouses: one for my local network, and one for remote access. It looks roughly like this:
In this configuration, if I remove the static_host_map and the lighthouse entry for
192.168.201.1
(so having a single lighthouse) it works perfectly, and 192.168.201.1 (who then connect to the lighthouse too) is accessible.On the other hand, with the 2 lighthouses, because 192.168.201.1 is local network only, it becomes inaccessible from outside. I am guessing that
static_host_map
is forcing the client to resolve192.168.201.1
aslocal-ip
192.168.201.1
does not handshake with192.168.201.2
(the relay) anymoreWhich now comes to the question / potential issue: how to have a fallback lighthouse for specific subnet? (Eg. in this case a local network lighthouse in case internet connection is lost and the public lighthouse becomes inaccessible).
In that scenario for some reason 192.168.201.1 does not ping 192.168.201.2 (even thought it could) but 192.168.201.2 tries to ping 192.168.201.1 even thought they are both lighthouses
Thanks for your help
Logs from affected hosts
logs from 192.168.201.1
logs from 192.168.201.2
Config files from affected hosts
The text was updated successfully, but these errors were encountered: