[WIP] Probe: Better mapping of NATted connections #3451
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In the case targeted, Kubernetes uses a bridge named 'cbr0'. The 'docker0' bridge should be locally-scoped in most cases, and exceptionally users can turn that off with --probe.docker.bridge=""
Deal separately with SRC vs DST mapping. Includes detailed rationale.
- Add SRC and DST NAT flag in status. - The port number was wrong from host1's perspective - it should see the mapped port on host2 not the real port inside the container. - Add nodes at the other end of the connection, like the real probe does. - Pass the local host name to MakeEndpointNodeID as it would be on the real probe.
rade
reviewed
Nov 30, 2018
| // Don't add the bridge in Kubernetes since container IPs are global and | ||
| // shouldn't be scoped | ||
| if flags.dockerBridge != "" && !flags.kubernetesEnabled { | ||
| if flags.dockerBridge != "" { |
There was a problem hiding this comment.
This is an improvement for when we have "unmanaged" containers running on a k8s host. However, it will also result in
-
spurious error messages when docker isn't running on a k8s host (as will happen by default on some future k8s releases)
-
incorrect treatment of the docker bridge as local if it is in fact the bridge used by k8s
We could avoid the first one by only erroring when the bridge name has been specified on the command line (rather than left to the default 'docker0' value).
Is the 2nd problem real?
There was a problem hiding this comment.
- I don't think it is practical to use the
docker0bridge as the Kubernetes bridge nowadays except on a single-node cluster.
I think Flannel used to connect one machine's Docker bridge to another, but nowadays that is driven via CNI with its own bridge. And Docker's own overlay networking will create a different bridge for each network.
rade
reviewed
Nov 30, 2018
rade
reviewed
Nov 30, 2018
rade
reviewed
Nov 30, 2018
probe/endpoint/nat.go
Outdated
|
|
||
| All of the above can be satisfied by these rules: | ||
| For SRC_NAT either add NAT orig source as a copy of NAT reply destination | ||
| or add NAT reply destination as a copy of NAT original source |
There was a problem hiding this comment.
why do we not need to replace adjacencies here?
rade
reviewed
Nov 30, 2018
bboreham
changed the title
Matching up examples to rules to code was hard, so I added letters (A), (B) etc. This helped to track down various inconsistencies. Now SRC and DST NAT are symmetrical, and the code is quite similar, apart from replacing the source vs destination of an adjacency.
|
@bboreham What's the status of this PR? Is there something blocking you from proceeding / merging it? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Now that we get NAT information via netlink we have a flag to distinguish
SRC_NATfromDST_NAT, so we can do a better job of mapping connections. See extensive comment innat.gofor explanation of the approach.One example how we get a better result is that incoming connections to a Kubernetes NodePort are visible with their remote IP address - previously they did not get mapped. Unfortunately, when using an AWS ELB the source endpoint is excluded because its address is on the same network as the host where the probe runs.
Also stop special-casing the Docker bridge in Kubernetes mode. Most often, Kubernetes uses a bridge named
cbr0so the 'docker0' bridge should be locally-scoped, and exceptionally users can turn that off with--probe.docker.bridge=""