Configure networking for Konvoy cluster
This section describes different networking components that come together to form a Konvoy networking stack. It assumes familiarity with Kubernetes networking.
A Service is an API resource that defines a logical set of pods and a policy by which to access them, and is an abstracted manner to expose applications as network services.
Kubernetes gives pods their own IP addresses and a single DNS name for a set of pods. Services are used as entrypoints to load-balance the traffic across the pods. A selector determines the set of Pods targeted by a Service.
For example, if you have a set of pods that each listen on TCP port
9191 and carry a label
app=MyKonvoyApp, as configured in the following:
apiVersion: v1 kind: Service metadata: name: my-konvoy-service namespace: default spec: selector: app: MyKonvoyApp ports: - protocol: TCP port: 80 targetPort: 9191
This specification creates a new
Service object named
"my-konvoy-service", that targets TCP port
9191 on any pod with the
Kubernetes assigns this Service an IP address. In particular, the
kube-proxy implements a form of virtual IP for Services of type other than
The name of a Service object must be a valid DNS label name.
A Service is not a Platform Service.
Service Topology is a mechanism in Kubernetes to route traffic based upon the Node topology of the cluster. For example, you can configure a Service to route the traffic to endpoints on specific nodes, or even based on the region or availability zone of the node’s location.
To enable this new feature in your Kubernetes cluster, use the feature gates
--feature-gates="ServiceTopology=true,EndpointSlice=true" flag. After enabling, you can control Service traffic routing by defining the
topologyKeys field in the Service API object.
In the following example, a Service defines
topologyKeys to be routed to endpoints only in the same zone:
apiVersion: v1 kind: Service metadata: name: my-konvoy-service namespace: default spec: selector: app: MyKonvoyApp ports: - protocol: TCP port: 80 targetPort: 9191 topologyKeys: - "topology.kubernetes.io/zone"
If the value of the
topologyKeys field does not match any pattern, the traffic is rejected.
EndpointSlices are an API resource that appears as a scalable and more manageable solution to network endpoints within a Kubernetes cluster. They allow for distributing network endpoints across multiple resources with a limit of 100 endpoints per EndpointSlice.
An EndpointSlice contains references to a set of endpoints, and the control plane takes care of creating EndpointSlices for any Service that has a selector specified. These EndpointSlices include references to all the pods that match the Service selector.
Like Services, the name of a EndpointSlice object must be a valid DNS subdomain name.
In this example, here’s a sample EndpointSlice resource for the example Kubernetes Service:
apiVersion: discovery.k8s.io/v1beta1 kind: EndpointSlice metadata: name: konvoy-endpoint-slice namespace: default labels: kubernetes.io/service-name: my-konvoy-service addressType: IPv4 ports: - name: http protocol: TCP port: 80 endpoints: - addresses: - "192.168.126.168" conditions: ready: true hostname: ip-10-0-135-39.us-west-2.compute.internal topology: kubernetes.io/hostname: ip-10-0-135-39.us-west-2.compute.internal topology.kubernetes.io/zone: us-west2-b
DNS for Services and Pods
Every new Service object in Kubernetes gets assigned a DNS name. The Kubernetes DNS component schedules a DNS name for the pods and services created on the cluster, and then the Kubelets are configured so containers can resolve these DNS names.
Considering previous examples, assume there is a Service named
my-konvoy-service in the Kubernetes namespace
default. A Pod running in namespace
default can look up this service by performing a DNS query for
my-konvoy-service. A Pod running in namespace
kommander can look up this service by performing a DNS query for
In general, a pod has the following DNS resolution:
Similarly, a service has the following DNS resolution:
You can find additional information about all the possible record types and layout here.
Ingress is an API resource that manages external access to the services in a cluster through HTTP or HTTPS. It offers name-based virtual hosting, SSL termination and load balancing when exposing HTTP/HTTPS routes from outside to services in the cluster.
The traffic policies are controlled by rules as part of the Ingress definition. Each rule defines the following details:
An optional host to which apply the rules.
A list of paths or routes which has an associated backend defined with a Service
name, a port
A backend is a combo of a Service and port names, or a custom resource backend defined as a CRD. Consequently HTTP/HTTPS requests to the Ingress that matches the host and path of the rule are sent to the listed backend.
An example of an Ingress specification is:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: konvoy-ingress namespace: default annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: rules: - http: paths: - path: /path pathType: Prefix backend: service: name: my-konvoy-service port: number: 80
In Kommander, you can expose services to the outside world using Ingress objects.
In contrast with the controllers in the Kubernetes control plane, Ingress controllers are not started with a cluster so you need to choose the desired Ingress controller.
An Ingress controller has to be deployed in a cluster for the Ingress definitions to work.
Kubernetes as a project currently supports and maintains GCE and nginx controllers.
These are four of the most known Ingress controllers:
HAProxy Ingress is a highly customizable community-driven ingress controller for HAProxy.
NGINX offers support and maintenance for the NGINX Ingress Controller for Kubernetes.
Traefik is a fully featured Ingress controller (Let’s Encrypt, secrets, http2, websocket), and has commercial support.
Ambassador API Gateway EXPERIMENTAL is an Envoy based Ingress controller with community and commercial support.
Traefik deploys by default as a well-suited Ingress controller.
NetworkPolicy is an API resource that controls the traffic flow at port level 3 or 4, or at the IP address level. It enables defining constraints on how a pod communicates with various network services such as
A Pod can be restricted to talk to other network services through a selection of the following identifiers:
Namespaces that have to access. There can be pods that are not allowed to talk to other namespaces.
Other allowed IP blocks regardless of the node or IP address assigned to the targeted Pod.
Other allowed Pods.
An example of a NetworkPolicy specification is:
apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: network-konvoy-policy namespace: default spec: podSelector: matchLabels: role: db policyTypes: - Ingress - Egress ingress: - from: - ipBlock: cidr: 172.17.0.0/16 except: - 172.17.1.0/24 - namespaceSelector: matchLabels: app: MyKonvoyApp - podSelector: matchLabels: app: MyKonvoyApp ports: - protocol: TCP port: 6379 egress: - to: - ipBlock: cidr: 10.0.0.0/24 ports: - protocol: TCP port: 5978
As shown in the example, when defining a pod or namespace based NetworkPolicy, you use a selector to specify what traffic is allowed to and from the Pod(s).
Adding entries to Pod /etc/hosts with HostAliases
The Pod API resource definition has a
HostAliases field that allows adding entries to the Pod’s container
/etc/hosts file. This field overrides the hostname resolution when DNS and other options are not applicable.
For example, to resolve
10.1.2.3, configure the
HostAliases values as follows:
apiVersion: v1 kind: Pod metadata: name: hostaliases-konvoy-pod spec: restartPolicy: Never hostAliases: - ip: "127.0.0.1" hostnames: - "foo.node.local" - "bar.node.local" - ip: "10.1.2.3" hostnames: - "foo.node.remote" - "bar.node.remote" containers: - name: cat-hosts image: busybox command: - cat args: - "/etc/hosts"