Kubernetes Universal Declarative Operator (KUDO) provides a declarative approach to building production-grade Kubernetes operators.
- KUDO is built to help manage day 2 operations of services on Kubernetes, including stateful services, through the management of operators.
- KUDO is built to support upgrades of services along with backup, recovery, and the needs of observability.
- KUDO is integrated into Kommander and does not require any initialization via the KUDO CLI
The following articles give an overview of KUDO:
It is strongly recommended to view the KUDO Zookeeper Documentation which covers the KUDO Zookeeper Operator in-depth. This document covers the integration aspects of KUDO Zookeeper Operator with D2iQ Kommander.
KUDO Zookeeper is located in the Kommander Catalog.
- Create a Workspace
- Attach a Cluster to the Workspace
- Create a Project in the Workspace
- Select the created Project
- View the available Platform Services in the Project Catalog
- Select the version you’d like to deploy from the version drop-down, and then select Deploy.
From the Project Catalog, select the desired version of Zookeeper and click Deploy.
Here is an example of what should appear in the Kommander UI, the dialog is populated with appropriate defaults:
- The ID field above is referred to as the Zookeeper
- Detailed parameter descriptions and defaults are available for each version of KUDO Zookeeper Operator, in this case version
Deploy to install Zookeeper.
- Kommander will proceed to install
kudo-systemnamespace if it does not already exist. This is the equivalent of issuing
kubectl kudo initmanually on the CLI.
- KUDO will then install Zookeeper in the Project namespace created using Kommander.
- The deployment progression can be viewed by looking at the
KUDO Zookeeper Service Status
The Operator Plan Status provides an overview of the service.
- Get the instances running in the project namespace.
Here we have
zookeeper installed in the Kommander Project
test-project with namespace
$ kubectl kudo get instances -n test-project-zc6tc List of current installed instances in namespace "test-project-zc6tc": . └── zookeeper
- Get the Plan status from the KUDO Zookeeper Operator.
Here is a status summary of all plans after installation in the KUDO Zookeeper Operator
$ kubectl kudo plan status --instance zookeeper -n test-project-zc6tc Plan(s) for "zookeeper" in namespace "test-project-zc6tc": . └── zookeeper (Operator-Version: "zookeeper-3.4.14-0.3.0" Active-Plan: "deploy") ├── Plan deploy (serial strategy) [COMPLETE], last updated 2021-04-09 13:33:17 │ ├── Phase zookeeper (parallel strategy) [COMPLETE] │ │ └── Step deploy [COMPLETE] │ └── Phase validation (serial strategy) [COMPLETE] │ ├── Step validation [COMPLETE] │ └── Step cleanup [COMPLETE] └── Plan validation (serial strategy) [NOT ACTIVE] └── Phase connection (serial strategy) [NOT ACTIVE] ├── Step connection [NOT ACTIVE] └── Step cleanup [NOT ACTIVE]
deployplan status of
COMPLETEindicates that KUDO Zookeeper has deployed successfully and is healthy.
If any issues are encountered during the above, refer to the Troubleshooting section.
The complete list of KUDO Zookeeper Parameters can be found under detailed parameter descriptions.
The current parameters set can be retrieved using the kubectl command with the two additional tools:
To retrieve the current parameters, use the following command in the terminal with appropriate
INSTANCE value set:
INSTANCE=zookeeper; kubectl -n test-project-zc6tc get instances -o json | jq ".items | select(.metadata.name == \"$INSTANCE\") | .spec.parameters" | yq -e --yaml-output '.' > zookeeper-params.yml
The above command generates a file called
zookeeper-params.yml with the current values of all the parameters in effect for the
Parameters can be updated using arguments to the KUDO CLI.
Example: Increasing Zookeeper node counts
- Increase the number of nodes using the KUDO CLI:
NOTE: As mentioned in the ZooKeeper Getting Started Guide, a minimum of three servers are required for a fault tolerant clustered setup, and it is strongly recommended that you have an odd number of servers.
kubectl kudo update --instance zookeeper -p NODE_COUNT=5 -n test-project-zc6tc
- Monitor the KUDO Zookeeper deployment plan:
kubectl kudo plan status --instance zookeeper -n test-project-zc6tc
- Wait for the deployment plan to have a status of
When the deployment plan is
COMPLETE, there should be 5 nodes as seen by the number of pods running:
kubectl get pods -n test-project-zc6tc
NAME READY STATUS RESTARTS AGE zookeeper-zookeeper-0 1/1 Running 0 37s zookeeper-zookeeper-1 1/1 Running 0 2m47s zookeeper-zookeeper-2 1/1 Running 0 3m48s zookeeper-zookeeper-3 1/1 Running 0 5m15s zookeeper-zookeeper-4 1/1 Running 0 5m15s
Example: Updating multiple parameters:
To update multiple parameters at once, it is recommended to submit the updated parameters using the KUDO CLI.
See Available Parameters to get the full list of current parameters as a file.
Apply the desired updates in
zookeeper-params.yml using the KUDO CLI:
kubectl kudo update -n test-project-zc6tc --instance=zookeeper -P zookeeper-params.yml
Wait for the deployment plan to
COMPLETE as shown in the node counts example.
The following parameters are treated as immutable by the operator after an instance has been deployed; any modifications to them will trigger the
While modifying the KUDO Zookeeper parameter for
DISK_SIZE is not permitted, users can resize the associated PVC.
Switching between persistent & ephemeral storage classes is not supported for
KUDO Zookeeper creates two Kubernetes Services:
- Client Service (CS)
- This service is intended for clients to connect to the Zookeeper service.
- The port used is set by the
CLIENT_PORTparameter which defaults to
- This service endpoint has the format
- High Availability Services (HS)
- This service is intended for Zookeeper servers to communicate with their quorum in replicated mode
- The port used to listen to request from other servers in the quorum is set by the
SERVER_PORTparameter which defaults to
- The port used by Zookeeper to perform a leader election is set by the
ELECTION_PORTparameter which defaults to
- This service endpoint has the format
Below, we demonstrate how to connect a client to KUDO Zookeeper, the reader will need to have
zkCli installed from Apache Zookeeper
List the available services in the project namespace created by Kommander:
kubectl -n test-project-zc6tc get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE zookeeper-cs ClusterIP 10.0.38.117 <none> 2181/TCP 8m52s zookeeper-hs ClusterIP None <none> 2888/TCP,3888/TCP 8m52s
kubectl port-forward service/zookeeper-cs 2181:2181 -n test-project-zc6tc
Forwarding from 127.0.0.1:2181 -> 2181 Forwarding from [::1]:2181 -> 2181
Connect to the KUDO Zookeeper via
bin/zkCli.sh -server 127.0.0.1:2181
[...output omitted...] Welcome to ZooKeeper! 2021-04-15 01:12:31,978 [myid:127.0.0.1:2181] - INFO [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@1171] - Opening socket connection to server localhost/127.0.0.1:2181. 2021-04-15 01:12:31,979 [myid:127.0.0.1:2181] - INFO [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@1173] - SASL config status: Will not attempt to authenticate using SASL (unknown error) 2021-04-15 01:12:31,996 [myid:127.0.0.1:2181] - INFO [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@1005] - Socket connection established, initiating session, client: /127.0.0.1:38166, server: localhost/127.0.0.1:2181 JLine support is enabled 2021-04-15 01:12:32,113 [myid:127.0.0.1:2181] - INFO [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@1438] - Session establishment complete on server localhost/127.0.0.1:2181, session id = 0x200021ea3ed0003, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: 127.0.0.1:2181(CONNECTED) 0] [zk: 127.0.0.1:2181(CONNECTED) 0] ls / [zookeeper]
KUDO provides the ability to collect logs and other diagnostics data for debugging and for bug-reports.
kubectl kudo diagnostics collect --instance zookeeper -n test-project-zc6tc
The diagnostics data contains the following:
- Installed Manager and its logs.
- Service account and services.
Data for the specified Operator
- The Operator, OperatorVersion and Instance resources.
- Deployed resources from the operator.
- Logs from deployed pods
To monitor all the events occurring in the namespace, it is helpful to look at event log:
kubectl get events -w -n test-project-zc6tc