How to Check if Zookeeper Is Running or up From Command Prompt?

1. Introduction

Apache ZooKeeper is a distributed coordination service that eases the development of distributed applications. The challenges with developing distributed applications lie in coordination, configuration management, leader election, etc. Apache Zookeeper solves these challenges well. This has led to its popularity in several open-source products, including Apache Hadoop, Solr, and others.

We’ll now look into the installation options for ZooKeeper and ways to check if a ZooKeeper node is running or not.

2. Installation

We can download the latest ZooKeeper release from here. Most common production operating systems are supported (except Mac). Specific system requirements can be found here. ZooKeeper can be installed in two modes: standalone and cluster (aka ensemble).

The standalone mode is popular for development purposes, but production applications should use the ensemble mode. In the ensemble mode, it’s recommended to have an odd number of ZooKeeper server nodes (znodes). We need to have at least three znodes for a fault-tolerant production installation.

While all nodes in ZooKeeper keep the same data, within the ZooKeeper ensemble, one node typically acts as a “leader” and the rest of the znodes act as “followers”. As such, it’s important to design the installation so that each znode can connect with every other znode.

The full installation steps are available on the Apache ZooKeeper installation guide.  For this article, we’ll assume standalone mode but the checks are the same for ensemble mode as well.

3. Checking ZooKeeper Node Status

Post installation of ZooKeeper, an important task is to check that the server is up and running. This is also something we need to know when we want to check connectivity to a znode from a remote application server.

First, we need to know the FQDN or IP address of the znode and the client port where it’s listening. For this article, we’re assuming a standalone ZooKeeper installed using the default configuration, and therefore the hostname is localhost and the port number is 2181.

Once we have the required information, there are a few different ways to check the znode status.

3.1. OS Commands

We log in to the server where the znode is located, and then we can fire some OS commands to check the status of ZooKeeper.

In case we’re using Unix/Linux/Mac, then the ps command is helpful:

$ ps -ef | grep Zookeeper

Here’s a sample output:

501  3969     1   0  9:13AM ttys000    0:03.75 /usr/bin/java -Dzookeeper.log.dir=.....
org.apache.zookeeper.server.quorum.QuorumPeerMain /opt/apache-zookeeper-3.9.3-bin/bin/../conf/zoo.cfg

The text after the Java command is replaced with “….” to remove unnecessary classpath and configuration details. The key point here is that ZooKeeper runs as a Java program, and the main class is QuorumPeerMain.

Given that ZooKeeper is a Java process and we have a JVM installed as well, we can run another command to check the status of ZooKeeper:

$ jps | grep Quorum

The jps command should also work for Windows, but the command will be slightly different. We can’t use the pipe (|) and the grep command afterward.

The output should be something like this:

$ jps
3969 QuorumPeerMain
5565 Jps

The OS commands are a quick way for administrators and monitoring tools to check whether ZooKeeper processes are running. However, sometimes the process may be running but not responsive. In such a case, the OS commands might suggest that ZooKeeper is up even though it isn’t responsive. There are a few different ways to truly know that ZooKeeper is running and responsive.

3.2. zkServer and zkCli Scripts

The ZooKeeper installation includes two command-line interface scripts – zkServer.sh and zkCli.sh. We generally start the Zookeeper server using zkServer.sh script. We can use the same script to check the status of a running server:

$ ./zkServer.sh status
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /opt/apache-zookeeper-3.9.3-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: standalone

The zkServer.sh is a server management script and only works when we log on to a znode. However, the zkCli.sh is a command line client for ZooKeeper and we can use it to check even the remote server status. It opens up a shell similar to a Linux or Windows shell but allows a ZooKeeper admin to interact with the ZooKeeper system. We can also use this to check basic connectivity with a znode:

$ ./bin/zkCli.sh -waitforconnection -timeout 3000 -server 127.0.0.1:2181

Here we’ve used the local loopback IP (127.0.0.1); however, we can use this to connect to a remote server as well, using the server’s IP. The zkCli.sh has many more useful management commands, and details can be found in the ZooKeeper admin guide.

If the zkCli.sh connects successfully, we’ll be inside the ZooKeeper shell:

Welcome to ZooKeeper!
JLine support is enabled
[zk: 127.0.0.1:2181(CONNECTED) 0] 

3.3. ZooKeeper Commands

So far, we’ve discussed scripts and OS commands to check server status. We note that in all of those ways we have to log on to the znode or at least we need a ZooKeeper installation on the source machine where we run the commands.

ZooKeeper also offers simple four-letter command words. We can use these to interact with ZooKeeper remotely and extract useful information from anywhere on the network. These commands aren’t enabled by default. We first whitelist them by adding the wildcard whitelist configuration to the zoo.cfg file on each znode:

4lw.commands.whitelist=*

Or we can only whitelist specific commands by using a comma-separated list:

4lw.commands.whitelist=stat, ruok, conf, isro

We need to restart the znode after adding these commands. Once we’ve restarted the znode, we’re ready to connect remotely to the ZooKeeper node using the commands. The command of our interest is ruok which is just a 4-letter version of the question “Are you OK?”. The server responds with an answer imok which is equivalent to “I’m OK”.

For this purpose, we can use network connectivity commands like telnet:

> telnet localhost 2181
Trying ::1...
Connected to localhost.
Escape character is '^]'.
ruok
imokConnection closed by foreign host.

Or, in Linux/Unix systems, we can use netcat:

$ echo ruok | nc localhost 2181
imo

4. Further Reading

In the past, Apache Kafka also used Zookeeper but has now moved to a different solution. However, for any applications that have the same issues around coordination and configuration management in a distributed environment, ZooKeeper provides a simple, lightweight, and robust solution.

We can use a Zookeeper ensemble with several different applications by making effective use of the ZooKeeper Namespaces and ACLs. ZooKeeper comes with client libraries in C and Java, and there are libraries for ZooKeeper in several other backend languages, including Python, Rust, Go, JavaScript (NodeJS), and PHP.

This means that organizations can develop distributed production applications using ZooKeeper to handle the coordination and configuration management tasks without having to deal with the complexity associated with those tasks. An example would be using service discovery with Spring and ZooKeeper. We may consume services from other applications so long as the provider application is registered as a service in the ZooKeeper ensemble.

5. Conclusion

In this article, we went through a quick overview of ZooKeeper and its installation options. We also looked at the ways to check the status of ZooKeeper server nodes (znodes). Apache ZooKeeper offers a powerful command line script zkCli.sh, and commands such as ruok, and stat. We can use these remotely to check the status of znodes.