1. Overview
MQTT (Message Queuing Telemetry Transport) is a simple and lightweight publish/subscribe messaging protocol designed for devices with limited network bandwidth or resources. IoT (Internet of Things) is a typical example of such a network.
In this tutorial, we’ll discuss how to use MQTT in Bash. First, we’ll give a brief introduction to MQTT and discuss its installation. Then, we’ll learn the basic MQTT command-line utilities. Finally, we’ll discuss a Bash script subscribing to MQTT topics and preparing SQL statements using the received topics.
2. Brief Introduction to MQTT
The publish/subscribe pattern is a highly scalable messaging pattern used in distributed systems. Publishers post messages to the distributed system. In MQTT jargon, a message is known as a topic. Subscribers register for topics. Publishers and subscribers are decoupled from each other, i.e., they’re unaware of each other’s existence.
There are two types of entities in the MQTT protocol, a broker and MQTT clients. The broker is the entity that knows and, therefore, decouples the publishers and subscribers in the system. It receives all the published topics and sends them to the subscribers of the topics.
A topic is a hierarchically structured string used for filtering and routing messages. There might be several topics in the system. For example, in a flight tracking system, one topic might contain the position information of an airplane, while another topic might contain the speed and direction of the airplane.
The broker can supply several types of quality of service for a topic. For example, the broker may send a topic to subscribers but may not be interested in the delivery of the topic (fire and forget). On the other hand, it can reliably send a topic so that each subscriber receives it (acknowledged delivery or assured delivery).
The broker can also save the data of all clients to provide persistent sessions. Additionally, it can provide security using authentication, authorization, and encryption. MQTT uses TCP by default under the hood.
Since the broker is the single point of failure in an MQTT system, there must be backup brokers that can substitute a broker in case of a failure.
3. Installation of MQTT
We’ll use the Mosquitto MQTT Broker. Mosquitto is available in the official repositories of Ubuntu.
Let’s install the mosquitto and mosquitto-clients packages using apt-get install:
$ sudo apt-get install mosquitto mosquitto-clients -yapt-get install requires root privileges, so we use it together with the sudo command. The -y option of apt-get is for answering “yes” to all prompts during installation.
The mosquitto package provides the MQTT message broker. The package mosquitto-clients, on the other hand, provides the command-line utilities, mosquitto_pub and mosquitto_sub, for publishing and subscribing to topics, respectively.
Having installed the packages, the Mosquitto MQTT Broker should start automatically. Let’s check whether it’s active and running using systemctl:
$ systemctl is-active mosquitto
activeThe is-active subcommand of systemctl checks whether the specified unit is active. The mosquitto service, i.e., the Mosquitto MQTT Broker, is running in our case.
Notably, the source code can also be downloaded and built as an alternative.
4. MQTT Command-Line Utilities
We’ll discuss the MQTT command-line utilities using a climate control system as an example. The climate control system belongs to a company named CompanyX and consists of devices measuring the preset and actual temperatures in each room on the floors of several buildings. Each device in the system has a unique ID.
The devices in the rooms publish the preset and measured temperatures together with the ID of the device periodically using MQTT topics. Therefore, the topic has three fields, namely id, set_temperature, and actual_temperature.
4.1. The Subscribers
Now, we’ll subscribe to the CompanyX/building1/floor1/room1 topic using the mosquitto_sub utility:
$ mosquitto_sub -t CompanyX/building1/floor1/room1The -t option of mosquitto_sub is used for specifying the topic name. In our case, we want to receive the CompanyX/building1/floor1/room1 topic instances. This is equivalent to specifying that we’re only interested in monitoring the temperatures in a specific room of CompanyX, i.e., the first room, room1, of the first floor, floor1, of the first building, building1.
A topic is leveled in MQTT using a forward slash (/) between levels, as we’ve just seen, similar to the directory structure in an operating system. We can also use wildcards to eliminate the need for a subscription to each topic separately. For example, let’s subscribe to receive all the temperature measurements in the first building, building1:
$ mosquitto_sub -t CompanyX/building1/#The CompanyX/building1/# topic specifies that we’re interested in receiving all the measurements in all of the rooms on all of the floors in building1.
4.2. The Publisher
Now, we have two subscribers waiting to receive temperature measurements.
Let’s publish a topic using the mosquitto_pub utility:
$ mosquitto_pub -t CompanyX/building1/floor1/room1 -m '"id": 111, "set_temperature": 23, "actual_temperature": 22.85'Similar to mosquitto_sub, the -t option of mosquitto_pub is for specifying the topic name. In this case, we publish the CompanyX/building1/floor1/room1 topic. This corresponds to a temperature measured by the device in the first room on the first floor of the first building.
We use the -m option to send a single message from the command line. The id of the device is 111, the preset temperature is 23, and the measured temperature is 22.85 in our case.
4.3. Delivery of the Topics
Having published the topic, let’s check whether the subscribers have received this topic:
$ mosquitto_sub -t CompanyX/building1/floor1/room1
"id": 111, "set_temperature": 23, "actual_temperature": 22.85
$ mosquitto_sub -t CompanyX/building1/#
"id": 111, "set_temperature": 23, "actual_temperature": 22.85
Both subscribers received the published topic, as expected. Now, let’s publish another topic corresponding to a measurement of a device in the first room on the second floor in the first building, i.e., CompanyX/building1/floor2/room1:
$ mosquitto_pub -t CompanyX/building1/floor2/room1 -m '"id": 121, "set_temperature": 21, "actual_temperature": 21.3'Let’s check the outputs of the subscribers again:
$ mosquitto_sub -t CompanyX/building1/floor1/room1
"id": 111, "set_temperature": 23, "actual_temperature": 22.85
$ mosquitto_sub -t CompanyX/building1/#
"id": 111, "set_temperature": 23, "actual_temperature": 22.85
"id": 121, "set_temperature": 21, "actual_temperature": 21.3
As is apparent from the output, only the subscriber waiting for CompanyX/building1/# received this topic, as expected.
5. Subscription in a Bash Script
As we’ve already seen, subscription to MQTT topics is easy using mosquitto_sub. However, just printing the received topics on the command line may not always be desirable. For example, we may want to write the received topics to a database. Additionally, we may want to add the reception time of the topic to each record.
We’ll use the following Bash script, topic_handler.sh, to prepare the SQL statement for writing each topic to an SQL database:
#!/bin/bash
trap ctrl_c INT
function ctrl_c() {
exit 0
}
topic_name=$1
sql_statement="INSERT INTO Temperatures (Timestamp, SensorId, SetTemperature, ActualTemperature) VALUES(timestamp_, sensor_id_, set_temp_, actual_temp_);"
while read topic
do
# Append timestamp to the message
timestamp=$(date +%s)
message=($echo "\"timestamp\":$timestamp, $topic")
# Extract the fields in the message
sensor_id=$(echo $message | awk -F, '{print $2}' | awk -F: '{print $2}')
set_temp=$(echo $message | awk -F, '{print $3}' | awk -F: '{print $2}')
actual_temp=$(echo $message | awk -F, '{print $4}' | awk -F: '{print $2}')
# Prepare the SQL statement
sql_mod=$(echo $sql_statement | sed "s/sensor_id_/$sensor_id/")
sql_mod=$(echo $sql_mod | sed "s/set_temp_/$set_temp/")
sql_mod=$(echo $sql_mod | sed "s/actual_temp_/$actual_temp/")
sql_mod=$(echo $sql_mod | sed "s/timestamp_/$timestamp/")
echo $sql_mod
done < <(mosquitto_sub -t $topic_name)
5.1. Dissection of the Script
Let’s break down the script to analyze it briefly:
trap ctrl_c INT
function ctrl_c() {
exit 0
}
This part is for exiting from the script using Ctrl+C. When we send the SIGINT signal to the running script or press Ctrl+C, we call the function ctrl_c() and exit gracefully using exit 0.
Next, we get the topic name as a command-line argument, and assign it to the variable topic_name:
topic_name=$1Then, we prepare the template SQL statement:
sql_statement="INSERT INTO Temperatures (Timestamp, SensorId, SetTemperature, ActualTemperature) VALUES(timestamp_, sensor_id_, set_temp_, actual_temp_);"This SQL statement inserts the fields of the received topic into a table named Temperatures. The table has four fields, namely Timestamp, SensorId, SetTemperature, and ActualTemperature.
Then, we subscribe to the topic using mosquitto_sub -t $topic_name, and wait to receive topics in a while loop:
while read topic
do
…
done < <(mosquitto_sub -t $topic_name)
We assign the received topic to the topic variable using the read command.
Within the while loop, we first get the current time using timestamp=$(date +%s). Then, we prepend the timestamp to the received topic as a new field:
# Prepend timestamp to the message
timestamp=$(date +%s)
message=($echo "\"timestamp\":$timestamp, $topic")Next, we extract the values of the fields in the received topic using awk, and assign them to the sensor_id, set_temp, and actual_temp variables:
# Extract the fields in the message
sensor_id=$(echo $message | awk -F, '{print $2}' | awk -F: '{print $2}')
set_temp=$(echo $message | awk -F, '{print $3}' | awk -F: '{print $2}')
actual_temp=$(echo $message | awk -F, '{print $4}' | awk -F: '{print $2}')Finally, we substitute the values in the template SQL statement using sed to get the actual SQL statement, and print it:
# Prepare the SQL statement
sql_mod=$(echo $sql_statement | sed "s/sensor_id_/$sensor_id/")
sql_mod=$(echo $sql_mod | sed "s/set_temp_/$set_temp/")
sql_mod=$(echo $sql_mod | sed "s/actual_temp_/$actual_temp/")
sql_mod=$(echo $sql_mod | sed "s/timestamp_/$timestamp/")
echo $sql_mod
We can use the statement in the sql_mod variable to write the topic to an SQL database.
5.2. Running the Script
Now, it’s time to run the script. Let’s subscribe to the CompanyX/building1/floor1/room1 topic:
$ ./topic_handler.sh CompanyX/building1/floor1/room1Let’s also subscribe to all of the temperature measurements in CompanyX:
$ ./topic_handler.sh CompanyX/#Now, let’s publish the topics CompanyX/building1/floor1/room1 and CompanyX/building2/floor2/room2 using mosquitto_pub:
$ mosquitto_pub -t CompanyX/building1/floor1/room1 -m '"id": 111, "set_temperature": 23, "actual_temperature": 22.85'
$ mosquitto_pub -t CompanyX/building2/floor2/room2 -m '"id": 222, "set_temperature": 21, "actual_temperature": 21.3'
The script that subscribes to CompanyX/building1/floor1/room1 receives only the first published topic:
./topic_handler.sh CompanyX/building1/floor1/room1
INSERT INTO Vehicles (Timestamp, SensorId, SetTemperature, ActualTemperature) VALUES(1706189938, 111, 23, 22.85);The script that subscribes to CompanyX/# receives both of the published topics:
$ ./topic_handler.sh CompanyX/#
INSERT INTO Vehicles (Timestamp, SensorId, SetTemperature, ActualTemperature) VALUES(1706189938, 111, 23, 22.85);
INSERT INTO Vehicles (Timestamp, SensorId, SetTemperature, ActualTemperature) VALUES(1706189985, 222, 21, 21.3);The SQL statements printed by the scripts are as expected.
6. Conclusion
In this article, we discussed how to use MQTT in Bash.
We started with an introduction to MQTT and saw how to install Mosquitto MQTT Broker and clients in Ubuntu.
Then, we learned to use the mosquitto_pub and mosquitto_sub command-line utilities.
Finally, we examined a Bash script that subscribes to the topic passed to it as an argument and prepares the SQL statements for writing the received topics to an SQL database.