In our last post, we have seen how to fetch sensor data from the edgex foundry to the server-side using the Rest API. Here we will discuss how to use the MQTT protocol to get the sensor data and visualize them using Grafana. While this is not a part of EdgeX Foundry, it can be useful to know how to capture, save, and visualize data.
If you don't have prior knowledge about MQTT for a quick understanding will give an example of an eCommerce website like Amazon or Mytra, if a product goes out of stock then a Notify Me button will appear instead of Buy Now. If you press the notify button, it will get you noticed whenever the product will be available.
Similarly, MQTT protocol has two clients (at least) like seller and buyer, one who publish the data called the publisher and the other who subscribe to the same data called subscriber and both clients depend on one platform like Amazon called MQTT-Broker who can manage multiple publishers and subscribers. Whenever a publisher publishes or sends data to the broker, the broker's job is to immediately send the data to all the subscribers who subscribe to the same publisher topic/data.
Here we are going to do the same process using edgex foundry(publisher), Mosquitto (Broker), and a messenger application(subscriber).
Four new containerized applications will be added to the same VM running EdgeX Foundry for this demo (Or you can use a different VM in the same network). Follow these steps:-
These are:-
1. Mosquitto: A MQTT broker that can be run locally.
Download and run:-
docker pull eclipse-mosquitto
docker run --name mosquitto -d -p 1883:1883 -p 9001:9001 eclipse-mosquitto
2. InfluxDB: A time-series database perfect for capturing sensor data over time. We need to configure it with grafana.
Download and run:-
docker pull influxdb
docker run \
-d \
--name influxdb \
-p 8086:8086 \
-e INFLUXDB_DB=sensordata \
-e INFLUXDB_ADMIN_USER=root \
-e INFLUXDB_ADMIN_PASSWORD=toor \
-e INFLUXDB_HTTP_AUTH_ENABLED=true \
influxdb
3. Grafana: A graphical dashboard which supports InfluxDB
Download and run:-
docker pull grafana/grafana
docker run -d --name=grafana -p 3000:3000 grafana/grafana
Attach the Influx DB to Grafana:-
Step1: Open grafana dashboard from browser using "ipaddress:3000"
Step2: It will ask to set a new password.
Step3: Set the influx DB as your first data source.
Step4: Do the following settings:
url: <influx db ip>:8086
database: sensordata
user: root
password: toor
4. Messenger: A python script in a container that captures MQTT messages and sends them into the InfluxDB. To run the script inside the container, follow the below steps.
Step1:- Download the file from here, extract it, and go to the subscriber folder.
Open the file “app.py’ in an editor and replace “mqtt-ip-adderss” with the actual IP address of the VM where the Mosquitto installed, the "topic-name" with the name of your topic (this should be unique), and the "Influx-ip-address" with IP address of the VM where the Influx Data base installed (Likely the EdgeX Foundry VM IP address in my case).
Note: There are two entries! Don’t use the loopback address (127.0.0.1) as the ports will clash with the mosquitto MQTT broker.
Step2:- Build the container for the messenger app
docker build -t subscriber .
Note: Make sure to execute the “docker build” command in the
“subscriber” directory.
Step3:- Run the messenger app
docker run -d --name subscriber subscriber:latest
Now all four containers are running.
Go to the Edgex setup and add the below service to the docker-compose.yml file to run mqtt on edgex.
app-service-mqtt:
image: edgexfoundry/docker-app-service-configurable:1.1.0
ports:
- "0.0.0.0:48101:48101"
container_name: edgex-app-service-configurable-mqtt
hostname: edgex-app-service-configurable-mqtt
networks:
edgex-network:
aliases:
- edgex-app-service-configurable-mqtt
environment:
<<: *common-variables
edgex_profile: mqtt-export
Service_Host: edgex-app-service-configurable-mqtt
Service_Port: 48101
MessageBus_SubscribeHost_Host: edgex-core-data
Binding_PublishTopic: events
# Added for MQTT export using app service
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_ADDRESS: "mqtt-broker-ip"
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_PORT: 1883
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_PROTOCOL: tcp
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_TOPIC: "topic-name"
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_PARAMETERS_AUTORECONNECT: "true"
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_PARAMETERS_RETAIN: "true"
WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_PARAMETERS_PERSISTONERROR: "false"
# WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_PUBLISHER:
# WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_USER:
# WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_ADDRESSABLE_PASSWORD:
# WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_PARAMETERS_QOS: ["your quality or service"]
# WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_PARAMETERS_KEY: [your Key]
# WRITABLE_PIPELINE_FUNCTIONS_MQTTSEND_PARAMETERS_CERT: [your Certificate]
depends_on:
- consul
# - logging # uncomment if re-enabled remote logging
- data
Or download the updated docker-compose.yml file from here.
And replace the "mqtt-broker-ip" with your VM IP where the mosquitto broker is installed. And the "topic-name" with your unique topic.
Note:- Topic name should be same in both app.py and docker-compose.yml.
Now run the same python script to get sensor data and send it to edgex foundry used in the previous post. (Watch the demo video)
After the script runs you can create a panel in Grafana to see the collected sensor data with the following settings.
Same you can create for Temperature data.
The summary of the data flow will be:-
1. Data goes from sensor to edgex by an HTTP request.
2. MQTT client(publisher) on edgex will be sending the real-time data to the MQTT broker(Mosquitto).
3. Another MQTT client (Subscriber) on messenger will get the sensor data and send it to the Influx database.
4. Grafana dashboard will show you the real-time data structure that is updating frequently to the influx database.
Here is the video demonstration of this post:-
This is how the MQTT protocol is useful for the Edgex foundry service.
