Real-Time Monitoring using Locust with InfluxDB & Grafana

In this post, I will be taking you through end to end steps to set up a real-time monitoring solution for Locust using InfluxDB & Grafana.

This post is also a resource to my Udemy course Performance Testing using Locust.

To visit more such posts, follow: #RealTimeMonitoring #PerformanceTesting

MILESTONE STEPS

Step 1 - Ensure prerequisite criteria

Step 2 - Create demo DB for storing Locust data

Step 3 - Create a sample locust script

Step 4 - Fetch response data in JSON using Locust events

Step 5 - Install & import python InfluxDB for sending data to InfluxDB

Step 6 - Enhance Locust Script to send fetched data

Step 7 - Verify measurements in InfluxDB

Step 8 - Configure data source & view graph in Grafana

• Make sure you have a python & locust module installed on your machine. Else, follow this tutorial

• Make sure you have an Influxdb instance available.

• In the current post, I will use InfluxDB instance installed on windows. You can use any instance.

• Login to InfluxDB CLI & create DB named DemoDB_Locust

• Any locust script will do. Below is the sample script on which we will work to fetch data and send it to the Influxdb instance:

Let's modify the above script by using locust events to fetch response data. For the same, we will,

• import locust events for fetching data on events of request success & failure

• import python inbuilt JSON module for working with JSON data

• import datetime & pytz modules for inserting timestamp of event

• import socket to insert hostname data in JSON

To the above script, we will add two event handlers to fetch data in JSON. These event handlers are further attached to locust events for request success & failure.

Make sure you have influxdb-python module installed in your development environment:

pip install influxdb

ALSO, SEE -

Update your script by importing InfluxDBClient:

from influxdb import InfluxDBClient

• Create a client object using InfluxDBClient by providing Influxdb instance credentials:

client = InfluxDBClient(host="localhost", port="8086")

Now in the script , event handler part we can add json_string as an argument to write_points method from influxdb-python. This method takes a list of dictionaries.Hence before passing json_string , we need to convert it to python dictionary format using json.loads method.

def individual_success_handle(request_type, name, response_time, response_length, **kwargs):
    SUCCESS_TEMPLATE = '[{"measurement": "%s","tags": {"hostname":"%s","requestName": "%s","requestType": "%s","status":"%s"' \
                       '},"time":"%s","fields": {"responseTime": "%s","responseLength":"%s"}' \
                       '}]'
    json_string = SUCCESS_TEMPLATE % (
    "ResponseTable", hostname, name, request_type, "success", datetime.datetime.now(tz=pytz.UTC), response_time,
    response_length)
    client.write_points(json.loads(json_string), time_precision='ms')

• The complete script is below:

• Run script from your development environment:

• Login to Influx CLI & verify measurement is added to DemoDB_Locust

Follow the below tutorial for configuring data source & viewing data:

---