Anuradha Agarwal
Performance Monitoring using InfluxDB, Python & Grafana
This post details steps to get started with InfluxDB client python library. We will use this python library to insert data into InfluxDB.
To visit more such posts follow hashtag - #RealTimeMonitoring #InfluxDB #Python
Telegraf has got a variety of plugins to monitor infrastructure. However, the case may that we need to send custom data to InfluxDB from an app. Similarly in performance testing, we may need to send request data to the time series database & monitor it in Grafana on a real-time basis. With below example, we can send JSON data to the DB using python library InfluxDB.
Also, the Python client library’s write_points function can take data in either JSON or InfluxDB Line Protocol. However, the point to note is that InfluxdB takes data in the format of command-line protocol format only. Hence, any data provided in JSON format gets eventually converted to Line Protocol anyway before being sent to InfluxDB.
PRE-MILESTONE STEPS
Step 01 - Install Python
Step 02 - Avail or Setup InfluxDB instance
Step 03 - Setup PyCharm or any Python Editor
MILESTONE STEPS
Step 1 - Install InfluxDB Python client library
Step 2 - Import InfluxDB Python client library
Step 3 - Create InfluxDB instance
Step 4 - Connect to your database
Step 5 - Get data to be inserted in JSON format
Step 6 - Use write_points API to insert JSON data
Step 7 - Query data from InfluxDB

Complete Series: Real-Time Monitoring Using InfluxDB & Grafana
Open a command prompt and verify that python is installed & setup by giving the command:
python --version

Give below command to install influxdb-python library
pip install influxdb
With this step influxdb module gets installed inside :
<python path>\Lib\site-packages
Open a command prompt and go to python shell by typing python
Import influxdb-python library
from influxdb import InfluxDBClient
Give the name of host and port on which InfluxDB server is running as below to connect to the client. This creates an instance of DB:
client=InfluxDBClient(host="localhost",port="8086")
Additional parameters can also be passed in same like username, password in case authentication is required
Verify database you intend to connect is available in the list
client.get_list_database()

Switch to database
client.switch_database('MyDatabase')
Now, we are ready to insert data in the database
InfluxDB takes data in the form of command-line protocol format. We can either directly insert data using this format or we can give JSON format data and it internally gets converted to command line format. Here in this post, we intend to give data in JSON format.
In JSON format, we can give data using attributes measurement, tags, fields & time.
[
{
"measurement": "measurement_name",
"tags": {
"tag1": "tag1_name",
"tag2": "tag2_name"
},
"time": "timeofsendingdata",
"fields": {
"field_1": field1_value,
"field_2":field2_value
}
}]
For example, below is sample request-response data which we want to send to InfluxdB in this example. Here we want to build a measurement table called "MyResponseData". Tags are the ones on which basis we can filter data and field is actual data.
Note this is hardcoded data and in the actual script would be replaced by variables storing dynamic data.
[
{
"measurement": "MyResponseData",
"tags": {
"requestName": "LaunchURL",
"requestType": "GET"
},
"fields": {
"responseduration": 127,
"responseLength":7253
}
}]
Now we have a database to which we have to write data and also sample data which is to be written, we will take help of write_points method to write sample data to the database
As per the syntax below, we can pass the argument called points to the method. Point is a list of dictionaries where each dictionary is a point( measurement to be written)
write_points(points,time_precision=None,database=None,retention_policy=None,tags=None,batch_size=None,protocol=u'json',consistency=None)
In our case, we have only one point in our dictionary:
[ { "measurement": "MyResponseData", "tags": { "requestName": "LaunchURL", "requestType": "GET" }, "fields": { "responseduration": 127, "responseLength":7253 } }
]
We can add one or more point to our existing data to show a list of points and we can define it in a variable
my_json_data=[ { "measurement": "MyResponseData", "tags": { "requestName": "LaunchURL", "requestType": "GET" }, "fields": { "responseduration": 127, "responseLength":7253 } }, { "measurement": "MyResponseData", "tags": { "requestName": "Login", "requestType": "POST" }, "fields": { "responseduration": 137, "responseLength": 7287 } } ]
To write this data, give below:
client.write_points(my_json_data)
This will write data to the database.
Note here in this data, we have not given timestamp, hence time column of measurement will get timestamps of the time of insertion of data and not the time data is generated
To specify timestamp, add time attribute to the data point. Here time is read using date-time module of python. It reads time in UTC. For same we need to import datetime python module:
import datetime
import pytz
{ "measurement": "MyResponseData", "tags": { "requestName": "LaunchURL", "requestType": "GET" },
"time":datetime.datetime.utcnow(), "fields": { "responseduration": 127, "responseLength":7253 } }
To query data, python way, use query object, and get data in ResultSet type of object:
results=client.query('SELECT * FROM MyResponseData')
Data can be filtered using tags like below:
login_points=list(results.get_points(measurement='NewResponse',tags={"requestName": "Login"}))
Below is the final sample script.
Please note, in real-time implementation json_body would not be hard-coded and fed from some source.
To view data in Grafana, set DB as a data source. Follow instructions with this post

Complete Series: Real-Time Monitoring Using InfluxDB & Grafana