Use InfluxDB

Last updated 31 July, 2020

Objective

InfluxDB is a proprietary time series database that integrates the open source collector Telegraf. In this guide, you will learn how to push and query using the Influx protocol with Metrics.

Requirements

• a valid OVH Metrics account.

Instructions

Compatibility

InfluxDB has the notion of databases. This concept doesn't exist within Metrics. If you need segmentation, you can use different Metrics project or isolate with an additional label.

Data Model

InfluxDB uses it own data model :

 <measurement>[,<tag_key>=<tag_value>[,<tag_key>=<tag_value>]] <field_key>=<field_value>[,<field_key>=<field_value>] [<timestamp>]

How to push data

The full documentation is available at https://docs.influxdata.com/influxdb/v1.2/guides/writing_data/

Authentification

To push data to the platform, you will need a WRITE TOKEN. Use Basic Auth directly inside the URL to pass it properly, like this :

https://metrics:[WRITE_TOKEN]@influxdb.[region].metrics.ovh.net

Pushing datapoints using cURL

 $ curl -i -XPOST \
     'https://DESC:TOKEN_WRITE@influxdb.gra1.metrics.ovh.net/write' \
     --data-binary \
     'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000'

Pushing datapoints using python

You can use the native Influx python client to push Influx Data on Metrics:

from influxdb import InfluxDBClient
json_body = [
    {
        "measurement": "measurement",
        "tags": {
            "host": "server01"
        },
        "time": "2020-07-30T11:00:00Z",
        "fields": {
            "series": 0.64
        }
    }
]

client = InfluxDBClient(host="influxdb.gra1.metrics.ovh.net", port=443, username="metrics", password="WRITE_TOKEN", database="metrics", ssl=True)
res = client.write_points(json_body)
print(res)

If res result is True, then you have correctly pushed your Influx data on the Metrics Data Platform.

How to query data

InfluxDB has its own Query DSL, that mimics SQL without being plain ANSI SQL.

 SELECT <field_key>[,<field_key>,<tag_key>] FROM <measurement_name>[,<measurement_name>]

Authentification

To query data to the platform, you will need a READ TOKEN. Use Basic Auth directly inside the URL to pass it properly, like this :

https://metrics:[READ_TOKEN]@influxdb.[region].metrics.ovh.net

Query using cURL

A quick example to use InfluxQL on Metrics with cURL would be:

curl --request GET \
  --url 'https://m:READ_TOKEN@influxdb.gra1.metrics.ovh.net/query?q=SELECT%20%22used_percent%22%20FROM%20%22disk%22%20WHERE%20%20time%20%3E%3D%20now()%20-%2020m&=%20'

This will execute the following InfluxQL query:

SELECT "used_percent" FROM "disk" WHERE  time >= now() - 20m

For the one used to query Influx, you will notice that the db mandatory parameter of Influx is not set in this query. With Metrics the database field is optional, as Metrics does not rely on databases to store its metrics. If you need segmentation, you can use different Metrics project or isolate with an additional label.

Query using python

You can use the native Influx python client to query Metrics with InfluxQL:

from influxdb import InfluxDBClient
query = 'SELECT "series" FROM "measurement" WHERE time > now() - 1d;'

client = InfluxDBClient(host="influxdb.gra1.metrics.ovh.net", port=443, username="metrics", password="READ_TOKEN", database="metrics", ssl=True)
result = client.query(query)

print("Result: {0}".format(result))

Data Exploration

The InfluxQL data exploration statements requests supported on the metrics platform are:

We support also all possibility to configure a query result using: ORDER BY time DESC, LIMIT, OFFSET or TIME clauses.

You can also use all InfluxQL tips on the query syntaxof the InfluxQL data exploration page on the Metrics Platform.

Mathematical operators

The following existing arithmetic operators in InfluxQL can be used on the Metrics platform:

The Modulo (%) operator is not yet supported accross several metrics. It can still be applied between metrics and values.

All the bitwise operation: Bitwise AND (&), Bitwise OR (|) and Bitwise Exclusive-OR (^) are only working on numbers (float numbers will be automaticallly cast to integers).

InfluxQL functions

The valid influxQL functions on the Metrics platform are (all parameter with a ? are optionals):

Data types and cast operations

The existing data types and cast operations of InfluxQL matches the one supported by the Metrics platform:

Regular expressions

You can apply Regular expression on the Metrics Platform, however we don't support the native InfluxQL regular expression but Warp 10™ native supported one.

GROUPBY clause

The existing GROUPBY clause of InfluxQL is supported as if on the Metrics platform (by time or by tag fields).

The null parameter for filling will not provide any null values on the Metrics platform as NULL ticks correspond to empty values in Warp 10™ .

WHERE clause

The existing WHERE clause of InfluxQL is supported as if on the Metrics platform.

Schema exploration statements

The existing SHOW statements of InfluxQL supported by the Metrics platform are:

As the concept of databases doesn't exists in Metrics, the SHOW DATABASES statement will always return only one database: metrics. For the SHOW TAG VALUES CARDINALITY statement: no measurement split are computed and only the global tag cardinality is shown (compare to the same statement on InfluxDB). To get split tag cardinality statement, refers all wanted measurement in FROM clause.

Database management statements

The existing database management statements of InfluxQL supported by the Metrics platform are:

As the CREATE DATABASE statement is used by some client, this statement was implemented in Metrics and always return. However no database exists in Metrics.

Database continuous queries

The InfluxQL continuous queries can not be performed yet on the Metrics platform.

Query parameters

Natively an InfluxQL requests expects two parameter q and db. As in metrics application the database notion doesn't exist the db parameter is optional. By default the time parameters are rendered with an rfc3339 string. To get a time value, you will need to specify epoch as query parameter with the wanted unit. The supported ones on the Metrics Data Platform are: ms, us and ns.

Use InfluxQL to query data from sources that were not pushed on the Influx format

On Metrics, you can push data with several different format: for example the Prometheus. As by default when a user push native influxQL data, we add a "." as separator between the measurement and its field keys in our internal representation. As for example, with Prometheus you can't have any "." in the data format. We added a new clause in where statements: the _separator to be able to query data from all kind of sources in InfluxQL.

The _separator allow the user to choose a custom selector which will splits its influx measurement from its field keys. This allow the user to use the promQL "_" as separator to split Prometheus metrics classnames.

Example:

SELECT mean("field") FROM "prometheus_data" WHERE  time >= now() - 6h AND _separator = "_" GROUP BY time(1h) fill(null)

This allow also the user to query InfluxData or any other kind of data with InfluxQL and to get the raw data representation.

Example:

SELECT mean("disk.used_percent") FROM "" WHERE  time >= now() - 6h AND _separator = "" GROUP BY time(1h) fill(null)

Set up InfluxDB on Grafana

Then select basic auth and fill the user with a non empty value, metrics for example. The password needs to be a valid Metrics READ_TOKEN. Please ensure that you have set the metrics key in the database field as described in the screen below.

Your Influx datasource is now set and ready to be queried.

Go further

• Documentation: Guides
• Vizualize your data: https://grafana.metrics.ovh.net/login
• Community hub: https://community.ovh.com
• Create an account: Try it free!

Ces guides pourraient également vous intéresser...