OVH Guides

Use OpenTSDB

Get an overview on how to use OpenTSDB for Metrics

Last updated 26 April, 2019

Objective

OpenTSDB is a Scalable Time Series Database design to store and serve massive amounts of time series data without losing granularity. In this guide, you will learn how to use OpenTSDB protocol with Metrics.

Requirements

  • a valid OVH Metrics account.

Instructions

Compatibility

We are currently supporting theses calls:

API Method Supported
/api/aggregators GET
/api/aggregators POST
/api/annotation GET
/api/annotation POST
/api/annotation PUT
/api/annotation DELETE
/api/config GET
/api/config POST
/api/dropcaches GET
/api/dropcaches POST
/api/histogram GET
/api/histogram POST
/api/put POST
/api/query GET
/api/query POST
/api/query DELETE
/api/query/last GET
/api/query/last POST
/api/rollup GET
/api/rollup POST
/api/search/lookup GET
/api/search/lookup POST
/api/search/* GET
/api/search/* POST
/api/serializers GET
/api/serializers POST
/api/stats GET
/api/stats POST
/api/suggest GET
/api/suggest POST
/api/tree GET
/api/tree POST
/api/tree PUT
/api/tree DELETE
/api/uid/* GET
/api/uid/* POST
/api/version GET
/api/version POST
TELNET TCP

We do not support pushing using telnet mode and API backend (config, caches, serializers, stats, version) related endpoints as we are not running an OpenTSDB.

OpenTSDB annotation can be used using strings values as metrics data

How to Push data

Authentification

To push data to the platform, you will need a WRITE TOKEN. Use Basic Auth directly inside the URL to pass it properly, like this :

https://metrics:[WRITE_TOKEN]@opentsdb.[region].metrics.ovh.net

Push datapoints using curl

The full documentation is available at http://opentsdb.net/docs/build/html/api_http/put.html. As an example you can push single point.

Create a file on your disk named opentsdb.json, and populate it with the following content:

{
    "metric": "sys.cpu.nice",
    "timestamp": 1346846400,
    "value": 18,
    "tags": {
        "host": "web01",
        "dc": "lga"
    }
}

In the curl command below, replace REGION per your own one : gra1 or bhs1. As the user doesn't matter on the metrics backend, all the information are stored in our cryptographical token, you can replace or let metrics as if.

$ curl -X POST -d @opentsdb.json 'https://metrics:TOKEN_WRITE@opentsdb.REGION.metrics.ovh.net/api/put'

If everyting happens correctly, the CURL would exit with a 200 code status.

Push datapoints using Python

Here's an example how you can use Python to push datapoints in OpenTSDB format using Requests:

>>> import requests

>>> url = 'https://metrics:TOKEN@opentsdb.REGION.metrics.ovh.net/api/put'
>>> payload = {}
>>> payload["metric"] = "sys.cpu.nice"
>>> payload["timestamp"] = "1346846400"
>>> payload["value"] = 18
>>> tags = { "host": "web01", "dc": "lga"}
>>> payload["tags"] = tags

>>> r = requests.post(url, json=payload)
>>> r.status_code

How to query data

Authentification

To query data to the platform, you will need a READ TOKEN. Use Basic Auth directly inside the URL to pass it properly, like this :

https://metrics:[READ_TOKEN]@opentsdb.[region].metrics.ovh.net

Query using curl

Now let's retrieve the previously pushed data.

The full documentation is available at http://opentsdb.net/docs/build/html/api_http/query/index.html

Let's write a query.json file which contains the following code:

{
    "start": 1346846000000,
    "end": 1346847300005,
    "queries": [
        {
            "metric": "sys.cpu.nice",
            "aggregator": "min",
            "downsample": "4m-avg",
            "tags": {
                "host": "*",
                "dc": "*"
            }
        }
    ]
}

This will get all the saved points and compute the query before returning the result. The curl command to execute this query is:

$ curl --data-binary @query.json 'https://DESC:TOKEN_READ@opentsdb.REGION.metrics.ovh.net/api/query'

You should expects a result similar to:

[{"metric":"sys.cpu.nice","tags":{"dce":"lga", "host": "web02"},"aggregateTags":["host","de"],
"dps": {"1346846340" :9, "1346846580" :9, "1346846820" :8, "1346847060" :8.5, "1346847300" :8.5}},{"m
etric":"sys.cpu.nice","tags":{"dce":"lga","host":"web01"},"aggregateTags":["host","dc"], "dps
":{"13468463460" :18, "1346846580" :18, "1346846820" :19, "1346847060" :19.5, "1346847300" :19.5}}]

Supported queries attributes

The Metrics platform offers almost a full support for OpenTSDB 2.3 queries.

OpenTSDB requests

The OpenTSDB requests attributes supported on the metrics platform are:

Attribute Type Supported
start Integer, String
end Integer, String
queries Array
noAnnotations Boolean
globalAnnotations Boolean
msResolution Boolean
showTSUIDs Boolean
showSummary (2.2) Boolean
showStats (2.2) Boolean
showQuery (2.2) Boolean
delete Boolean
timezone (2.3) String
useCalendar (2.3) Boolean

We do not support annotations (as in Metrics annotations can be stored in a series). showTSUIDs isn't implemented as our series are stored using an Hash of their classnames and tags.

The allowed strings date format are defined at http://opentsdb.net/docs/build/html/user_guide/query/dates.html.

OpenTSDB sub-queries

The OpenTSDB sub-queries attributes supported on the metrics platform are:

Attribute Type Supported
aggregator String
metric String
rate Boolean
rateOptions Map
downsample String
tags Map
filters (2.2) List
explicitTags (2.3) Boolean
percentiles (2.4) Boolean

Settings explicitTags will result only on the series that have all theirs labels key in tags map and/or in filters list.

OpenTSDB rate-options

The OpenTSDB rate-options attributes supported on the metrics platform are:

Attribute Type Supported
counter Boolean
counterMax Integer
resetValue Integer
dropResets Boolean

OpenTSDB Filters

The OpenTSDB Filters attributes supported on the metrics platform are:

Attribute Type Supported
type String
tagk String
filter String
groupBy Boolean

OpenTSDB Aggregators

The OpenTSDB aggregators supported on the metrics platform are:

Attribute Interpolation Grouping/Downsampling Supported
avg Linear Interpolation Both
count Not counted when missing Both
dev Linear Interpolation Both
ep50r3 Linear Interpolation None
ep50r7 Linear Interpolation None
ep75r3 Linear Interpolation None
ep75r7 Linear Interpolation None
ep90r3 Linear Interpolation None
ep90r7 Linear Interpolation None
ep95r3 Linear Interpolation None
ep95r7 Linear Interpolation None
ep99r3 Linear Interpolation None
ep99r7 Linear Interpolation None
ep999r3 Linear Interpolation None
ep999r7 Linear Interpolation None
first None Downsampling
last None Downsampling
mimmin Not compared when missing Both
mimmax Not compared when missing Both
min Linear Interpolation Both
max Linear Interpolation Both
none Not counted when missing Grouping
p50 Linear Interpolation Both
p75 Linear Interpolation Both
p90 Linear Interpolation Both
p95 Linear Interpolation Both
p99 Linear Interpolation Both
p999 Linear Interpolation Both
sum Linear Interpolation Both
zimsum Zero when missing Both

OpenTSDB Downsampling fill policies

The OpenTSDB downsampling fill policies supported on the metrics platform are:

Policy Supported
None
NaN
Null
Zero

Go further


These guides might also interest you...