Djehouty is intended to be a set of logging formatters and handlers to easily send log entries into Logs Data Platform.

This package includes:

  • for GELF:
    • a TCP/TLS handler to send log entries over TCP with TLS support.
    • a formatter to convert logging record into GELF(1.1).
  • for LTSV:
    • a TCP/TLS handler to send log entries over TCP with TLS support.
    • a formatter to convert logging record into LTSV.

Requirements

In order to achieve this amazing task, we still need to review our check list. For this tutorial you will need:

Install

Using pip

You can use pip to install Djehouty, make sure you have the latest version:

$ sh-4.2# pip install --upgrade pip
[...]
Successfully installed pip-<version> 
$ sh-4.2# pip install --upgrade djehouty
[...]
Successfully installed djehouty-<version> setuptools-18.3.1

Using sources

Djehouty is available on the OVH github repository and can be installed manually:

$ sh-4.2$ git clone git@github.com:ovh/djehouty.git
Cloning into 'djehouty'...
remote: Counting objects: 58, done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 58 (delta 26), reused 0 (delta 0)
Receiving objects: 100% (58/58), 9.62 KiB | 0 bytes/s, done.
Resolving deltas: 100% (26/26), done.
Checking connectivity... done.

$ sh-4.2$ cd djehouty
$ sh-4.2$ python setup.py install
[...]
Using /usr/lib/python2.7/site-packages
Finished processing dependencies for djehouty==<version>

How to send logs

The following examples assume that you already have a working stream. Moreover you will have to change the address and the ports of the endpoint for the one you have been assigned to. To retrieve the address and the ports of your endpoint, head to the About page in the Manager. To send log messages, just use the handler of the desired format with the following parameters ('*' means required):

Parameter Gelf LTSV
host * your assigned endpoint. Ex : <your_cluster>.logs.ovh.com
port * Refer to the PaaS Logs ports list
level logging.DEBUG or highter
use_tls True or False (depends on the chosen port)
static_fields * {"X-OVH-TOKEN": "xxxx"}
null_character True Not Supported

The complete list of parameters supported by Djehouty can be found on github.

Example: Use case with GELF over TCP/TLS

import logging
from djehouty.libgelf.handlers import GELFTCPSocketHandler

gelf_logger = logging.getLogger('djehouty-gelf')
gelf_logger.setLevel(logging.DEBUG)
gelf_logger.addHandler(GELFTCPSocketHandler(
    host            = "<your_cluster>.logs.ovh.com",
    port            = 12202,
    static_fields   = {"X-OVH-TOKEN": "XXXXX-XXXXXX"},
    use_tls         = True,
    level           = logging.DEBUG,
    null_character  = True,
))

gelf_logger.info('test')

Example: Use case with LTSV over TCP/TLS

import logging
from djehouty.libltsv.handlers import LTSVTCPSocketHandler

ltsv_logger = logging.getLogger('djehouty-ltsv')
ltsv_logger.setLevel(logging.DEBUG)
ltsv_logger.addHandler(LTSVTCPSocketHandler(
    host            = "<your_cluster>.logs.ovh.com",
    port            = 12201,
    static_fields   = {"X-OVH-TOKEN": "XXXXX-XXXXXX"},
    use_tls         = True,
    level           = logging.DEBUG
))

ltsv_logger.info('test')

Send additional meta data

If you have many handler, you can use the logging.LoggerAdapter class to add extra.

The following example uses the LTSV logger defined above:

mylogger = logging.LoggerAdapter(
    ltsv_logger,
    extra = {"myvar": 5}
)
mylogger.info('test')

You can add specific log meta for each entry using the extra parameter, the following example uses the LTSV logger defined above:

ltsv_logger.info("Hello '%s'", 'Cedric', extra={"lang": 'en'})
ltsv_logger.info("Bonjour '%s'", 'Cedric', extra={"lang": 'fr'})

Getting Help