Describe Deployed Model Inputs and Outputs

Last updated 7th February, 2020.

Objective

This tutorial will help you identify the expected inputs and outputs of a deployed model on OVHcloud ML Serving.

Requirements

• Having previously deployed a serialized model and knowing the url on which it can be requested. Detailed steps on how to deploy a serialized model are explained here
• Owning a token with model-evaluation role for the deployed model. Steps for generating such token are described here

Describe the models inputs and outputs

Each serialized model takes a list of named tensors as inputs and also returns a list of named tensors as outputs.

A named tensors is a N-Dimensional array with :

• A identifier name. Example: my-tensor-name
• A data type. Example: integer or double or string
• A shape. Example: (5) for a vector of length 5, (3, 2) for a matrix which first dimension is of size 3 and second dimension is of size 2. Etc.

You can get access to the model inputs and outputs by calling the http GET method on /describe path of the model.

Example of a describe query with curl

curl \
    -H 'Authorization: Bearer <evaluation-token>' \
    -X GET \
    http://<your-model-url>/describe

Example of a describe response

You will get a JSON object describing the list of inputs tensors that are needed to query your model as well as the list of outputs tensors that will be returning.

{
    "inputs": [
        {
            "name": "sepal_length",
            "type": "float",
            "shape": [-1]
        },
        {
            "name": "sepal_width",
            "type": "float",
            "shape": [-1]
        },
        {
            "name": "petal_length",
            "type": "float",
            "shape": [-1]
        },
        {
            "name": "petal_width",
            "type": "float",
            "shape": [-1]
        }
    ],
    "outputs": [
        {
            "name": "output_label",
            "type": "long",
            "shape": [-1]
        },
        {
            "name": "output_probability",
            "type": "float",
            "shape": [-1, 2]
        }
    ]
}

In this example, the deployed model is waiting for 4 tensors as inputs :

• sepal_length of shape (-1) (i.e. a vector of any size)
• sepal_width of shape (-1) (i.e. a vector of any size)
• petal_length of shape (-1) (i.e. a vector of any size)
• petal_width of shape (-1) (i.e. a vector of any size)

It will answer a response with 2 tensors as outputs :

• output_label of shape (-1) (i.e. a vector of any size)
• output_probability of shape (-1, 2) (i.e. a matrix which first dimension is of any size and which second dimension is of size 2)

Going further

• You can check the OVHcloud documention on how to query custom models.

These guides might also interest you...