OVH Guides

Describe Deployed Model Inputs and Outputs

Learn how to describe model inputs and outputs

Last updated 7th February, 2020.

Objective

This tutorial will help you identify the expected inputs and outputs of a deployed model on OVHcloud serving engine.

Requirements

  • Having previously deployed a serialized model and knowing the url on which it can be requested. Detailed steps on how to deploy a serialized model are explained here
  • Owning a token with model-evaluation role for the deployed model. Steps for generating such token are described here

Describe the models inputs and outputs

Each serialized model takes a list of named tensors as inputs and also returns a list of named tensors as outputs.

A named tensors is a N-Dimensional array with :

  • A identifier name. Example: my-tensor-name
  • A data type. Example: integer or double or string
  • A shape. Example: (5) for a vector of length 5, (3, 2) for a matrix which first dimension is of size 3 and second dimension is of size 2. Etc.

You can get access to the model inputs and outputs by calling the http GET method on /describe path of the model.

Example of a describe query with curl

curl \
    -H 'Authorization: Bearer <evaluation-token>' \
    -X GET \
    http://<your-model-url>/describe

In this example don't forget to replace the <evaluation-token> with your model-evaluation token and <your-model-url> with the url that was generated by serving engine on your model deployment.

Example of a describe response

You will get a JSON object describing the list of inputs tensors that are needed to query your model as well as the list of outputs tensors that will be returning.

{
    "inputs": [
        {
            "name": "sepal_length",
            "type": "float",
            "shape": [-1]
        },
        {
            "name": "sepal_width",
            "type": "float",
            "shape": [-1]
        },
        {
            "name": "petal_length",
            "type": "float",
            "shape": [-1]
        },
        {
            "name": "petal_width",
            "type": "float",
            "shape": [-1]
        }
    ],
    "outputs": [
        {
            "name": "output_label",
            "type": "long",
            "shape": [-1]
        },
        {
            "name": "output_probability",
            "type": "float",
            "shape": [-1, 2]
        }
    ]
}

In this example, the deployed model is waiting for 4 tensors as inputs :

  • sepal_length of shape (-1) (i.e. a vector of any size)
  • sepal_width of shape (-1) (i.e. a vector of any size)
  • petal_length of shape (-1) (i.e. a vector of any size)
  • petal_width of shape (-1) (i.e. a vector of any size)

It will answer a response with 2 tensors as outputs :

  • output_label of shape (-1) (i.e. a vector of any size)
  • output_probability of shape (-1, 2) (i.e. a matrix which first dimension is of any size and which second dimension is of size 2)

Going further


These guides might also interest you...