How to use Data Processing through the OVHcloud API

Find out how to run your Apache Spark jobs with the OVHcloud API

Last updated 14th May, 2021

Objective

This guide helps you to use the Data Processing endpoints through the OVHcloud API in order to develop your own applications with Data Processing.

To read an introduction about the Data Processing service you can visit Data Processing Overview.

Requirements

Instructions

Step 1: Get your OVHcloud API credentials

To be able to use the OVHcloud API, you must set up some configurations that will allow it to authenticate. To do so, you will need an application key, a secret application key and a consumer key. These can be obtained here.

You need to add the rights GET/POST/PUT on the endpoint /cloud/project/*/dataProcessing/*.

Creating API Keys for your script

Keep those credentials as they will be needed to use the OVHcloud API.

To store and use them you can create a configuration file or set them as environment variables. The way to proceed depends on which medium you chose to reach the API.

Most of the wrappers provided by OVHcloud use an ovh.conf file that looks like the following:

endpoint=ovh-eu
application_key=my_app_key
application_secret=my_application_secret
consumer_key=my_consumer_key

Create this file in your application's directory and fill it with your own 3 keys.

Step 2: Chose your method to reach the OVHcloud API

Once you have your credentials, you will be able to query the OVHcloud API. To do so, you can choose between two options.

  • Using a HTTP client or the curl command line: You can find the documentation to learn how to use the OVHcloud API here.

  • Using one of the wrappers provided by OVHcloud which will handle all the authentication for you: Several wrappers (for Python, Java, C#, PHP, Go, etc.) are available on the OVHcloud GitHub. All the information needed to use them can be found in their respective README.md files.

Step 3: Take a look at the OVHcloud API endpoints

In order to know which endpoints are available through the OVHcloud API, you can have a look at the API web page and click on the Explore button.

To use this UI, you have to be logged in with your OVHcloud account credentials.

Here you can find all the Data Processing endpoints in the "cloud" section.

Raw tab in UI

Scroll down into the "cloud" section until you reach the /cloud/project/{serviceName}/dataProcessing/... endpoints.

Once you have expanded the section, you can try out the endpoints directly in the UI by clicking on them.

The "serviceName" parameter for each endpoint of the cloud section requires your Public Cloud project ID.

For further information about an endpoint, the Response Class tab under the Execute button shows what the API response will look like. Switch the tabs to display wrapper code examples.

Response class tab in UI

You can also see what your request body for a specific endpoint looks like in a json format via the Raw tab, which appears after the request has been executed once.

Raw tab in UI

For the POST on cloud/project/{serviceName}/dataProcessing/jobs endpoint, the engineParameters property is a mandatory list of key/value parameters.

To know which parameters to use, query the cloud/project/{serviceName}/dataProcessing/capabilites endpoint to list the several parameters depending on the engine you are using.

It will also show you which of those parameters are required and how they should be formed.

Step 4: Make sure the Data Processing service is activated for your Public Cloud project

Using the Data Processing endpoints requires the Data Processing service to be activated for your Public Cloud project.

You can check your project activation status by calling the GET cloud/project/{serviceName}/dataProcessing/activation endpoint.

You can also verify this in the OVHcloud Control Panel: In the Public Cloud section, select your project and click on Data Processing in the left-hand menu. If your project is activated, you should see a list (or an empty list) of jobs. The page invites you to activate your current project otherwise.

To learn how and why to activate your project, you can refer to this documentation: How to activate the Data Processing service for your cloud project.

Step 5: Use Data Processing with your application

By following the previous steps you have your credentials, your OVHcloud API access method and all the information about the request body and response for each of Data Processing endpoints.
Now you can begin to use the Data processing endpoints from your own application.

Bear in mind that for submitting a job, you need to have your application code stored in your Public Cloud Object Storage.
If this is your first time using Data Processing, we advise to try it out first by following this documentation for a Java job or this one for Python.

Go further

To learn more about using Data Processing and how to submit a job and process your data, we invite you to look at the Data Processing documentations page.

You can send your questions, suggestions or feedbacks in our community of users on https://community.ovh.com/en/ or in our public Gitter


Did you find this guide useful?

Please feel free to give any suggestions in order to improve this documentation.

Whether your feedback is about images, content, or structure, please share it, so that we can improve it together.

Your support requests will not be processed via this form. To do this, please use the "Create a ticket" form.

Thank you. Your feedback has been received.


These guides might also interest you...

OVHcloud Community

Access your community space. Ask questions, search for information, post content, and interact with other OVHcloud Community members.

Discuss with the OVHcloud community

In accordance with the 2006/112/CE Directive, modified on 01/01/2015, prices exclude VAT. VAT may vary according to the customer's country of residence.