Monitoring your jobs using Grafana and the Apache Spark dashboard
Find out how to monitor your jobs using Grafana and the Apache Spark dashboard
Last updated 06th March, 2020
Objective
This guide will tell you how you can monitor you jobs after submitting them.
In this guide, we are assuming that you're using the OVHcloud Manager to use the Data Processing platform and you have already submitted a job.
To read an introduction about Data Processing service you can visit Data Processing Overview.
Requirements
- Access to the OVHcloud Manager
- A job that has been already submitted to Data Processing
Instructions
Apache Spark Dashboard
After submitting your job, follow these steps to open its Apache Spark dashboard:
- Find your job in the list of jobs in the Data Processing page of the OVHcloud Manager.
- Click on the name of the job to open its dashboard.
- Click on
Open job user interfacein the Actions panel to open the Apache Spark dashboard. If you do this right after submitting a job, you will see a warm up page for a few seconds while your job is starting.
The Apache Spark dashboard for your job will then start automatically.
Grafana Dashboard
After you submit a job to the Data Processing platform, follow these steps to monitor your job using a Grafana dashboard.
- Find your job in the list of jobs in the Data Processing page in the OVHcloud Manager.
- Click on the name of the job to open its dashboard.
- In the dashboard, click on
Monitor jobin the Actions panel.
- Copy and save the metrics token that is now displayed in the popup.
- Open https://grafana.dataconvergence.ovh.com:3000/d/VhF0A_Gnz/ovhcloud-data-processing-overview?orgId=2 and login with your OVHcloud account.
- Paste the metrics token that you copied and saved before in the
tokenfield. - You should now be able to see your job's dashboard if your job ID is already set in the
jobIdfield. Otherwise, you can simply write the ID of the jobs you want to monitor in this field.
Go further
To learn more about using Data Processing and how to submit a job and process your data, we invite you to look at Data Processing documentations page.
You can send your questions, suggestions or feedbacks in our community of users on https://community.ovh.com/en/ or on our Discord in the channel #dataprocessing-spark
Did you find this guide useful?
Please feel free to give any suggestions in order to improve this documentation.
Whether your feedback is about images, content, or structure, please share it, so that we can improve it together.
Your support requests will not be processed via this form. To do this, please use the "Create a ticket" form.
Thank you. Your feedback has been received.