My customer accountContact commercialWebmailOVHcloud Blog

Welcome to OVHcloud.

Log in to order, manage your products and services, and track your orders

Log in

Optimised method for uploading files to Object Storage

Optimised method for uploading files to Object Storage

When you want to upload large files to Object Storage (including videos or disk images for example), you can use the Swift OpenStack client in order to optimise the file transfer by breaking down the files. This guide explains how you can use this feature to upload your files to Object Storage more quickly.

Prerequisites

Swift OpenStack lets you store files of any size by breaking them down into several segments.

When a Swift client is used to upload a file, the Swift proxy server determines the correct storage node responsible for the data (based on a hash of the object name). Therefore, it is highly likely that the segments will be stored in several storage nodes, which means you can write your data at a higher speed.

As a result you can upload a 10 GB file in 100 X 100 MB segments:

root@server:~$ swift upload --segment-size 104857600 --segment-threads 100
container_name 10Gio.dat
Argument Description
--segment-size Segment size in bytes
--segment-threads Number of segments

You can measure the upload speed using iftop.


Did you find this guide useful?

Please feel free to give any suggestions in order to improve this documentation.

Whether your feedback is about images, content, or structure, please share it, so that we can improve it together.

Your support requests will not be processed via this form. To do this, please use the "Create a ticket" form.

Thank you. Your feedback has been received.


These guides might also interest you...

OVHcloud Community

Access your community space. Ask questions, search for information, post content, and interact with other OVHcloud Community members.

Discuss with the OVHcloud community

In accordance with the 2006/112/CE Directive, modified on 01/01/2015, prices incl. VAT may vary according to the customer's country of residence
(by default, the prices displayed are inclusive of the UK VAT in force).