Last updated 3rd June 2021
Web PaaS supports internal "storage as a service" to provide a file store that can be shared between different application containers.
The network storage service enables a new kind of
mount that refers to a shared service rather than to a local directory. Any application can use both
service mounts, or neither.
(This is a reference to a version of our network storage implementation, not to a version of a 3rd party application.)
Define the service
First, declare a new service in the
services.yaml file like so:
files: type: network-storage:1.0 disk: 256
This example creates a service named
files that is of type
network-storage, and gives it 256 MB of storage total.
Declare the mount
Second, add the following entry to your mounts list:
mounts: 'my/files': source: service service: files source_path: files
This block will declare a writeable mount on the application container at the path
my/files, which will be provided by the
files service defined above. The
source_path specifies the path within the network service that the mount points to. It is often easiest to have it match the name of the mount point itself but that is not required.
Note that you do not need to add a relationship to point to the
files service. That is handled automatically by the system.
The application container can now read from and write to the
my/files path just as if it were a local writeable mount.
There is a small performance hit for using a network mount over a local mount. In most cases it should not be noticeable. However, high-volume sequential file creation (that is, creating a large number of small files in rapid succession) may see a more significant performance hit. If that is something your application does regularly then a local mount will be more effective.
If your project contains more than one application (that is, multiple directories with their own
.platform.app.yaml files), they can all use the same network mounts if desired. If the
source_path is the same in both
.platform.app.yaml files then the files will be shared between both applications, even if the mount location is different.
It is also possible to have one application mount a
source_path that is a subdirectory of another application's mount. For example:
mounts: 'web/uploads': source: service service: files source_path: uploads
mounts: 'process': source: service service: files source_path: uploads/incoming 'done': source: service service: files source_path: uploads/done
In this example,
app1 will have access to the entire
uploads directory by writing to
app2, by contrast, will have two mounts that it can write to:
process mount will refer to the same directory as the
web/uploads/incoming directory does on
app1, and the
done mount will refer to the same directory as the
web/uploads/done directory on
When defining a Worker instance it is important to keep in mind what mount behavior is desired. Unless the
mounts block is defined within the
workers sections separately, a top level
mounts block will apply to both instances. However,
local mounts will be a separate storage area for each instance while
service mounts will refer to the same file system. For example:
name: app type: php:7.2 disk: 1024 mounts: 'network_dir': source: service service: files source_path: our_stuff 'local_dir': source: local source_path: my_stuff web: locations: "/": root: "public" passthru: "/index.php" workers: queue: commands: start: | php worker.php
In this case, both the web instance and the
queue worker will have two mount points:
local_dirmount on each will be independent and not connected to each other at all, and they will each take 1024 MB of space.
network_dirmount on each will point to the same network storage space on the
filesservice. They will both be able to read and write to it simultaneously. The amount of space it has available will depend on the
diskkey specified in
How do I give my workers access to my main application's files?
The most common use case for
network-storage is to allow a CMS-driven site to use a worker that has access to the same file mounts as the web-serving application. For that case, all that is needed is to set the necessary file mounts as
For example, the following
.platform.app.yaml file (fragment) will keep Drupal files directories shared between web and worker instances while keeping the Drush backup directory web-only (as it has no need to be shared). (This assumes a service named
files has already been defined in
name: 'app' type: 'php:7.2' relationships: database: 'db:mysql' hooks: # ... web: locations: '/': # ... disk: 1024 mounts: # The public and private files directories are # network mounts shared by web and workers. 'web/sites/default/files': source: service service: files source_path: files 'private': source: service service: files source_path: private # The backup, temp, and cache directories for # Drupal's CLI tools don't need to be shared. # It wouldn't hurt anything to make them network # shares, however. '/.drush': source: local source_path: drush 'tmp': source: local source_path: tmp 'drush-backups': source: local source_path: drush-backups '/.console': source: local source_path: console # Crons run on the web container, so they have the # same mounts as the web container. crons: drupal: spec: '*/20 * * * *' cmd: 'cd web ; drush core-cron' # The worker defined here will also have the same 6 mounts; # 2 of them will be shared with the web container, # the other 4 will be local to the worker. workers: queue: commands: start: | cd web && drush queue-run myqueue
How can I migrate a local storage to a network storage?
There is no automated way of transferring data from one storage type to another. However, the process is fundamentally "just" moving files around on disk, so it is reasonably straightforward.
Suppose you have this mount configuration:
mounts: web/uploads: source: local source_path: uploads
And want to move that to a network storage mount. The following approximate steps will do so with a minimum of service interruption.
1) Add a new
network-storage service, named
files, that has at least enough space for your existing files with some buffer. You may need to increase your plan's disk size to accommodate it.
2) Add a new mount to the network storage service on a non-public directory:
mounts: new-uploads: source: service service: files source_path: uploads
(Remember the `source_path` can be the same since they're on different storage services.)
3) Deploy these changes. Then use
rsync to copy all files from the local mount to the network mount. (Be careful of the trailing
rsync -avz web/uploads/* new-uploads/
4) Reverse the mounts. That is, point the
web/uploads directory to the network mount instead:
mounts: web/uploads: source: service service: files source_path: uploads old-uploads: source: local source_path: uploads
Commit and push that. Test to make sure the network files are accessible.
5) Cleanup. First, run another rsync just to make sure any files uploaded during the transition are not lost. (Note the command is different here.)
rsync -avz old-uploads/* web/uploads/
Once you're confident all the files are accounted for, delete the entire contents of `old-uploads`. If you do not, the files will remain on disk but inaccessible, just eating up disk space needlessly. Once that's done you can remove the `old-uploads` mount and push again to finish the process. You are also free to reduce the `disk` size in the `.platform.app.yaml` file if desired, but make sure to leave enough for any remaining local mounts.
Did you find this guide useful?
Please feel free to give any suggestions in order to improve this documentation.
Whether your feedback is about images, content, or structure, please share it, so that we can improve it together.
Your support requests will not be processed via this form. To do this, please use the "Create a ticket" form.
Thank you. Your feedback has been received.