Skip to content

Commit

Permalink
Further documentation and README updates for remote storate functiona…
Browse files Browse the repository at this point in the history
…lity.
  • Loading branch information
jcohen02 committed Sep 20, 2019
1 parent 655523f commit 2d753e0
Show file tree
Hide file tree
Showing 3 changed files with 274 additions and 118 deletions.
104 changes: 99 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ When/if the client subsequently submits the form associated with the filepond in

There are two different approaches for handling files that need to be stored permanently on a server after being uploaded from a filepond client via django-drf-filepond. _These two approaches are not mutually exclusive and you can choose to use one approach for some files and the other approach for other files if you wish._

###### 1. Manual handling of file storage
##### 1. Manual handling of file storage

Using this approach, you move the file initially stored as a temporary upload by _django-drf-filepond_ to a storage location of your choice and the file then becomes independent of _django-drf-filepond_. The following example shows how to lookup a temporary upload given its unique upload ID and move it to a permanent storage location. The temporary upload record is then deleted and _django-drf-filepond_ no longer has any awareness of the file:

Expand All @@ -182,11 +182,43 @@ os.rename(tu.get_file_path(), '/path/to/permanent/location/%s' % tu.upload_name)
tu.delete()
```

###### 2. Use django-drf-filepond's API to store a temporary upload to permanent storage
##### 2. Use django-drf-filepond's API to store a temporary upload to permanent storage

**Note:** You must use this approach for storing any files that you subsequently want to access using filepond's `load` function.

Using this approach, the file is stored to a location under the _django-drf-filepond_ file storage directory as set by the `DJANGO_DRF_FILEPOND_FILE_STORE_PATH` setting.
Using this approach, the file is stored either to local storage or to a remote storage service depending on the file store configuration you are using.

###### 2.1 `store_upload`

`store_upload` stores a temporary upload, uploaded as a result of adding it to the filepond component in a web page, to permanent storage.

If you have configured *django-drf-filepond* to use local file storage by setting the `DJANGO_DRF_FILEPOND_FILE_STORE_PATH` parameter in your application settings, the file will be stored to a location under this directory.

If you have configured a remote file store via *django-storages*, the stored upload will be sent to the configured storage backend via *django-storages*.

_**Parameters:**_

`upload_id`: The unique ID assigned to the upload by *django-drf-filepond* when the file was initially uploaded via filepond.

`destination_file_path`: The location where the file should be stored. This location will be appended to the base file storage location as defined using the `DJANGO_DRF_FILEPOND_FILE_STORE_PATH` parameter, or, for remote storage backends, the location configured using the relevant *django-storages* parameters. If you pass an absolute path beginning with `/`, the leading `/` will be removed. The path that you provide should also include the target filename.

_**Returns:**_

A `django_drf_filepond.models.StoredUpload` object representing the stored upload.

Raises `django.core.exceptions.ImproperlyConfigured` if using a local file store and `DJANGO_DRF_FILEPOND_FILE_STORE_PATH` has not been set.

Raises `ValueError` if:

- an `upload_id` is provided in an invalid format
- the `destination_file_path` is not provided
- a `django_drf_filepond.models.TemporaryUpload` record for the provided `upload_id` is not found





*Example:*

```python
from django_drf_filepond.api import store_upload
Expand All @@ -197,9 +229,71 @@ su = store_upload(upload_id, destination_file_path='target_dir/filename.ext')
# The path will created under the file store directory and the original
# temporary upload will be deleted.

```
```

###### 2.2 `get_stored_upload` / `get_stored_upload_file_data`

Get access to a stored upload and the associated file data.

`get_stored_upload`: Given an `upload_id`, return the associated `django_drf_filepond.models.StoredUpload` object.

The `destination_file_path` parameter passed to `store_upload` should be relative to the base upload location as defined by `DJANGO_DRF_FILEPOND_FILE_STORE_PATH`. If you pass a path that begins with `/`, the leading `/` will be removed and the path will be interpreted as being relative to `DJANGO_DRF_FILEPOND_FILE_STORE_PATH`. The path that you provide should include the filename that you would like the file stored as.
Throws `django_drf_filepond.models.StoredUpload.DoesNotExist` if a database record doesn't exist for the specified `upload_id`.

`get_stored_upload_file_data`: Given a StoredUpload object, return the file data for the upload as a Python [file-like object](https://docs.python.org/3/glossary.html#term-file-like-object).

_**Parameters:**_

`stored_upload`: A `django_drf_filepond.models.StoredUpload` object for which you want retrieve the file data.

_**Returns:**_

Returns a tuple `(filename, bytes_io)` where `filename` is a string representing the name of the stored file being returned and `bytes_io` is an `io.BytesIO` object from which the file data can be read. If an error occurs, raises an exception:

- `django_drf_filepond.exceptions.ConfigurationError`: Thrown if using a local file store and `DJANGO_DRF_FILEPOND_FILE_STORE_PATH` is not set or the specified location does not exist, or is not a directory.
- `FileNotFoundError`: Thrown if using a remote file store and the file store API reports that the file doesn't exist. If using a local file store, thrown if the file does not exist or the location is a directory and not a file.
- `IOError`: Thrown if using a local file store and reading the file fails.

*Example:*

```python
from django_drf_filepond.api import get_stored_upload
from django_drf_filepond.api import get_stored_upload_file_data

# Given a variable upload_id containing a 22-character unique
# upload ID representing a stored upload:
su = get_store_upload(upload_id)
(filename, bytes_io) = get_store_upload_file_data(su)
file_data = bytes_io.read()
```

###### 2.3 `delete_stored_upload`

`delete_stored_upload` deletes a stored upload record and, optionally, the associated file that is stored on either a local disk or a remote file storage service.

_**Parameters:**_

`upload_id`: The unique ID assigned to the upload by *django-drf-filepond* when the file was initially uploaded via filepond.

`delete_file`: `True` to delete the file associated with the record, `False` to leave the file in place.

_**Returns:**_

Returns `True` if the stored upload is deleted successfully, otherwise raises an exception:

- `django_drf_filepond.models.StoredUpload.DoesNotExist` exception if no upload exists for the specified `upload_id`.
- `django_drf_filepond.exceptions.ConfigurationError`: Thrown if using a local file store and `DJANGO_DRF_FILEPOND_FILE_STORE_PATH` is not set or the specified location does not exist, or is not a directory.
- `FileNotFoundError`: Thrown if using a remote file store and the file store API reports that the file doesn't exist. If using a local file store, thrown if the file does not exist or the location is a directory and not a file.
- `OSError`: Thrown if using a local file store and the file deletion fails.

```python
from django_drf_filepond.api import delete_stored_upload

# Given a variable upload_id containing a 22-character unique
# upload ID representing a stored upload:
delete_stored_upload(upload_id, delete_file=True)
# delete_file=True will delete the file from the local
# disk or the remote storage service.
```

### License

Expand Down
54 changes: 41 additions & 13 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -132,8 +132,8 @@ only required setting.
4.2 Remote storage of filepond uploads via django-storages
---------------------------------------------------------------------

The `django-storages <https://github.com/jschneier/django-storages>`_
library provides support for a number of different remote file storage
The `django-storages library <https://github.com/jschneier/django-storages>`_
provides support for a number of different remote file storage
backends. The `django-storages documentation <https://django-storages.readthedocs.io/en/latest>`_
lists the supported backends.

Expand Down Expand Up @@ -161,22 +161,50 @@ or, for the Amazon S3 backend::
DJANGO_DRF_FILEPOND_STORAGES_BACKEND = 'storages.backends.s3boto3.S3Boto3Storage'
...

For each storage backend, there are a number of additional django-storages
configuration options that must be specified. These are detailed in the
django-storages documentation.
For the Azure Storage backend, set::

...
DJANGO_DRF_FILEPOND_STORAGES_BACKEND = 'storages.backends.azure_storage.AzureStorage'
...

The following is an example of a complete set of configuration parameters
for using an Amazon S3 storage backend for django-drf-filepond via
django-storages::
For the Google Cloud Storage backend, set::

...
DJANGO_DRF_FILEPOND_STORAGES_BACKEND = 'storages.backends.gcloud.GoogleCloudStorage'
...

*django-storages* provides support for several other storage backends including
`Digital Ocean <https://django-storages.readthedocs.io/en/latest/backends/digital-ocean-spaces.html>`_
and `Dropbox <https://django-storages.readthedocs.io/en/latest/backends/dropbox.html>`_.

For each storage backend, there are a number of additional *django-storages*
configuration options that must be specified. These are detailed in the
*django-storages* documentation. The specific set of parameters that you
need to provide depends on your chosen storage backend configuration.

As an example, if you are using the Amazon S3 storage backend
and want to store uploads into a bucket named *filepond-uploads* in the
*eu-west-1* region, with the bucket and files set to be accessible only by
the user specified using the access/secret key, you would provide the
following set of parameters in your application's ``settings.py`` file::

DJANGO_DRF_FILEPOND_STORAGES_BACKEND = 'storages.backends.s3boto3.S3Boto3Storage'
AWS_ACCESS_KEY_ID = '<YOUR AWS ACCESS KEY>'
AWS_SECRET_ACCESS_KEY = '<YOUR AWS SECRET KEY>'
AWS_STORAGE_BUCKET_NAME = 'django-drf-filepond'
AWS_AUTO_CREATE_BUCKET = True
AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
AWS_S3_REGION_NAME = 'eu-west-1'
...
AWS_STORAGE_BUCKET_NAME = 'filepond-uploads'
AWS_DEFAULT_ACL = 'private'
AWS_BUCKET_ACL = 'private'
AWS_AUTO_CREATE_BUCKET = True

Note that the ACL for the bucket and the default ACL for files are set to
private. There may well be other security-related parameters that you will
want/need to set to ensure the security of the files on your chosen storage
backend. The configuration here provides an example but you should read the
*django-storages* docuemntation for your chosen backend and documentation
for the associated storage platform to ensure that you understand the
parameters that you are setting and any related potential security issues
that may result from your configuration.

.. note:: django-storages is now included as a core dependency of
django-drf-filepond. However, the different django-storages backends
Expand Down
Loading

0 comments on commit 2d753e0

Please sign in to comment.