This project is used to demonstrate privacy-respecting services that can be under the control of an adversary.
- Python 3.6
- Bottle
- Gunicorn
- SQLAlchemy
The dependencies can be installed using
cd PrivacyService
# Setup virtualenv
virtualenv .
source bin/activate
pip3 install -r requirements.txtProject dependencies can be installed using pip3, as shown above. It is highly recommended to use a virtual environment to avoid cluttering the global namespace. Afterwards, the server can be invoked using either
# Invoke the server directly for local development only
python3 pservice.py
# or use gunicorn
gunicorn --workers=4 pservice:appNote that the server port specified in the config.py file is only used in the first case. When using gunicorn, gunicorn's default parameters are used instead.
The development server used by bottle itself adds the Content-Length and Content-Type request headers to all incoming requests, clashing with superfluous header detection. Gunicorn does not do this; The easiest solution is to temporarily turn off superfluous header detection for local development.
Per the gunicorn documentation, it is highly recommended to run gunicorn behind a proxy server such as nginx, otherwise it is trivial to launch denial-of-service attacks against the service. This is also where it is appropriate to add TLS support. The underlying application does not need to handle encryption and decryption in this case. A full guide on how to setup the server using nginx remains to be done.
The server can be configured by editing the python config.py file.
If a request contains superfluous HTTP headers, that is headers that are not required to answer the request, the request will be denied by default by replying with HTTP 400 (Bad Request). This is done in order to prevent fingerprinting attacks. Clients that send superfluous headers cannot use the service and a service can be tested if it accepts superfluous HTTP headers.
This behaviour is disabled by default and can be enabled by by setting the SUPERFLUOUS_HEADERS_ALLOWED preference in the config.py file to False. Note that bottle's development server adds request headers to incoming requests (see above).
If an API endpoint is invalid, e.g., because a specified key for the key-value storage API has an invalid format, the service will reply with 404 (Not Found).
If values in the request have an unexpected format, e.g., if a non-numerical value is used for the Content-Length header, the service will reply with 400 (Bad Request). This will also be replied if the request contains superfluous HTTP headers and SUPERFLUOUS_HEADERS_ALLOWED was not set.
It is possible to configure the application to log all incoming requests, including the type of access (Retrieve, Update, Delete), the key used, the full HTTP request and values before and after the requested action. Logging functionality is enabled globally using the REQUEST_LOGGING key in config.py. This is done in order to demonstrate the information collected by a potentially malicious service operator. Your IP address will be logged as well.
With REQUEST_LOGGING enabled, (only) requests containing the header X-AppPETs-BadProvider with a non-zero value are logged.
The demo service at https://services.app-pets.org has logging enabled. Your requests are logged if you provide the X-AppPETs-BadProvider header with your requests.
The Key-value storage can be used to store and retrieve values for given keys. The keys are 256 bit hexadecimal encoded strings consisting of 64 characters. The value is handled as binary.
There is no user authentication or data protection, so anyone knowing a key can do retrieve or overwrite its value. Overwriting is done by uploading data for an existing key. The previous value stored for that key is then lost.
Additionally, there is neither detection nor prevention of uploading non-encrypted data.
The entry point for the API is:
https://services.app-pets.org/storage/v1/<key>
To upload data use POST and attach the asset in binary format to the HTTP body of the request. To download data use GET and the value will be attached in binary form to the HTTP body of the response. To delete entries use the DELETE method.
If there already exists a value for <key> the previously stored value will be overwritten.
POST /storage/v1/<key> HTTP/1.1
Host: services.app-pets.org
Content-Type: application/octet-stream
Content-Length: <content.length>
<content>HTTP/1.0 200 OK
Server: BaseHTTP/0.6 Python/3.6.0
Date: Wed, 25 Jan 2017 13:00:00 GMTGET /storage/v1/<key> HTTP/1.1
Host: services.app-pets.orgIf no value exists for the given <key>, the service will reply with HTTP 404 (Not Found).
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: <content.length>
<content>DELETE /storage/v1/<key> HTTP/1.1
Host: services.app-pets.orgRegardless if the key is existing the response will be 200. The response will only indicate an error if the request is erroneous, e.g., if <key> has an invalid format.
HTTP/1.0 200 OK
Server: BaseHTTP/0.6 Python/3.6.0
Date: Wed, 25 Jan 2017 13:00:00 GMT