Deploying a Resnet34 Convolutional Network through FastAPI

This repository is a didactic template on serving a pretrained Resnet34 from PyTorch through FastAPI. We work towards turning the model tested on the notebook here into an endpoint for deploying an image classifier.

The Resnet34 model is created with the convert.py script, make convert will run it and place resnet34.onnx in the models folder.

We use PyTorch, GNU Make, ONNX, Docker, and Docker Compose in our sample deployment workflow.

Project Structure for Deployment

• src/: Python source code.
  • api.py: Main FastAPI application file.
• models/: Models directory.
  • resnet34.onnx: ResNet34 model for image classification.
• Dockerfile: Specifies how to build the Docker image.
• docker-compose.yaml: Docker Compose file to run the service.
• Makefile: Contains shortcuts for building and running Docker containers.
• requirements.txt: Lists the Python dependencies.

The folders fastapi_intro, markdown_notes, unsplash_images, and notebooks are content for the live workshop.

Getting Started

Reading through the following notes will give you an intuition about why are we doing things like we do:

• Resnet notebook to perform image classification (Google Colab)
• Intro to FastAPI
• Serving FastAPI apps through Uvicorn
• Understanding image and output validation with Pydantic
• Converting PyTorch models to ONNX format for serving
• Understanding the Dockerfile
• Understanding the docker-compose.yml file
• Understanding the relationship between GNU Make and Docker Compose

Prerequisites

Ensure you have the following installed on your system:

• Anaconda
• Docker
• Docker Compose

Build and Run the Service

1. Clone this repository:

   git clone https://github.com/andandandand/model-deployment-workshop

2. Navigate to the project directory:

   cd model-deployment-workshop
   

3. Create the resnet34.onnx network on the models folder:

   make convert 

4. Build the Docker image:

   make build

5. Run the Docker container:

   make run

6. Navigate to http://localhost:8000/docs to view the Swagger UI.

Development

To ease development, a volume is mapped from the src/ directory on your host to the corresponding directory in the container, allowing for changes made to the local source files to be immediately reflected inside the container.

License

Authors: Antonio Rueda-Toicen, Imran Kocabiyik

Berlin Computer Vision Group

This work is licensed under a Creative Commons Attribution 4.0 International License.

This is a tutorial repo. Please don't use it for production applications. We take no liability of any damages or losses.

Security Considerations

The application in this repo does not implement additional security measures, and the users are strongly encouraged to assess and implement suitable security controls, practices, and measures fitting their use case, especially when deploying in a production environment. This includes, but is not limited to, securing API endpoints, implementing proper authentication and authorization mechanisms, securing data in transit and at rest, and regularly updating dependencies to mitigate known vulnerabilities.

Users should also review and adjust user permissions within the Docker containers and on the host system to adhere to the principle of least privilege, ensuring that services and users only have the minimum level of access—or permissions—needed to accomplish a task.

Resources

• FastAPI
• Docker