Skip to content
This repository has been archived by the owner on Jul 21, 2024. It is now read-only.

HOWTO: Setup provider: Your own render farm (local) ‐ for Linux users

Jump to bottom
Rabit edited this page Oct 21, 2023 · 2 revisions

This document describes the steps to control your resources and build your own render farm using Linux - to use all your resources. It requires you to know some linux & shell basics and to show uses docker (which you can skip and just think about it as a virtual machine).

Overview

Basically local provider is just a stripped cloud provider - it will do almost nothing to setup the environment and will just use address and login/password to connect to manually started up Manager and use the manually started up Agents. So it's your responsibility to run all the Agents to use the available resources and the Manager to make it work as expected.

BlendNet structure

First of all, check How BlendNet is working - this will help you to understand the execution processes and give you a big picture about what's happening under the hood.

Setup process

1. Prepare a list of available resources

Almost anything can be used, if it can run headless blender and connect to the same network. Of course, the main restriction for render and composition is the amount of memory (RAM) available on the node. Also, make sure the nodes got enough disk space to store the project dependencies and results.

For example purposes we will use docker on the host system. It makes not much sense, but will allow to separate the resources and show the basic mechanics.

2. Prepare the TLS certificates

With the cloud providers - BlendNet creates a self-signed CA using Manager and openssl. To simplify the example we will generate key and self-signed certificate to use for all services. But you can create a proper setup with your own self-signed CA or even use third-party trusted CA to generate worldwide-trusted certificates.

To generate a simple key and certificate (not prod-ready) use the next openssl command:

host$ openssl req -x509 -nodes -newkey rsa:4096 \
        -keyout server.key -out server.crt -days 365 \
        -subj "/C=US/ST=N/L=N/O=N/OU=N/CN=blendet-service"
Can't load /home/user/.rnd into RNG
140071220240832:error:2406F079:random number generator:RAND_load_file:Cannot open file:../crypto/rand/randfile.c:88:Filename=/home/user/.rnd
Generating a RSA private key
............................................++++
............++++
writing new private key to 'server.key'
-----

And now you will get 2 files:

host$ ls -lh
total 8.0K
-rw-r--r-- 1 user user 1.9K Aug 30 18:12 server.crt
-rw------- 1 user user 3.2K Aug 30 18:11 server.key

These two files will be used for each Agent and Manager. For the Addon you will need just server.crt file.

3. Run the Agent on each available resource

Ok, now when the key and certificate are prepared - you need to copy them to the host work directory. Also, you will need to download the desired version of blender and the whole BlendNet addon.

  1. To simulate that we will run docker with Ubuntu 20.04 and mounting the dirs with BlendNet, blender and the certificate: 
    host$ docker run --rm --volume="$HOME/Work/state-of-the-art/BlendNet:/srv/blendnet:ro" \
        --volume="$HOME/local/blender-2.90.0-linux64:/srv/blender:ro" \
        --volume="$PWD:/tmp/cert:ro" \
        -m 4G --cpus 4 -it ubuntu:20.04
  2. Now you need to prepare the working directory, it should be writable for the service, because it will store caches and probably other useful things there: 
    docker$ cp -a /tmp/cert /workspace ; cd /workspace
  3. Next step - is to ensure all the required libs are installed, so let's install them: 
    docker$ apt update; apt install --no-install-recommends -y libxrender1 libxi6 libgl1 libice6 libsm6 libxkbcommon-x11-0
  4. Everything looks ready to start the Agent, so let's do that: 
    docker$ /srv/blender/blender -b -noaudio -P /srv/blendnet/agent.py
    Output: 
    Blender 2.90.0 (hash 0330d1af29c0 built 2020-08-31 11:54:54)
found bundled python: /srv/blender/2.90/python
INFO: Found provider "gcp"
WARN: Unable to load "gcp" provider due to init error: [Errno 2] No such file or directory: 'gcloud': 'gcloud'
INFO: Found provider "local"
INFO: Found provider "aws"
WARN: Metadata is not available
INFO: Importing base from "local" provider
DEBUG: Disabled stdout/stderr buffering
DEBUG: Creating Agent instance
INFO: using the cache directory "./BlendNet_cache"
DEBUG: Starting tasks watcher
DEBUG: Creating Processor
Adding PUT path "agent/*/config"
Adding GET path "agent"
Adding GET path "info"
Adding GET path "log"
Adding PUT path "task/*/file/**"
Adding GET path "status"
Adding GET path "task/*"
Adding GET path "task/*/details"
Adding GET path "task/*/file/**"
Adding GET path "task/*/file"
Adding GET path "task/*/messages"
Adding DELETE path "task/*"
Adding GET path "task/*/status/result/*"
Adding GET path "task/*/run"
Adding PUT path "task/*/config"
Adding GET path "task/*/status"
Adding GET path "task/*/stop"
Adding GET path "task"
Serving at ('', 9443)
  5. You need to find the IP address of the host where Agent is running. Usually it's enough to run ip a, ifconfig or check the network settings on the host. For my Agent it's 172.17.0.2.

We just started one Agent, but of course you can run as much Agents as you want.

Advanced:

  • You can check the Agent availability. You will need to find the IP address of the Agent, with docker it's quite simple: 
    host$ docker ps
CONTAINER ID  IMAGE         COMMAND      CREATED         STATUS         PORTS  NAMES
5bd71d3dc320  ubuntu:20.04  "/bin/bash"  15 minutes ago  Up 15 minutes         brave_goodall
    Get the IP address: 
    host$ docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' 5bd71d3dc320
172.17.0.2
    Run curl to check the output 
    host$ curl --user 'None:None' --insecure https://172.17.0.2:9443/api/v1/info
{"success": true, "data": {"engine": "Agent", "platform": {"python_info": "3.7.7 (default, Jul  9 2020, 12:52:49) \n[GCC 9.3.1 20200408 (Red Hat 9.3.1-2)]", "system": "Linux", "arch": "x86_64", "name": "0bdb7615b951", "details": "Linux-5.4.0-050400-generic-x86_64-with-debian-bullseye-sid", "cpu": 48}, "blender": {"version": [2, 90, 0], "version_string": "2.90.0", "build_date": "2020-08-31", "render_threads": 48}}}
    Unfortunately it can't determine the right amount of CPUs, bit at least it shows the information - means the Agent is working properly.

Specific Agent configs are described here.

4. Run the Manager

The Manager is needed to provide one endpoint to the multiple Agents and merge the results, so you will get the complete render. There is no way to run without Manager - let's start one:

  1. Same as in Agent
  2. Same as in Agent
  3. Same as in Agent
  4. Manager need to know what kind of CA cert to use, so let's copy the server.crt to use as CA: 
    docker$ cp server.crt ca.crt
  5. Let's run the Manager: 
    docker$ /srv/blender/blender -b -noaudio -P /srv/blendnet/manager.py
    Output: 
    Blender 2.90.0 (hash 0330d1af29c0 built 2020-08-31 11:54:54)
found bundled python: /srv/blender/2.90/python
INFO: Found provider "gcp"
WARN: Unable to load "gcp" provider due to init error: [Errno 2] No such file or directory: 'gcloud': 'gcloud'
INFO: Found provider "local"
INFO: Found provider "aws"
WARN: Metadata is not available
INFO: Importing base from "local" provider
DEBUG: Disabled stdout/stderr buffering
DEBUG: Creating Manager instance
INFO: using the cache directory "./BlendNet_cache"
DEBUG: Starting tasks watcher
DEBUG: Creating Processor
Adding GET path "agent/*/log"
Adding PUT path "agent/*/config"
Adding GET path "agent"
Adding GET path "info"
Adding GET path "log"
Adding PUT path "task/*/file/**"
Adding GET path "status"
Adding GET path "task/*"
Adding GET path "task/*/details"
Adding GET path "task/*/file/**"
Adding GET path "task/*/file"
Adding GET path "task/*/messages"
Adding DELETE path "task/*"
Adding GET path "task/*/status/result/*"
Adding GET path "task/*/run"
Adding PUT path "task/*/config"
Adding GET path "task/*/status"
Adding GET path "task/*/stop"
Adding GET path "task"
Serving at ('', 8443)
  6. You need to find the IP address of the host where Manager is running. Usually it's enough to run ip a, ifconfig or check the network settings on the host. For my Manager it's 172.17.0.3.

Advanced

  • You can ping the manager to check that it's working or not. You will need to find the IP address of the Manager, with docker it's quite simple: 
    host$ docker ps
CONTAINER ID  IMAGE         COMMAND      CREATED         STATUS         PORTS  NAMES
353d6cabdce7  ubuntu:20.04  "/bin/bash"  1 minute ago    Up 17 minutes         kind_satoshi
5bd71d3dc320  ubuntu:20.04  "/bin/bash"  15 minutes ago  Up 15 minutes         brave_goodall
    Get the IP address: 
    host$ docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' 353d6cabdce7
172.17.0.3
    Run curl to check the output 
    host$ curl --user 'None:None' --insecure https://172.17.0.3:8443/api/v1/status
{"success": true, "data": {"load": [5.02, 8.48, 7.47], "memory": {"MemTotal": 128808.98828125, "MemFree": 45314.27734375, "MemAvailable": 100215.40625}, "disk": {"total": 78124.0, "available": 24148.7265625}, "running": [], "terminating": false}}
  • You can add the manager.json file with your own configs, like this one: 
    {
  "name": "my-turbo-blendnet-manager",
  "ip": "<IP OR NS TO OVERRIDE THE resources Manager IP>",
  "internal_ip": "<IP OR NS TO OVERRIDE THE resources Manager INTERNAL_IP>",
  "listen_host": "",
  "listen_port": 8443,
  "auth_user": "<MANAGER USER>",
  "auth_password": "<MANAGER PASSWORD>",
  "agent_listen_port": 9443,
  "agent_auth_user": "None",
  "agent_auth_password": "None",
  "agent_upload_workers": 4
}

Specific Manager configs are described here.

5. Setup Addon and add Agent to the Manager

Now we need to tell the Manager about the started Agent to register and use it. But prior to that BlendNet Addon should be setup properly. It is different from the usual cloud provider, so let's see the whole process:

  1. First of all you need to go to Blender Preferences and choose Add-ons tab. After that find blendnet in the list of addons, open it and choose the Provider:

    Preferences - BlendNet Addon Local Provider

  2. Set the Manager configuration as described above:

    • Address - IP or DN of the Manager, in our case it's "172.17.0.3"
    • CA certificate - we using simple self-signed certificate, so choose it
    • Port - will be used to establish the connection, default "8443" is fine in our case
    • User and Password - in our case it's Manager default "None" and "None".

  3. Set the Agent configurations - it will be used by default in the popup later:

    • Port - we used Agent defaults, so "9443" will be fine for us
    • User and Password - defaults for Agent is "None" and "None", so use them.

  4. Close the preferences and switch to your Scene Render Properties tab. Make sure Cycles is chosen as the Render Engine. If everything is setup properly - you will see the next pic:

    Scene - BlendNet Addon Manager UI 1

  5. Now it's just an empty Manager, but we need the connected Agent, so scroll down and find Agents (0) panel with + button:

    Scene - BlendNet Addon Manager Agents UI

  6. Right now there are no Agents, so click to + button and put the required params into the popup:

    Scene - BlendNet Addon Agent popup

  7. Click Ok button and shortly you will see that the agent was added to the list. The same way you can add all the prepared Agents to the Manager.

    Scene - BlendNet Addon Agent added

Done

If everything was setup correctly - you can open your project and click Run Image Task to get the results of your hard work (please check the general practices on the First steps page to get how to do that).

Scene - BlendNet Addon Local Provider render

Advanced

  • You can use commandline to add Agent to the Manager: 
    host$ curl --user 'None:None' --insecure -X PUT https://172.17.0.3:8443/api/v1/agent/agent-1/config \
           --data '{"address": "172.17.0.2", "port": 9443, "auth_user": "None", "auth_password": "None"}'
Clone this wiki locally