Github Docker Builds

[tagyoureit]

I finally took a crack at automatically building Docker images via Github actions. Would be great to have a few people test this out and make sure it works as expected.

Docker homepage: https://hub.docker.com/r/tagyoureit/njspc-dp-combined-app

Currently, the only build is Ubuntu with njsPC and dP. Builds happen automatically with each commit to the main njsPC repo.

[treyrich]

I just pulled this down and started testing it out. My first question is, what is the reasoning behind combining the dashpanel and the controller into a single image as opposed to separating them? This would allow for the avoidance of pm2 entirely.

Glad to see this getting integrated!

[tagyoureit]

I'm not a Docker expert and was working based off the excellent work that @emes started a long time ago. I was going for more simplicity on the setup side, but can also offer other options such as separate images if there is a need or that's standard practice. PM2 has been our go-to app monitor for a long time and it plays nicely with docker and allowed for starting of separate apps inside the same image. Agree it wouldn't be needed if there were separate images.

[treyrich]

Got it. Yeah I can see how this would make sense if you don't have much experience with Docker. For sure the best practice is to break these out into separate images, and then let whatever container runtime you're using handle lifecycle management/monitoring.

Besides this being standard practice, and therefore what people expect, there are several practical benefits to doing things this way as well, many are spelled out well here in this SO answer.

I would be happy to put together a PR to assist with this if you'd like? Or I'm also happy to simply review/test and provide feedback. Let me know how I can help!

[tagyoureit]

A PR would be great if you want to make some changes. TIA.

[treyrich]

Ok. So I've created a draft PR. I'd love to get some input before I finalize it in case there was a reason for all of the different dockerfiles and all the commented code in the workflow. Assuming that there wasn't any reasoning behind those, I think it's pretty close. I do think that we'll want to change the name of the docker image as this isn't 'combined'.

Assuming this is acceptable I'm happy to put together a similar PR for the dash panel project, though I don't personally use that project.

[tagyoureit]

It looks good to me. I had trouble getting QEMU\Buildx set up... too many tutorials all competing with each other.

Most of the comments were part of the trial & error or for future builds. But they aren't needed. I put a couple minor questions in the PR. Thanks and please do put together a similar workflow for dashPanel.

[plewenson]

Hey there! With the recent updates you all pushed, I figured now might be a good time to try and update my NJPC and DashPanel docker containers.

As it's been over a year since I first got the containers up and running, I needed to refer back to the wiki and quickly ran into a several file/directory permission issues. Similar to this comment however I couldn't get his PID/GID solution to work as I'm not able to SSH/BASH into my container.

It could just be my incorrectly following directions or possibly something with my specific install which uses a Synology server running their Docker Container Manager which then interfaces to my pool over a WiFi RS-485 adapter.

Is it better to discuss them here or open an issue related to what I'm experiencing?

In general, when using the latest images and docker-compose.yaml:

• both containers fail to start with respective permission errors in the logs whenever trying to map the config.json files and data/log directories.
• when running the docker compose without volume mapping, the PoolController (8.0.4) starts successfully, but DashPanel 8.0.0 fails with a config.json error. When explicitly calling for the older 7.6.1 DashPanel image in the docker compose the container starts successfully.

I'm definitely not a docker expert, but I do have experience with docker-compose and running several containers that all map to local files/directories. The main reason is so when updating images, the config files and logs don't get lost when the container resets.

Happy to help share my settings and logs if needed.

[treyrich]

Hey, I'll take a stab at responding to this.

The first thing to realize is that in many docker images the processes are running as root, because that is the default behavior of docker. (See https://www.docker.com/blog/understanding-the-docker-user-instruction/) This makes many docker images really easy to use, but also makes them potential security risks, so the best practice is to limit a container to a non-root user.

Permissions in containers can be quite confusing, the reason is that the user ids and group ids from inside a running container correspond with the same user ids and group ids of the host. So what that means is that if I have a user named "bucky" whose user id is 1000 on my host, then I run a docker container with a user named "node" whose user id is 1000 in my container, for permissions purposes these two users are the same due to their IDs being the same. To give a practical example, if you created a volume for your container mapped to a directory on your host and the container created a new file in that directory, when you view the permissions of this new file from the host you would see the owner as "bucky" even though in the container "node" created the file.

Armed with this knowledge, we can typically figure out the permissions issues being faced in deploying a particular container. For example, if you're mapping a volume for your config with permissions restricting reading/writing to the owner of the file, and the owner of the file does not have a user id of 1000, then the njspc docker image would be unable to interact with that file as it is making use of the default "node" user provided by node docker images (which has an id of 1000).

I'm not sure if this fully addresses your questions, but I'm going to pause here and see if this is helpful. Let me know how far you get from what I've shared so far, and then if you're still having troubles and there's specific issues I'm happy to give some pointers in your particular environment if needed.

[plewenson]

@treyrich, thanks for the excellent info here! One learns more and more with each docker container they deploy and this explains a lot.

...the other thing that finally hit me is the the wiki still references the 'msmi/nodejs-poolcontroller' image and not the new image you and @tagyoureit worked on over at tagyoureit/njspc.

Armed with your explanation and after doing a bunch more reading, I updated the image I was pointing to, and got latest main controller running (8.0.4) using persistent storage on my local volumes!

Unfortunately, I couldn't get my docker compose set correctly to get a container going for the dashpanel using the new combined image.

On closer inspection, I see that your PR to build a new, separate image for the dashpanel failed. So I'm still using the last working build that worked for me (7.6.1).

Thanks again for all the effort here! It's really appreciated!