A Tecthulhu JSon to GPIO, I2C and Serial gateway
This repository contains the implementation of a gateway server design to run on the Debian based Raspberry Pi 3.
The requirements for this software are current driven by several data sources from the Magnus Reawakens project hosted by Niantic.
- A photograph and text describing what is believed to be the source of JSon data feed, https://plus.google.com/u/1/+HRichardLoeb/posts/4jzaJ8J9W7c
- The presence of aan RFID capability from the above the purpose and capability of which is unknown, possibly to prevent theft or tampering of the Tecthulhu device
The gateway is intended to act as a client to a JSon data feed using either stock TCP/IP server ports, or using a Serial port. In the case of the TCP/IP ports it is assumed that an HTTP 1.1 client will be used. In the case of the serial device line delimited JSon is assumed to be present. As time passes details concerning the final host interface for the JSon will become available and this will be modified to meet the changing requirements.
The gateway is also intended to respond to the JSon messages by triggering GPIO I2C pins, or sending serial data to a serial device.
Audio playback will also be supported triggered using JSon from the tecthulhu module.
The Team NorCal project has multiple software sub projects that will be implemented by multiple teams. As a result the interfaces between the Pi and Audrino modules will be done using the simplest integration possible to reduce risk and complexity.
The ASCII output messages for driving the Audrino will be specified by the Audrino project.
The ASCII protocol is used between the pi-gateway and arduinos that successfully respond to
the '**********************\n' commands with a Magnus response.
On the console of the pi-gateway when these devices are detected you will you
see a message such as "arduino at /dev/ttyACM0 has the role of 'Magnus Resonators Node'".
All messages between the arduinos and the pi-gateway are line delimited.
Percentage values in this message format are representing using ASCII characters ranging from ' ' to 'R' inclusive in 2% steps. For example a ' ' represents 0%, '!' for 2%, '9' would be 50 and so on, until 'R' which would be 100. To calculate the percentage from a character treat the characters as integers and subtract the value for ' ' 32, then multiply that result by 2. A full table of values can be generated by going to https://play.golang.org/p/BQh4hYLaRs and running the code.
The basic format of the message is as follows :
Fnnnnnnnnp12345678mmmm\n
F represents the current faction that holds the portal. When in uppercase this represents a change in the owning faction. 'r', or 'R' for resistance, 'n', or 'N' for neutral, 'e', or 'E' for enlightened.
The nnnnnnnn component of our message is an ASCII string of the resonator levels on the portal arranged started with the eastern point and going counter-clockwise. Due east being in position 0, NW at position 1, north at position 2 and so on.
This is followed by a single character percentage formatted value of the precentage health of the portal.
Then, a set of 8 percentage encoded single character values (1-8) each one representing the health of the resonators starting with E and then going counter clockwise.
The next set of four ASCII characters contain information about the mods (m) that are present on the portal.
- ' ' - No mod present in this slot
- '0' - FA Force Amp
- '1' - HS-C Heat Shield, Common
- '2' - HS-R Heat Shield, Rare
- '3' - HS-VR Heat Shield, Very Rare
- '4' - LA-R Link Amplifier, Rare
- '5' - LA-VR Link Amplifier, Very Rare
- '6' - SBUL SoftBank Ultra Link
- '7' - MH-C MultiHack, Common
- '8' - MH-R MultiHack, Rare
- '9' - MH-VR MultiHack, Very Rare
- 'A' - PS-C Portal Shield, Common
- 'B' - PS-R Portal Shield, Rare
- 'C' - PS-VR Portal Sheild, Very Rare
- 'D' - AXA AXS Shield
- 'E' - T Turret
Finally a '\n' character terminates the message.
Native builds on the Pi are the default , this is primarily how the code will be maintained and extended when onsite at Camp Navarro. When using the Pi 3 builds can be performed by doing a git clone of the guthub repo then doing the following
sudo apt-get install -y portaudio19-dev libasound2-dev libvorbis-dev alsa-utils alsa-tools alsa-oss alsaplayer mpg321 alsaplayer-alsa alsa-base cd ~ sudo apt-get install -y wget wget --quiet -O go.tgz https://storage.googleapis.com/golang/go1.8.3.linux-armv6l.tar.gz tar xzfo go.tgz export PATH=$PATH:/home/$USER/go/bin export GOROOT=/home/$USER/go git clone https://github.com/TeamNorCal/pi-gateway.git cd pi-gateway export GOPATH=`pwd` export GOBIN=`pwd`/bin go get -d . && go build -o bin/pi-gateway .
Having done this the binaries will be found in the bin directory of your cloned repo.
The instructions for performing cross platform builds are included inside the Dockerfile.
Two proposals, python and Go, for an implementation language are afloat and await a decision.
The Golang proposal is to use a compiled language to allow the Pi processor to reduce Pi CPU and memory requirements. This is raised due to a desire to handle audio, TCP/IP IO, as well as device IO. In pactice audio demands are taking 1 of the Quad CPUs available.
The pi-gateway project supports cross platform builds for the gateway allowing it to be developed on a non Pi host, including AWS or GCP, and then binaries targetted at Pi or other ARM processors. This is done using Docker.
The pi-gateway can be installed using systemd. The pi-gateway.service file should be modified to ensure that the appropriate tecthulhu server referenced, and then copied into the /lib/systemd/system/ directory.
These instructions assume that the pi-gateway has been git cloned into your /home/pi/pi-gateway directory and that the binaries and other files are provided to the systemd daemon from this location. If you have used a binary manually copied to the system or other arrangement you will need to modify the pi-gateway.service file before using the following instructions.
If you are testing with an HttpRoller instance you should examine the HttpRoller directory for the HttpRoller.service file and install this as well using the instructions found in that repository. When configuring the HttpRoller daemon you must use the path argument to point the server at the location for the pi-gateway simulator scenarios. Another option is only install the pi-gateway as a daemon and then star the HttpRoller using a shell for testing purposes, stopping and starting it to change scenarios.
If HttpRoller is not used then you must modify the pi-gateway.service file to point at an alternative REST server offering JSON payloads for use with the gateway.
By default logging for this server is sent to the system journal accessed using the journalctl command.
Once the service files have been copied they can be included by using the commands:
sudo cp pi-gateway.service /lib/systemd/system/pi-gateway.service sudo chmod 644 /lib/systemd/system/pi-gateway.service sudo systemctl daemon-reload sudo systemctl enable pi-gateway.service
After headless installation the Pi should be rebooted. Logging output from the pi-gateway unit can be seen using the following command:
sudo journalctl -u pi-gateway
To follow the output use the '-f' option.
The number of simulators are available. All offer pros and cons and also more than one can be used for testing purposes.
A simulator is also provided for the tecthulhu in the form of JSON files that can be served using a static web server or a testing web server such as the HttpRoller.
HttpRoller is not application aware and does not need a DSL. You only need know the tecthulhu payloads and drop files in to the appropriate directories based upon the time slot you want them served.
HttpRoller provides the ability to change the URLs being served and the content of web pages as a function of time. This allow complex scenarios such as a portal being attacked and then claimed by a new factor to be simulated as a loop.
A number of scenarios are included in the pi-gateway repository at pi-gateway/simulator/scenarios/...
cd pi-gateway export GOPATH=`pwd` export GOBIN=`pwd`/bin go get github.com/karlmutch/HttpRoller bin/HttpRoller -listen=127.0.0.1:12345 -path="./simulator/scenarios/stable"
Manually retriving information from the simulator is as simple as:
wget -O- --quiet 127.0.0.1:12345/module/status/json
A more complex test case drains all resonators over a 115 second period and can be run with the test data generator as follows:
bin/HttpRoller -listen=127.0.0.1:12345 -path="./simulator/scenarios/XM_drain_all"
A programatic means of generating test JSON files dose exist to simplify complex ingress portal state changes. The github.com/karlmutch/tecthulhu-generator tool is intended as an adhoc tool written in Go that can be modified to create large numbers of json files for test cases across time.
More tests are being written.
A number of projects existing on the public internet for serving tecthulhu web pages and can be found at
http://tecthulhu.boop.blue/.... Source code can be found at
https://pypi.python.org/pypi/tecthulhu/1.1.
Not all URIs are supported by this implementation and controlling the scenarios for portal states is up to the provider.
A further self hosted simulator can be found at
https://github.com/bbulkow/MagnusFlora. This solution requires self hosting and is probably the best application aware simulator and comes with an ansible role. It does however have lots of moving parts, is complicated to modify DSL based files, deploy, and operate.
Various APIs for use with the Pi to support the CPU requirements for a Quad core CPU that can concurrently use the music and audio APIs along with being able to respond to JSon and GPIO requests will be trialed prior to the event.
Audio support - https://github.com/xlab/portaudio-goi, https://github.com/xlab/libvpx-go Native GPIO - https://github.com/stianeikeland/go-rpio Serial IO - https://reprage.com/post/using-golang-to-connect-raspberrypi-and-arduino
https://github.com/kidoman/embd Custom Hardware (if needed, not likely but an out if we need it) - https://gobot.io/documentation/platforms/raspi/ Googles Low Level Library - https://periph.io/
##Audio support based upon triggered events
In order to run the audio portion of the project the following libraries need to be installed.
sudo apt-get install portaudio19-dev libasound2-dev libvorbis-dev alsa-utils alsa-tools alsa-oss alsaplayer mpg321 alsaplayer-alsa alsa-base
You will also need to enable the dmix audio output otherwise the software will fail. Instructions for enabling the default audio mixer can be found at, http://alsa.opensrc.org/Dmix#5._The_simple_approach:.
The raspberry pi kernel also needs its support for mmap enabled. You should have the following line added to your /boot/config.txt file:
dtoverlay=i2s-mmap
The following page has information about setting the volume, https://bbs.archlinux.org/viewtopic.php?id=134532. Use the PCM device as the name not Master.
The best page I have found for the HiFiBerry is at https://www.hifiberry.com/build/documentation/configuring-linux-3-18-x/
The PCM mix device can vary between installs, one page that has an approach for configuring the sound can be found at, https://www.raspberrypi.org/forums/viewtopic.php?p=1147994.
Another at the reference mentioned above.
http://investigate.ingress.com/2017/03/16/glyph-music/
#Does not work sudo apt-get install libsndfile-dev go get github.com/mkb218/gosndfile/sndfile
#libav-tools apt-get install libav-tools
PulseAudio notes and bluetooth notes at http://blog.mrverrall.co.uk/2013/01/raspberry-pi-a2dp-bluetooth-audio.html https://gist.github.com/oleq/24e09112b07464acbda1 sudo apt-get install libpulse-dev https://github.com/mesilliac/pulse-simple/ https://github.com/moriyoshi/pulsego
#Simple but does lots of graphics, has direct OGG example for decoding sudo apt-get install libsfml-dev https://bitbucket.org/krepa098/gosfml2/wiki/Home
https://github.com/vchimishuk/alsa-go https://github.com/cvanderschuere/alsa-go https://github.com/runningwild/gorbis