Installation on Mac

Installing Rodan

Clone a copy of the git repository in your desired folder. For the rest of these instructions, "your desired folder" will be referred to by $DEV_FOLDER. If done correctly, $DEV_FOLDER/Rodan/manage.py should be a valid file. Make sure that this clone is recursive; the RodanClient repository should recursively populate into $DEV_FOLDER/Rodan/client and a few other directories should follow suit. To perform a recursive clone:

git clone --recursive [email protected]:DDMAL/Rodan.git Rodan

RabbitMQ

Rodan uses Celery for performing all workflow operations. Celery uses the RabbitMQ back-end for message passing. Visit http://www.rabbitmq.com/download.html for instructions on installation.

Salt-Minion and various dependencies

• Salt is an open-source remote access program. Sending one command to a properly-designed Salt Master (read: server) can send the same command out to a certain batch of Salt Minions (read: clients) to remotely install or execute the command on all the machines. We use the "script" functionality where a bunch of pre-written commands can be sent in sequence to a set of minions; it just so happens that we'll be sending them to one minion at a time, but it makes your life a lot easier.
• There are installation instructions for Salt itself available at http://docs.saltstack.com/topics/installation/osx.html, but these seem to not work recently as saltstack seems to have been removed from Homebrew. That in mind...
• Install python, zmq, and swig using Homebrew (order may be important). If you're using a lab machine, Python may already be installed using Homebrew; speak to either Andrew to make sure you transfer ownership correctly. Install zmq using brew install zeromq --with-python. Install swig with brew install swig.
• Install dependencies and run pip install salt==0.16.0 to get both the Salt-Master and Salt-Minion packages. You will not need Salt-Master, but there's no way to only install minion.

Salt configuration

• Make sure that the file /etc/salt/minion exists. If it does not, create it and insert the line master: 132.206.14.141 (or whatever the master's current IP is).
  • You may need to sudo this command.

Starting Salt-Minion

• Enter sudo salt-minion --log-level=debug in a new Terminal tab to get debug statements. Starting the salt-minion with the master IP in its conf file will immediately send an authentication request to the master to let it know that a new machine is listening.
• If this spits out an error about salt-minion not being found, that just means that salt-minion wasn't in your $PATH variable and Terminal couldn't find it. Good news is that it should exist - run sudo /usr/local/share/python/salt-minion --log-level=debug instead and you should be in business.

Acknowledging the minion and starting the first step

• SSH into the master (132.206.14.141, account name is ddmaluser, ask someone for the password) and run the following commands:

$ sudo service salt-master status
    (this will report back whether the master process is running. If it says "stop/waiting", enter "sudo service salt-master start" and the process will start. You also may need to restart the minion on your own machine)
$ sudo salt-key -L
    (this should give you a list of all accepted minions and all unaccepted minions, you should see your machine under unaccepted)
$ sudo salt-key -A
    (this should accept all minions' requests for authentication)
$ sudo salt-key -L
    (note the machine that you just authenticated and make sure it's under "Accepted Keys")
$ sudo python /srv/salt/saltRenamer.py (your username on the minion) (desired folder to install everything in)
    (this will generate a salt script that has permissions tailored to your specific username's account on the minion machine, and will install everything to the folder. see below for an example)
$ sudo salt '(name)' state.sls stepOne(your username)
    (where (name) is the name of the minion you want to communicate with, single quotes are needed in the final command. the stepOne reference will be the file generated by the python script above, without the .sls extension. see below for an example)

The renamer command should look like sudo python saltRenamer.py ahorwitz /home/ahorwitz/development/ - note that the parentheses are removed and that there is a terminal slash on the folder name. I have not tested this using the ~ shortcut for your home folder, but don't risk it.

The state.sls command should look like sudo salt 'Jellyroll.local' state.sls stepOneahorwitz - note that the parentheses are removed.

Keep the salt-minion window open while you run this. If nothing starts moving on the minion screen (a lot of stuff will pop up almost immediately), make sure no one else is using the master currently, then restart the master by entering sudo service salt-master restart.

Twiddle your thumbs for a little bit here.

Between the Salt files

After stepOne.sls has finished running, there are two things that could happen:

1. A bunch of text pops up. This will be red, green, and blue if you're using the default Terminal and will end with a list titled "Summary" with subheaders "Succeeded" and "Failed." If 0 commands failed, you're good to go. This should be foolproof, but if some failed, send Andrew Horwitz an email and tell him to fix it so you can get work done.

2. A Terminal line pops up with no printout from the previous Salt command. This happens occasionally and unpredictably; my best guess is that the master and the minion lose communication. This is not a problem, as the process is still running. On the salt-minion ssh prompt on your own machine (not the one on Salt-Master), check to make sure it's actively updating - the process prints out a ps command that lists all running processes every so often. If the ps commands stop, you can proceed on to the next step; you can also re-run the command from the Master to make sure everything completed. If this happens twice, move on to the next step anyway.

If none failed, the reason why this is in two chunks is that Salt can not handle interactive bash scripts. There are a few that you need to take care of, but first...

chown

• Run sudo chown -R (your username):staff * on the directory on your machine where you had the Salt system install the dependencies. This will fix any files that may have been owned by root.

Cappuccino install

• Cappuccino is what our client is designed with. Put short, it's an interface between Objective-C and Javascript that allows for some really cool functionality on webpages.
• To install, on the minion, manually run $DEV_FOLDER/cappuccino/bootstrap.sh --clone-http --github-ref MASTER and follow instructions on http://www.cappuccino-project.org/learn/build-source.html. Installing using the default commands should work.
• Run the source ____ command that appears slightly above the NOTE: at the end of execution.

Virtualenv creation

If you've never worked with one before, a virtual environment can be described as a Python sandbox - activating the virtual environment overrides your default Python commands and lets you install extra/different/newer modules without affecting the system Python. For some reason, Salt can not create virtual environments on Mac. So...

• Run pip install virtualenv and make sure you have the program.
• Run virtualenv rodan_env from the Rodan home directory to create the virtual environment. (name is imperative as the second Salt script depends on it)
• Activate the virtual environment by running source $DEV_FOLDER/rodan_env/bin/activate from the Rodan home directory.
• Run pip install -r requirements.txt from $DEV_FOLDER/Rodan. You may receive a few notices that look like this:

$> Downloading/unpacking background-estimation==2.2.0pre2 (from -r requirements.txt (line 12))
$>   Could not find any downloads that satisfy the requirement background-estimation==2.2.0pre2 (from -r requirements.txt (line 12))
$> No distributions at all found for background-estimation==2.2.0pre2 (from -r requirements.txt (line 12))
$> Storing complete log in /Users/$YOUR_HOME/.pip/pip.log

• PIP was not able to install a package because it does not know where to download it from; this is because our custom packages are not in Python's databases. To bypass the installation of this requirement, edit $DEV_FOLDER/Rodan/requirements.txt and comment out (#) the line with the requirement and rerun the PIP install. This will happen for multiple packages.

• There is a version automatically pulled from the Salt-Master server at $DEV_FOLDER/requirements.txt, but it is likely out of date; use this or the hashmarks from this at your own risk.

• Twiddle your thumbs for a bit longer this time, but you at least get to watch it happen.

Django DB prep

• Django is our webserver! Our webserver needs a database to store the information in, and Django defaults to SQLite for database management. We recommend using PostgreSQL - if you do not have it on your computer yet, run brew install postgresql and install. Create a user and password that you are okay storing in plaintext (something like 'ddmal' or 'rodan' is fine for the password), create a database for Rodan (name is arbitrary), and start the server.
• Open $DEV_FOLDER/Rodan/rodan/settings_production.py.example with your favorite editor and change the SQLite reference under the "DATABASES" heading to the appropriate final extension. Name should be the name of the database, user and password should be the pair from above (in plaintext!), and host and port should not need to be changed for PostgreSQL. Uncomment these four lines.
• Copy (cp (origin) (destination)) this file to $DEV_FOLDER/Rodan/rodan/settings_production.py (removing the .example at the end)
• On the minion, run:

$ source $DEV_FOLDER/Rodan/rodan_env/bin/activate
(this turns on the virtual environment you created)
$ python $DEV_FOLDER/Rodan/manage.py syncdb
(this creates database entries for the default Django applications)
$ python $DEV_FOLDER/Rodan/manage.py schemamigration --initial rodan
(this creates a specification for the Rodan-specific tables)
$ python $DEV_FOLDER/Rodan/manage.py migrate
(this imports that specification)

CALL HORWITZ OVER TO YOUR MACHINE AT THIS POINT

If you are the first person to get here, call Horwitz and have him come over to your machine and watch what you're doing.

Install boost using "brew install boost --with-python".

##Second Salt script

• On the master, run sudo salt '(name from above)' state.sls stepTwo(your username). This takes a while, usually longer than any other step.
• The results will be similar to stepOne and may appear blank. You can do the same things as before to check the success, and if the server doesn't start in the next steps, try rerunning it and see what it says. If you do get results, you'll see the same "Summary:Succeeded/Failed" output; if failed is 0, proceed, otherwise frantically email me again with the output and ask me what went wrong.
• Everything should be in order now! Proceed on to the instructions for starting the server.
• Once you're sure the server started up correctly, be a dear and delete your custom stepOne/stepTwo files so that directory stays neat for future users. Make sure you don't delete the originals though.