Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README and basic example #40

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Update README and basic example #40

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 9 additions & 94 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,104 +2,19 @@

![Cali logo](docs/cali.png)

Oh, you found this repo, huh?
Cali is a library which lets you create CLIs to run containers, without the hassle of having to alias long `docker run` commands.
It also has added bonus of being able to modify how the container is run programatically, so your users don't need to worry.

Well, fair warning, we're still figuring out how the open source version of Cali should work.
By default, it binds your current working directory when running a container. But you can also tell it to clone a git repository and run from that.

Feel free to play with it, but while we're still on v0.x.x, be sure to pin to specific versions in your dependency manager of choice.

---

## Overview

**Ever wanted to be able to ensure that all your developers are working using the same versions of the same tools?**

**Want to be free from dowloading huge Vagrant boxes?**

**Want to be able to update and distribute new versions of your tooling quickly and easily?**

The answer to all these questions at our company was a resounding *YES!*

We needed a way to reduce the overheads associated with onboarding new starters, so instead of a document giving a list of software that needed to be downloaded, e.g. ChefDK, Vagrant, Terraform _etc. etc._ they could just download a single _thing_ which gave them a ready made development environment whether they used Mac or Linux PCs. Vagrant was the obvious choice and for a while it worked, but as we scaled up as a tech team both in numbers and in diversity of tools, this quickly became unmanageable. Different teams started wanting different boxes with different thing and different versions of those things. We soon ended up with several huge box files to maintain, they were slow to update, slow to upload and slow to download and horrible to maintain. We spent a lot of time troubleshooting...

> "Oh yes I have seen that error before, are you using Vagrant version X with VirtualBox version Y?

> "Err... not sure this will work with Windows sorry!"

Then along came Docker for Mac and shortly after, Docker for Windows and this opened the world of containers up to non-Linux users and gave us the ability to distribute our tools using Docker across Mac, Linux and Windows for the first time. Being a Chef house, the first experiment was to run ChefDK out of a docker container and was distributed as some bash aliases (early days!).

```
alias buildtools="docker run --rm -it -v \$PWD:/root/build -v /var/run/docker.sock:/var/run/docker.sock -v $HOME/.aws:/root/.aws -e \"BUILD_ROOT=\$PWD\" -v ~/.gitconfig:/root/.gitconfig ourprivatedockerregistry.io/buildtools:stable"
alias kitchen="buildtools kitchen"
```

Typing `kitchen` would then execute Test Kitchen for Chef against your $PWD and no matter what laptop you were using, as long as you had docker, all our devs were now using the same version of ChefDK for developing Chef code against.

Clearly this was suboptimal and as soon as more tools came along such as Terraform which wanted to be able to obtain short leased AWS credentials from HashiCorp Vault, a new solution was needed to distribute and orchestrate these containers was needed. Also, would it not be cool if that same tool could be run directly, or as a container on CI? Then your developers and your CI is using exactly the same stuff?

So we broke out those Go ninja skills and started using the Docker API to programatically manage these containers, remembering at each step that at some point in the future, this tool would itself be containerised and and would need to be able to schedule its ephemeral job containers either directly on the host its running on, or tantalisingly, on a swarm. Each tool, whether it be ChefDK or Terraform or whatever would either need to be able to work with $PWD or be able to check out some git repo and do its thing within that clone, all within containers. The final pieces of the puzzle were including auth with Hashicorp Vault to allow us to connect our corporate AD to get on-demand AWS credentials and also an update command which would self update the code when a new version was available on our internal Artifact repository.

This has been a great success and we are really happy with how it works and how little day to day support we have to provide our developers to use it. Its been such a success that we are getting requests to put functionality into it which we feel does not really belong there. Rather than saying *NO!* to such feature requests, we are doing the only sane thing and open sourcing the guts of the system to allow anybody to go and create their own development tools to do whatever they want.

## Building a CLI tool
We're still figuring out how the open source version of Cali should work, and it needs quite a bit of refactoring before it's ready for a v1.

```
package main

import "github.com/skybet/cali"

func main() {
cli := cali.NewCli("cali")
cli.SetShort("Example CLI tool")
cli.SetLong("A nice long description of what your tool actually does")

cmdTerraform(cli)

cli.Start()
}

func cmdTerraform(cli *cali.Cli) {

terraform := cli.NewCommand("terraform [command]")
terraform.SetShort("Run Terraform in an ephemeral container")
terraform.SetLong(`Starts a container for Terraform and attempts to run it against your code. There are two choices for code source; a local mount, or directly from a git repo.

Examples:

To build the contents of the current working directory using my_account as the AWS profile from the shared credentials file on this host.
# cali terraform plan -p my_account

Any addtional flags sent to the terraform command come after the --, e.g.
# cali terraform plan -- -state=environments/test/terraform.tfstate -var-file=environments/test/terraform.tfvars
# cali terraform -- plan -out plan.out
`)
terraform.Flags().StringP("profile", "p", "default", "Profile to use from the AWS shared credentials file")
terraform.BindFlags()

terraformTask := terraform.Task("hashicorp/terraform:0.9.9")
terraformTask.SetInitFunc(func(t *cali.Task, args []string) {
t.AddEnv("AWS_PROFILE", cli.FlagValues().GetString("profile"))
})
}
```

Now you can run containerised terraform :D

```
$ example terraform init
Terraform initialized in an empty directory!

The directory has no Terraform configuration files. You may begin working
with Terraform immediately by creating Terraform configuration files.
```

Or build from a git repo...

```
$ example terraform plan --git [email protected]:someone/terraform_code.git --git-branch master --git-path path/to/code
```
Feel free to play with it, but while we're still on v0.x.x, be sure to pin to specific versions in your dependency manager of choice.

## Docs

The API [docs](https://godoc.org/github.com/skybet/cali) are available on GoDoc or take a look (and help edit) [the wiki](https://github.com/skybet/cali/wiki).
The API [docs](https://godoc.org/github.com/skybet/cali) are available on GoDoc

For a basic example, see [examples/basic](https://github.com/skybet/cali/blob/master/examples/basic)

Take a look (and help edit) [the wiki](https://github.com/skybet/cali/wiki) for more usage examples
64 changes: 51 additions & 13 deletions examples/basic/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,31 @@ package main

import "github.com/skybet/cali"

func main() {
cli := cali.NewCli("cali")
var (
cli = cali.NewCli("example")
)

// For simplcity, just one init.
// In reality, for apps with many commands, you would split each command into its own file
func init() {
cli.SetShort("Example CLI tool")
cli.SetLong("A nice long description of what your tool actually does")

cmdTerraform(cli)

cli.Start()
cmdTerraform()
}

func cmdTerraform(cli *cali.Cli) {
// cmdTerraform defines a Cali command for running Terraform
func cmdTerraform() {

terraform := cli.NewCommand("terraform [command]")
terraform.SetShort("Run Terraform in an ephemeral container")
terraform.SetLong(`Starts a container for Terraform and attempts to run it against your code. There are two choices for code source; a local mount, or directly from a git repo.
// New command
command := cli.NewCommand("terraform [command]")

// Short helptext. Shows up with: example --help
command.SetShort("Run Terraform in an ephemeral container")

// Long helptext. Shows up with: example terraform --help
// This is where you'd outline detailed usage instructions
command.SetLong(`Starts a container for Terraform and attempts to run it against your code. There are two choices for code source; a local mount, or directly from a git repo.

Examples:

Expand All @@ -27,11 +37,39 @@ Examples:
# cali terraform plan -- -state=environments/test/terraform.tfstate -var-file=environments/test/terraform.tfvars
# cali terraform -- plan -out plan.out
`)
terraform.Flags().StringP("profile", "p", "default", "Profile to use from the AWS shared credentials file")
terraform.BindFlags()

terraformTask := terraform.Task("hashicorp/terraform:0.9.9")
terraformTask.SetInitFunc(func(t *cali.Task, args []string) {
// Flags are pflags:
// https://godoc.org/github.com/spf13/pflag
//
// e.g.
// example terraform --profile=foo
// example terraform -p foo
//
// In Cali, these are also Viper configs
// https://godoc.org/github.com/spf13/viper
//
// e.g. in .example.yml (in $PWD or $HOME)
// profile: foo

command.Flags().StringP("profile", "p", "default", "Profile to use from the AWS shared credentials file")
command.BindFlags()

// Task is "the thing the command does"
// Can either be a function or a string
// Here we use a string, which Cali interprets as a Docker image
// The Run function of this task will be essentially: docker run ... hashicorp/terraform:latest
task := command.Task("hashicorp/terraform:latest")

// SetInitFunc is optional, and runs before the Run function.
task.SetInitFunc(func(t *cali.Task, args []string) {
t.AddEnv("AWS_PROFILE", cli.FlagValues().GetString("profile"))

// TODO: explicitly bind ~/.aws, if creds aws
// https://github.com/skybet/cali/issues/7
// Relies on Cali understanding of Credentials, which I've not got an issue to track my ideas in yet
})
}

func main() {
cli.Start()
}