Doc changes

README.md

@@ -1,90 +1,15 @@
-[1]: https://github.com/opsgang/fetch
 # libs
 
-... reusable scripts - retrieve specific versions with [opsgang/fetch][1] :)
+... reusable script bundles to make automation easy and consistent.
 
-[![Run Status](https://api.shippable.com/projects/5a588d01e0a7bb07007efbd7/badge?branch=master)](https://app.shippable.com/github/opsgang/libs)
-
-**You need to have GNU coreutils installed for these scripts to work.**
-
-Mac users: We use `realpath`, and GNU versions of `sed`, `awk`, `sort`.
-
-BSD flavours will not work, so `homebrew` some GNU-ey goodness if you plan to use these scripts locally.
-
-## USAGE
-
-Typically we source all scripts under bash/habitual/ in our scripts
-
-To grab these to a local dir e.g. ./lib, you can use fetch:
-
-```bash
-
-# retrieve latest v 1.x (but < 2.x) of files under bash/habitual
-fetch --repo="https://github.com/opsgang/libs" --tag="~> 1.0" --source-path=/bash/habitual ./lib/habitual
-
-# Now source all habitual libs in your scripts with something like:
-for lib in $(find ./habitual -type f | grep -v 'README' | grep -vP '\.(awk|md|markdown|txt)$'); do
-    ! . $lib && echo "ERROR $0: ... could not source $lib" && return 1
-done
-
-```
-
-## DOCUMENTATION
-
-Generated from simple inline markup.
-
-To generate it yourself:
+Built and tested with [shippable](https://shippable.com).
 
-```bash
-cd bash
-libs=$(find ./ -path './t' -prune -o -name '*.functions' -print)
-for lib in $libs; do awk -f bashdoc-to-md.awk $lib > $lib.md ; done
-```
-
-## BUILDS / PKGS
-
-Currently these run tests against the bash functions.
-
-TODO:
-
-On a git tag push event, create bundles of related scripts and upload
-as binary assets to a github release. e.g. all utility scripts for running terraform,
-or building an AMI _opsgang_ style.
-
-These could then be retrieved with [opsgang/fetch][1].
-
-## TESTS
-
-```bash
-
-cd ./bash # all tests must be run from this dir.
-
-# to run all tests for a particular script e.g. habitual/std.functions:
-t/habitual/std.functions
-
-# to run individual tests for a script e.g. for habitual/functions
-t/habitual/std.functions t_can_source_multiple_files t_check_var_defined
-
-# to run all tests for all scripts under ./habitual
-t() {
-    local suite="$1" rc=0
-    [[ "$(basename $(realpath .))" != "bash" ]] && echo 'ERROR: run from ./bash dir' && return 1
-    [[ -z "$suite" ]] && echo 'ERROR: pass suite name' >&2 && return 1
-
-    libs=$(find $suite -path './t' -prune -o -name '*.functions' -print)
-    for lib in $libs; do
-        rc=0
-        f="t/${lib#./}"
-        [[ ! -x "$f" ]] && echo "no tests for $lib" && continue
-        $f || rc=1
-    done
+[![Run Status](https://api.shippable.com/projects/5a588d01e0a7bb07007efbd7/badge?branch=master)](https://app.shippable.com/github/opsgang/libs)
 
-    return $rc
-}
+---
 
-# ... then you can run things like ...
-t ./ || echo "FAILURES"       # run tests for all functions.
-t habitual || echo "FAILURES" # run tests for all functions under habitual
+## AVAILABLE LIBS
 
-```
+* bash: see [this README](bash/README.md)
 
+---

bash/README.md

@@ -0,0 +1,167 @@
+[1]: https://github.com/opsgang/fetch#tag-constraint-expressions "OG fetch tag constraints"
+[2]: https://github.com/opsgang/fetch/releases "OG fetch releases"
+[3]: bashdoc-to-md.awk.md
+[4]: https://github.com/settings/tokens
+# libs/bash
+
+_Make your life easier and consistent when scripting automation, by sourcing these libs._
+
+Tested bundles are available on the [releases page](https://github.com/opsgang/libs/releases).
+
+> **You need to have GNU coreutils, and realpath installed for these scripts to work.**
+>
+> They also rely on GNU versions of `grep`, `sed`, `awk`, `sort` and `find`.
+>
+> BSD flavours will not work, so Mac users should `homebrew` some GNU-ey goodness
+> before using these scripts locally.
+
+* [BUNDLES](#bundles)
+    * [HOWTO: USE ONE](#howto-use-one)
+    * [HOWTO: GET ONE](#howto-get-one)
+
+* [TESTS](#tests)
+
+* [DOCS - bashdoc](#docs)
+
+---
+
+---
+
+## BUNDLES
+
+On a release, we create attached bundles (_read_ .tgz) that contains all libs required to perform
+a specific automation goal. e.g to run [terraform](https://terraform.io) in a consistent and
+non-interactive way.
+
+The bundles all contain an entrypoint script that does all of the sourcing of the
+other libs for you.
+
+---
+
+### HOWTO: USE ONE
+
+Just source the entrypoint script which is always called `opsgang.sourcelibs`.
+
+e.g. If you've installed the bundle under /my/dir ...
+
+```bash
+# In my calling script, use the opsgang libs:
+. /my/dir/opsgang.sourcelibs || exit 1
+```
+
+---
+
+### HOWTO: GET ONE
+
+#### a quick aside - the github access token
+
+> Github will rate-limit your downloads a lot more if you don't send
+> an authorization token with the request.
+>
+> Just create a github [personal access token][4] for the user that will
+> perform the downloads, and add this to the curl commands in the examples.
+
+e.g.
+
+* add to the curl `-H 'Authorization: token {{mytoken}}'`, where {{mytoken}}
+    is replaced with your shiny github token.
+
+#### ... I want an exact version or the latest
+
+> You can download and untgz a version (or latest) in one line if you want.
+
+e.g. to download _latest_ release of `terraform_run.tgz` bundle to `/my/dir`:
+
+```bash
+curl -sSL -H 'Accept: application/octet-stream' \
+    https://github.com/opsgang/libs/releases/download/latest/terraform_run.tgz \
+    | tar -xvz -C /my/dir
+```
+
+... or replace /latest in url with desired tag e.g. /0.0.1
+
+#### ... I want to specify a version constraint like '~>1.0'
+
+> You can use the repo's download helper to install the version of a bundle that
+> meets your criteria.
+
+e.g to download and untgz the latest 1.x version of terraform\_run.gz to `/my/dir/`:
+
+```bash
+curl --retry 3 -sSL
+    https://raw.githubusercontent.com/opsgang/libs/master/bash/bundles/dl_release.sh \
+| GITHUB_TOKEN=$token bash -s -- terraform_run.gz '~>1.0' /my/dir
+```
+
+> GITHUB\_TOKEN can be omitted, but your downloads will be rate-limited.
+> See 
+>
+> The local download dir will be created if needed.
+>
+> See the [opsgang/fetch README][1] for a description of version constraints.
+>
+> If you already have [opsgang/fetch][2] in your \$PATH, dl\_release.sh will run quicker.
+
+---
+
+---
+
+## TESTS
+
+Each lib file has a corresponding test script that is run as part of CI.
+
+Specific Tests in a test script can be run (or all by default).
+
+A convenience function is in the example below, if you want to run all available test scripts.
+
+```bash
+
+cd ./bash # all tests must be run from this dir.
+
+# to run all tests for a particular script e.g. habitual/std.functions:
+t/habitual/std.functions
+
+# to run individual tests for a script e.g. for habitual/functions
+t/habitual/std.functions t_can_source_multiple_files t_check_var_defined
+
+# to run all tests for all scripts under ./habitual
+t() {
+    local suite="$1" rc=0
+    [[ "$(basename $(realpath .))" != "bash" ]] && echo 'ERROR: run from ./bash dir' && return 1
+    [[ -z "$suite" ]] && echo 'ERROR: pass suite name' >&2 && return 1
+
+    libs=$(find $suite -path './t' -prune -o -name '*.functions' -print)
+    for lib in $libs; do
+        rc=0
+        f="t/${lib#./}"
+        [[ ! -x "$f" ]] && echo "no tests for $lib" && continue
+        $f || rc=1
+    done
+
+    return $rc
+}
+
+# ... then you can run things like ...
+t ./ || echo "FAILURES"       # run tests for all functions.
+t habitual || echo "FAILURES" # run tests for all functions under habitual
+
+```
+
+---
+
+---
+
+## DOCS
+
+> Each lib file has its own .md documentation that sits alongside it in this repo.
+
+The markdown is generated from a simple inline markup ([see here for more info][3]).
+
+To generate it yourself for all lib files.
+
+```bash
+cd bash
+libs=$(find ./ -path './t' -prune -o -name '*.functions' -print)
+for lib in $libs; do awk -f bashdoc-to-md.awk $lib > $lib.md ; done
+```
+