Doc updates and install usability improvements

This makes some documentation updates to the top level README.md along
with slightly expanded usage instructions and minor changes to the install
script - mostly to correctly handle multiple networks and partial handling
for DPGs.

Author: hickeng

README.md

@@ -3,10 +3,29 @@
 
 # vSphere Integrated Containers
 
-VIC is a container runtime for vSphere, allowing developers familiar with Docker to develop in containers and deploy them alongside traditional VM-based workloads on vSphere clusters, and allowing for these workloads to be managed through the vSphere UI in a way familiar to existing vSphere admins.
+vSphere Integrated Containers (VIC) is a container runtime for vSphere, allowing developers familiar with Docker to develop in containers and deploy them alongside traditional VM-based workloads on vSphere clusters, and allowing for these workloads to be managed through the vSphere UI in a way familiar to existing vSphere admins.
 
 See [VIC Containers Architecture](arch.md) for a high level overview.
 
+## Project Status
+
+VIC can currently pull images, create and start containers in a very limited fashion - most significantly you cannot attach to a container to see its output or interact with it. Interaction is only via the network - containers will be exposed directly on the VCH external network rather than via port forwarding.
+
+We are working hard to add functionality while building out our [foundation](arch.md#port-layer-abstractions) so continue to watch the repo for new features. Initial focus is on the production end of the CI pipeline, building backwards towards developer laptop scenarios.
+
+This extremely limited set of current capabilities may come as a surprise to people who are familiar with [Project Bonneville](http://blogs.vmware.com/cloudnative/introducing-project-bonneville/) that was [reasonably fully featured](https://www.youtube.com/watch?v=XkFQw8ueT1w) when demonstrated at VMworld in 2015.
+Project Bonneville was research aimed at determining best approaches to enabling container workflows in a vSphere environment and therefore enabled a broad set of features, but not in a manner that made it a viable product for large scale consumption. Building on top of research code is a great shortcut for fast time-to-market, but does not provide a good foundation for an enterprise quality product. vSphere Integrated Containers is a full re-architecture and re-write, building off the knowledge gained during Project Bonneville while keeping almost zero code.
+
+
+## Installing
+
+Once built, the result can be installed with the following command. This is just an example - see the install help output for all options, and a slightly more indepth example [here](usage.md):
+```
+install.sh -g -t '<user>:<password>@<target-host>' -i <datastore-name> <vch-name>
+```
+
+Starting a container currently requires the container metadata be specified on the command line (environment, working directory, and absolute path to the command) as the image metadata parsing is still in progress (#195, #411). Container output is found in a log file on the datastore ([datastore]/containerid/containerid.log).
+
 ## Contributing
 
 See [CONTRIBUTING](CONTRIBUTING.md) for details on submitting changes and the contribution workflow.
@@ -31,16 +50,10 @@ make all
 ```
 
 There are three primary components generated by a full build, found in `$BIN` (the ./bin directory by default). The make targets used are the following:
-1. install.sh - make install
-2. appliance.iso - make appliance
-3. bootstrap.iso - make bootstrap
-
-## Installing
+1. install.sh - `make install`
+2. appliance.iso - `make appliance`
+3. bootstrap.iso - `make bootstrap`
 
-Once built, the result can be installed with the following command. This is just an example - see the install help output for all options:
-```
-install.sh -t 'user:password@my-esx-fqdn' -g target-cert-name -i datastore1 -p /ha-datacenter/host/my-esx-fqdn vch-name
-```
 
 ## Building binaries for development
 
@@ -68,6 +81,7 @@ $ make gvt vendor
 
 This will install the [gvt](https://github.com/FiloSottile/gvt) utility and retrieve the build dependencies via `gvt restore`
 
+
 ## Building the ISOs
 
 The component binaries above are packaged into ISO files, appliance.iso and bootstrap.iso, that are used by the installer. The generation of the ISOs is split into the following targets:

install/install.sh

@@ -43,12 +43,13 @@ datastore=${GOVC_DATASTORE}
 externalNet="VM Network"
 managementNet="VM Network"
 clientNet="VM Network"
+compute="/ha-datacenter/host/*"
 
 applianceIso="${DIR}/appliance.iso"
 bootstrapIso="${DIR}/bootstrap.iso"
 
 
-while getopts "fvt:g:p:i:d:e:m:b:a:c:x:y:" flag
+while getopts "fvt:gp:i:d:e:m:b:a:c:x:y:" flag
 do
   case $flag in
     v)
@@ -66,9 +67,9 @@ do
      # Required. Compute resource path
      # This should fully specify the path to the compute resource, e.g.
      # /ha-datacenter/host/myCluster/Resources/myRP
+     # /ha-datacenter/host/myCluster/host-fqdn
       compute=${OPTARG}
-      # primarily internal convenience for constructing other paths
-      datacenter=$(echo $compute | cut -d '/' -f 2)
+      export GOVC_HOST=${OPTARG}
       ;;
 
     d)
@@ -80,11 +81,16 @@ do
       # Required. Image datastore path - sets cdatastore too for default
       idatastore=${OPTARG}
       cdatastore=${OPTARG}
+      export GOVC_DATASTORE=${idatastore}
       ;;
 
     e)
       # Optional. The external network (can see hub.docker.com)
       externalNet=${OPTARG}
+      if [ "$clientNet" == "VM Network" ]; then
+        echo "Setting client network to specified external network"
+        clientNet=${externalNet}
+      fi
       : ${GOVC_NETWORK:=$externalNet}
       ;;
 
@@ -116,16 +122,7 @@ do
       ;;
 
     g)
-      # Optional. Generate the cert and key and store them in $OPTARG-{cert,key}.pem
-      keyf="${OPTARG}-key.pem"
-      certf="${OPTARG}-cert.pem"
-      # generate
-      echo "# Generating certificate/key pair - private key in ${keyf}"
-      openssl req -batch -nodes -new -x509  -keyout "${keyf}" -out "${certf}" > /dev/null 2>&1
-
-      # load the bits
-      key=$(cat "$keyf")
-      certificate=$(cat "$certf")
+      tlsGenerate="true"
       ;;
 
     x)
@@ -158,10 +155,26 @@ if [ -z "$1" -o -z "$GOVC_URL" -o -z "$vchName" -o -z "$compute" -o -z "$idatast
      usage
 fi
 
+if [ -n "$tlsGenerate" ]; then
+    # Optional. Generate the cert and key and store them in $OPTARG-{cert,key}.pem
+    keyf="${vchName}-key.pem"
+    certf="${vchName}-cert.pem"
+    # generate
+    echo "# Generating certificate/key pair - private key in ${keyf}"
+    openssl req -batch -nodes -new -x509  -keyout "${keyf}" -out "${certf}" > /dev/null 2>&1
+
+    # load the bits
+    key=$(cat "$keyf")
+    certificate=$(cat "$certf")
+fi
+
+# primarily internal convenience for constructing other paths
+datacenter=$(echo $compute | cut -d '/' -f 2)
 
 # if not set explicitly, bridge network has vch name
 : ${bridgeNet:=$vchName}
 
+
 # for now it's all insecure
 export GOVC_INSECURE=true
 export GOVC_PERSIST_SESSION=true
@@ -225,10 +238,6 @@ for net in "${!networks[@]}"; do
 done
 
 
-# ensure we have defaults for datastore and compute if needed
-if [ -z "${datastore}" ]; then
-   datastore=$(govc datastore.info | grep -e "^Name:" | awk '{print$2}')
-fi
 if [ -z "${compute}" ]; then
    compute=$(govc pool.info */Resources| grep -e "^\\s*Path:" | awk '{print$2}')
 fi
@@ -269,7 +278,7 @@ govc vm.power -on=true -vm.uuid="${uuid}" > ${powerstatus} 2>&1 || {
 echo "# Setting network identities"
 # Generate the network interface identiy map - MACs aren't generated until power on
 for net in "${!networks[@]}"; do
-  mac=$(govc device.ls -vm.uuid $uuid | grep "${net}" | grep ethernet | awk '{print $1}' | xargs govc device.info -vm.uuid $uuid | grep MAC | awk '{print$3}')
+  mac=$(govc device.info -vm.uuid $uuid -net "${net}" | grep MAC | awk '{print$3}')
   govc vm.change -e guestinfo.vch/networks/${networks[$net]}=$mac -vm.uuid="${uuid}"
 done
 # jump through some hoops to deal with quote behaviour for arrays

usage.md

@@ -0,0 +1,54 @@
+## Installing Virtual Integrated Containers
+
+The intent is that vSphere Integrated Containers (VIC) should not _require_ an installation step - deploying a [Virtual Container Host](doc/arch/vic-container-abstraction.md#virtual-container-host) (VHC) directly without any prior steps should always be possible. At the current time this is the only approach available.
+
+Installation will be required for capabilities such as [self-provisioning](doc/design/validating-proxy.md) and management network isolation via [vmomi proxy](doc/design/vmomi-authenticating-agent.md).
+
+## Deploying a Virtual Container Host
+
+### Requirements
+
+The first three requirements derive from a placeholder install script. These will be addressed in #121.
+
+1. bash - deploying a Virtual Container Host is currently done with an install script written in `bash`
+2. govc - [These directions](https://github.com/vmware/govmomi/tree/master/govc#govc) instruct how to install `govc` - release 0.5.0
+3. ESX - while this can and will function against vCenter, the placeholder installation script doesn't handle distributed port groups at this time
+4. DHCP - the VCH currently requires there be DHCP on the external network (-e flag if not "VM Network")
+
+
+Replace the `<fields>` in the example with values specific to your environment - this will install to the top level resource pool of the host (specify -g to generate certificates and configure TLS). Add -f to remove an existing folder or VM with the same name:
+```
+bin/install.sh -g -t '<user>:<password>@<target-host>' -i <datastore-name> <vch-name>
+```
+This will, if successful, produce output similar to the following:
+```
+# Generating certificate/key pair - private key in vch-name-key.pem
+# Logging into the target
+# Uploading ISOs
+[02-04-16 23:16:55] Uploading... OK
+[02-04-16 23:16:58] Uploading... OK
+# Creating vSwitch
+# Creating Portgroup
+# Creating the Virtual Container Host appliance
+# Adding network interfaces
+# Setting component configuration
+# Configuring TLS server
+# Powering on the Virtual Container Host
+# Setting network identities
+# Waiting for IP information
+#
+# SSH to appliance (default=root:password)
+# [email protected]
+#
+# Log server:
+# https://x.x.x.x:2378
+#
+# Connect to docker:
+# docker -H x.x.x.x:2376 --tls --tlscert='vch-name-cert.pem' --tlskey='vch-name-key.pem'
+DOCKER_OPTS="--tls --tlscert='vch-name-cert.pem' --tlskey='vch-name-key.pem'"
+DOCKER_HOST=x.x.x.x:2376
+```
+
+
+
+[Issues relating to Virtual Container Host deployment](https://github.com/vmware/vic/labels/component%2Fvic-machine)