Added support for directly using path to binaries

NEW FEATURES
- Implemented Issue 21 "Add support for directly using path to binaries"

BUGS FIXED
- unpack would not act on old tarball where files were not
  explicitly marked as regular.
TESTING
- Added test for Issue 21

Author: datacharmer

Diff for: Changelog

@@ -1,3 +1,15 @@
+1.8.0	08-Jul-2018
+	NEW FEATURES
+	- Implemented Issue 21 "Add support for directly using path to binaries"
+
+	BUGS FIXED
+	- unpack would not act on old tarball where files were not 
+	  explicitly marked as regular.
+	- Fixed Issue 22 "dbdeployer should check whether the binaries are for the
+	current OS"
+	TESTING
+	- Added test for Issue 21
+
 1.7.0	01-Jul-2018
 	NEW FEATURES
 	- Added option for custom history file for each sandbox

Diff for: README.md

@@ -3,7 +3,7 @@
 [DBdeployer](https://github.com/datacharmer/dbdeployer) is a tool that deploys MySQL database servers easily.
 This is a port of [MySQL-Sandbox](https://github.com/datacharmer/mysql-sandbox), originally written in Perl, and re-designed from the ground up in [Go](https://golang.org). See the [features comparison](https://github.com/datacharmer/dbdeployer/blob/master/docs/features.md) for more detail.
 
-Documentation updated for version 1.7.0 (30-Jun-2018 20:44 UTC)
+Documentation updated for version 1.8.0 (08-Jul-2018 08:03 UTC)
 
 ## Installation
 
@@ -13,7 +13,7 @@ Get the one for your O.S. from [dbdeployer releases](https://github.com/datachar
 
 For example:
 
-    $ VERSION=1.7.0
+    $ VERSION=1.8.0
     $ origin=https://github.com/datacharmer/dbdeployer/releases/download/$VERSION
     $ wget $origin/dbdeployer-$VERSION.linux.tar.gz
     $ tar -xzf dbdeployer-$VERSION.linux.tar.gz
@@ -47,7 +47,7 @@ For example:
 The program doesn't have any dependencies. Everything is included in the binary. Calling *dbdeployer* without arguments or with ``--help`` will show the main help screen.
 
     $ dbdeployer --version
-    dbdeployer version 1.7.0
+    dbdeployer version 1.8.0
 
 
     $ dbdeployer -h
@@ -179,6 +179,7 @@ The easiest command is ``deploy single``, which installs a single sandbox.
     For example:
     	dbdeployer deploy single 5.7     # deploys the latest release of 5.7.x
     	dbdeployer deploy single 5.7.21  # deploys a specific release
+    	dbdeployer deploy single /path/to/5.7.21  # deploys a specific release in a given path
 
     For this command to work, there must be a directory $HOME/opt/mysql/5.7.21, containing
     the binary files from mysql-5.7.21-$YOUR_OS-x86_64.tar.gz
@@ -233,6 +234,7 @@ The ``deploy replication`` command will install a master and two or more slaves,
 
     		$ dbdeployer deploy replication 5.7    # deploys highest revision for 5.7
     		$ dbdeployer deploy replication 5.7.21 # deploys a specific revision
+    		$ dbdeployer deploy replication /path/to/5.7.21 # deploys a specific revision in a given path
     		# (implies topology = master-slave)
 
     		$ dbdeployer deploy --topology=master-slave replication 5.7
@@ -293,17 +295,62 @@ You can issue the command ``dbdeployer deploy single 8.0``, and it will use 8.0.
 
 ## Multiple sandboxes, same version and type
 
-If you want to deploy several instances of the same version and the same type (for example two single sandboxes of 8.0.4, or two group replication instances with different single-primary setting) you can specify the data directory name and the ports manually.
+If you want to deploy several instances of the same version and the same type (for example two single sandboxes of 8.0.4, or two replication instances with different settings) you can specify the data directory name and the ports manually.
 
     $ dbdeployer deploy single 8.0.4
     # will deploy in msb_8_0_4 using port 8004
 
     $ dbdeployer deploy single 8.0.4 --sandbox-directory=msb2_8_0_4
     # will deploy in msb2_8_0_4 using port 8005 (which dbdeployer detects and uses)
 
-    $ dbdeployer deploy replication 8.0.4 --sandbox-directory=rsandbox2_8_0_4 --base-port=18600
+    $ dbdeployer deploy replication 8.0.4 --concurrent
+    # will deploy replication in rsandbox_8_0_4 using default calculated ports 19009, 19010, 19011
+
+    $ dbdeployer deploy replication 8.0.4 \
+        --gtid \
+        --sandbox-directory=rsandbox2_8_0_4 \
+        --base-port=18600 --concurrent
     # will deploy replication in rsandbox2_8_0_4 using ports 18601, 18602, 18603
 
+## Using the direct path to the expanded tarball
+
+If you have a custom organization of expanded tarballs, you may want to use the direct path to the binaries, instead of a combination of ``--sandbox-binary`` and the version name.
+
+For example, let's assume your binaries are organized as follows:
+
+    $HOME/opt/
+             /percona/
+                     /5.7.21
+                     /5.7.22
+                     /8.0.11
+            /mysql/
+                  /5.7.21
+                  /5.7.22
+                  /8.0.11
+
+You can deploy a single sandbox for a Percona server version 5.7.22 using any of the following approaches:
+
+    #1
+    dbdeployer deploy single --sandbox-binary=$HOME/opt/percona 5.7.22
+
+    #2
+    dbdeployer deploy single $HOME/opt/percona/5.7.22
+
+    #3
+    dbdeployer defaults update sandbox-binary $HOME/opt/percona 
+    dbdeployer deploy single 5.7.22
+
+    #4
+    export SANDBOX_BINARY=$HOME/opt/percona 
+    dbdeployer deploy single 5.7.22
+
+Methods #1 and #2 are equivalent. They set the sandbox binary directory temporarily to a new one, and use it for the current deployement
+
+Methods #3 and #4  will set the sandbox binary directory permanently, with the difference that #3 is set for any invocation of dbdeployer system-wide (in a different terminal window, it will use the new value,) while #4 is set only for the current session (in a different terminal window, it will still use the default.)
+
+Be aware that, using this kind of organization may see conflicts during deployment. For example, after installing Percona Server 5.7.22, if you want to install MySQL 5.7.22 you will need to specify a ``--sandbox-directory`` explicitly.
+Instead, if you use the prefix approach defined in the "standard and non-standard basedir names," conflicts should be avoided.
+
 ## Ports management
 
 dbdeployer will try using the default port for each sandbox whenever possible. For single sandboxes, the port will be the version number without dots: 5.7.22 will deploy on port 5722. For multiple sandboxes, the port number is defined by using a prefix number (visible in the defaults: ``dbdeployer defaults list``) + the port number + the revision number (for some topologies multiplied by 100.)
@@ -755,18 +802,18 @@ Should you need to compile your own binaries for dbdeployer, follow these steps:
 2. Run ``go get github.com/datacharmer/dbdeployer``.  This will import all the code that is needed to build dbdeployer.
 3. Change directory to ``$GOPATH/src/github.com/datacharmer/dbdeployer``.
 4. From the folder ``./pflag``, copy the file ``string_slice.go`` to ``$GOPATH/src/github.com/spf13/pflag``.
-5. Run ``./build.sh {linux|OSX} 1.7.0``
-6. If you need the docs enabled binaries (see the section "Generating additional documentation") run ``MKDOCS=1 ./build.sh {linux|OSX} 1.7.0``
+5. Run ``./build.sh {linux|OSX} 1.8.0``
+6. If you need the docs enabled binaries (see the section "Generating additional documentation") run ``MKDOCS=1 ./build.sh {linux|OSX} 1.8.0``
 
 ## Generating additional documentation
 
 Between this file and [the API API list](https://github.com/datacharmer/dbdeployer/blob/master/docs/API/API-1.1.md), you have all the existing documentation for dbdeployer.
 Should you need additional formats, though, dbdeployer is able to generate them on-the-fly. Tou will need the docs-enabled binaries: in the distribution list, you will find:
 
-* dbdeployer-1.7.0-docs.linux.tar.gz
-* dbdeployer-1.7.0-docs.osx.tar.gz
-* dbdeployer-1.7.0.linux.tar.gz
-* dbdeployer-1.7.0.osx.tar.gz
+* dbdeployer-1.8.0-docs.linux.tar.gz
+* dbdeployer-1.8.0-docs.osx.tar.gz
+* dbdeployer-1.8.0.linux.tar.gz
+* dbdeployer-1.8.0.osx.tar.gz
 
 The executables containing ``-docs`` in their name have the same capabilities of the regular ones, but in addition they can run the *hidden* command ``tree``, with alias ``docs``.
 

Diff for: cmd/replication.go

@@ -83,6 +83,7 @@ Use the "unpack" command to get the tarball into the right directory.
 	Example: `
 		$ dbdeployer deploy replication 5.7    # deploys highest revision for 5.7
 		$ dbdeployer deploy replication 5.7.21 # deploys a specific revision
+		$ dbdeployer deploy replication /path/to/5.7.21 # deploys a specific revision in a given path
 		# (implies topology = master-slave)
 
 		$ dbdeployer deploy --topology=master-slave replication 5.7

Diff for: cmd/single.go

@@ -19,7 +19,6 @@ import (
 	"fmt"
 	"os"
 	"path"
-	"path/filepath"
 	"regexp"
 	"strings"
 
@@ -75,11 +74,7 @@ func GetAbsolutePathFromFlag(cmd *cobra.Command, name string) string {
 	if err != nil {
 		common.Exit(1, fmt.Sprintf("Error getting flag value for --%s", name))
 	}
-	value, err = filepath.Abs(value)
-	if err != nil {
-		common.Exit(1, fmt.Sprintf("Error getting absolute path for %s", value))
-	}
-	return value
+	return common.AbsolutePath(value)
 }
 
 func check_if_abridged_version(version, basedir string) string {
@@ -114,6 +109,7 @@ func FillSdef(cmd *cobra.Command, args []string) sandbox.SandboxDef {
 	basedir := GetAbsolutePathFromFlag(cmd, "sandbox-binary")
 
 	sd.BasedirName = args[0]
+	version_from_option := false
 	sd.Version, _ = flags.GetString("binary-version")
 	if sd.Version == "" {
 		sd.Version = args[0]
@@ -122,6 +118,34 @@ func FillSdef(cmd *cobra.Command, args []string) sandbox.SandboxDef {
 		if old_version != sd.Version {
 			sd.BasedirName = sd.Version
 		}
+	} else {
+		version_from_option = true
+	}
+
+	if common.DirExists(sd.BasedirName) {
+		sd.BasedirName = common.RemoveTrailingSlash(sd.BasedirName)
+		sd.BasedirName = common.AbsolutePath(sd.BasedirName)
+		// fmt.Printf("OLD bd <%s> - v: <%s>\n",basedir, sd.Version )
+		target := sd.BasedirName
+		old_basedir := basedir
+		basedir = common.DirName(sd.BasedirName)
+		if old_basedir != defaults.Defaults().SandboxBinary {
+			// basedir was set using either an environment variable
+			// or a command line option
+			if old_basedir != basedir {
+				// The new basedir is different from the one given by command line or env
+				common.Exit(1, "The Sandbox Binary directory was set twice,",
+					fmt.Sprintf(" using conflicting values: '%s' and '%s' ", old_basedir, basedir))
+			}
+		}
+		sd.BasedirName = common.BaseName(sd.BasedirName)
+		if !version_from_option {
+			sd.Version = sd.BasedirName
+		}
+		if !common.IsVersion(sd.Version) {
+			common.Exit(1, fmt.Sprintf("No version detected for directory %s", target))
+		}
+		// fmt.Printf("NEW bd <%s> - v: <%s>\n",basedir, sd.Version )
 	}
 
 	sd.Port = common.VersionToPort(sd.Version)
@@ -136,11 +160,14 @@ func FillSdef(cmd *cobra.Command, args []string) sandbox.SandboxDef {
 		sd.Port = sd.UserPort
 	}
 
-	sd.Basedir = path.Join(basedir, sd.Version)
+	sd.Basedir = path.Join(basedir, sd.BasedirName)
 	// sd.Basedir = path.Join(basedir, args[0])
 	if !common.DirExists(sd.Basedir) {
 		common.Exit(1, fmt.Sprintf("basedir '%s' not found", sd.Basedir))
 	}
+
+	common.CheckTarballOperatingSystem(sd.Basedir)
+
 	sd.SandboxDir = GetAbsolutePathFromFlag(cmd, "sandbox-home")
 
 	common.CheckSandboxDir(sd.SandboxDir)
@@ -231,6 +258,7 @@ containing an unpacked tarball. The place where these directories are found is d
 For example:
 	dbdeployer deploy single 5.7     # deploys the latest release of 5.7.x
 	dbdeployer deploy single 5.7.21  # deploys a specific release
+	dbdeployer deploy single /path/to/5.7.21  # deploys a specific release in a given path
 
 For this command to work, there must be a directory $HOME/opt/mysql/5.7.21, containing
 the binary files from mysql-5.7.21-$YOUR_OS-x86_64.tar.gz

Diff for: common/fileutil.go

@@ -303,6 +303,14 @@ func DirName(filename string) string {
 	return filepath.Dir(filename)
 }
 
+func AbsolutePath(value string) string {
+	filename, err := filepath.Abs(value)
+	if err != nil {
+		Exit(1, fmt.Sprintf("Error getting absolute path for %s", value))
+	}
+	return filename
+}
+
 func Mkdir(dir_name string) {
 	err := os.Mkdir(dir_name, 0755)
 	if err != nil {

Diff for: common/strutils.go

@@ -215,3 +215,8 @@ func Exit(exit_code int, messages ...string) {
 	}
 	os.Exit(exit_code)
 }
+
+func RemoveTrailingSlash(s string) string {
+    re := regexp.MustCompile(`/$`)
+	return re.ReplaceAllString(s, "")
+}

Diff for: common/version.go

@@ -15,9 +15,9 @@
 
 package common
 
-var VersionDef string = "1.7.0" // 2018-06-23
+var VersionDef string = "1.8.0" // 2018-07-05
 
 // Compatible version is the version used to mark compatible archives (templates, configuration).
 // It is usually major.minor.0, except when we are at version 0.x, when
 // every revision may bring incompatibility
-var CompatibleVersion string = "1.7.0" // 2018-06-23
+var CompatibleVersion string = "1.8.0" // 2018-07-05