Merge commit 'c0dc38dd9f2cf40af9f06f5b984b53ce4f3610be' into develop

core/CHANGELOG.md

@@ -3,43 +3,52 @@ Change Log
 
 ### Release 0.1.2-alpha (Date: tbd.)
 
-- Not released yet.
-
+- Developer/alpha release (**NOT FOR PRODUCTION USE!**)
+- Features:
+  + Extracted non-core plugins, allow easy plugin installation through
+    `sy plugin (list|install|remove)` #26/#104
+    - Shipped plugins now only 'local'
+    - Installable plugins:
+      [FTP](https://github.com/syncany/syncany-plugin-ftp),
+      [SFTP](https://github.com/syncany/syncany-plugin-sftp) (no host checking),
+      [WebDAV](https://github.com/syncany/syncany-plugin-webdav) (HTTP only),
+      [Amazon S3](https://github.com/syncany/syncany-plugin-s3)
+  + Ignore files using wildcards in .syignore (e.g. *.bak, *.r??) #108
+  + Added Arch Linux 'syncany-git' package #99
+  + Allow speicifying HTTP(S)/WebDAV proxy and other global system properties #109
+- Bugfixes
+  + Fix semantic in TransferManager `test()` (incl. all plugins) #103/#102
+  + WebDAV plugin fix to create "multichunks"/"databases" folder #110
+  + Fix "Plugin not supported" stack trace #111
+  + Windows build script fix for "Could not normalize path" #107
+  + Fix database file name leak of username and hostname #114
+  + Check plugin compatibility before installing (check appMinVersion) #104
+  + Don't ignore local/remote notifications if sync already running #88
+  + Uninstall plugins on Windows (JAR locked) #113/#117
+  + Rotate logs to max. 4x25 MB #116
+  + Fix multichunk resource close issue #118/#120
+
 ### Release 0.1.1-alpha (Date: 14 Apr 2014)
 
+- Developer/alpha release (**NOT FOR PRODUCTION USE!**)
 - Features:
   + Ignoring files using .syignore file #66/#77
   + Arch Linux package support; release version #80 and git version #99
   + Additional command-specific --help texts
 - Windows-specific: 
   + Add Syncany binaries to PATH environment variable during setup #84/#91
   + Fixed HSQLDB-path issue #98
-- Bugfixes
+- Bugfixes:
   + Timezone fix in tests #78/#90
   + Reference issue "Cannot determine file content for checksum" #92/#94
   + Atomic 'init' command (rollback on failure) #95/#96
-- Other things
+- Other things:
   + Tests for 'connect' command  
   + Tests for .syignore
 
 ### Release 0.1.0-alpha (Date: 30 March 2014)
 
-- First developer/alpha release (NOT FOR PRODUCTION USE!)
+- First developer/alpha release (**NOT FOR PRODUCTION USE!**)
 - Command line interface (CLI) with commands
   + init: initialize local folder and remote repository
-  + connect: connect to an existing remote repository
-  + up: index and upload local files
-  + down: download changes and apply locally
-  + status: list local changes
-  + ls-remote: list remote changes
-  + watch: watches local dir, subscribes to pub/sub, and calls down/up 
-           command in a set interval
-  + restore: restores a given set of files (experimental)
-  + log: Outputs formatted file histories (experimental)
-  + genlink: Generates syncany:// links to share
-  + cleanup: Deletes old file versions and frees remote space
-- Storage plugins:
-  + Local: Allows to store repository files in a local/mounted folder 
-  + FTP: Allows the use of an FTP folder as repository
-  + WebDAV: Allows using a WebDAV folder as repository (currently no HTTPS)
-
+  + connect

core/README.md

@@ -5,7 +5,7 @@ Syncany [![Build Status](https://travis-ci.org/binwiederhier/syncany.png?branch=
 
 Syncany is an open-source cloud storage and filesharing application. It allows
 users to backup and share certain folders of their workstations using any kind
-of storage, e.g. FTP, Amazon S3 or Google Storage.
+of storage, e.g. FTP, SFTP, WebDAV and Amazon S3.
 
 While the basic idea is similar to Dropbox, Syncany is
 open-source and additionally provides data encryption and more flexibility in
@@ -31,7 +31,7 @@ Download and install Syncany
 You can download the current binary packages and installers from the [releases page](https://github.com/binwiederhier/syncany/releases), or from the Syncany [download site](http://syncany.org/dist/). **Please be aware that this is still ALPHA code! Do not use it for important files.**
 
 **Latest release:**   
-Syncany 0.1.1-alpha, 14 April 2014, [[tar.gz]](https://syncany.org/dist/syncany-0.1.1-alpha.tar.gz) [[zip]](https://syncany.org/dist/syncany-0.1.1-alpha.zip) [[deb]](https://syncany.org/dist/syncany_0.1.1-alpha_all.deb) [[exe]](https://syncany.org/dist/syncany-0.1.1-alpha.exe)
+Syncany 0.1.2-alpha, 27 April 2014, [[tar.gz]](https://syncany.org/dist/releases/syncany-0.1.2-alpha.tar.gz) [[zip]](https://syncany.org/dist/releases/syncany-0.1.2-alpha.zip) [[deb]](https://syncany.org/dist/releases/syncany_0.1.2-alpha_all.deb) [[exe]](https://syncany.org/dist/releases/syncany-0.1.2-alpha.exe)
 
 Quick [install and usage instructions](https://github.com/binwiederhier/syncany/wiki/CLI-quick-howto) can be found in the wiki.   
 If you like it a bit more detailed, [there's lots more you can explore](https://github.com/binwiederhier/syncany/wiki).
@@ -43,18 +43,24 @@ Sample usage: Try Syncany
 Usage is pretty similar to a version control system. If you have used Git or
 SVN, it should feel a lot alike.
 
-**1. Initialize a local directory**
+**1. Choose and install a storage plugin**   
+First choose the storage backend you'd like to use by doing `sy plugin list` and then `sy plugin install`. As of today, we've implemented plugins for [FTP](https://github.com/syncany/syncany-plugin-ftp), [SFTP](https://github.com/syncany/syncany-plugin-sftp), [WebDAV](https://github.com/syncany/syncany-plugin-webdav) and [Amazon S3](https://github.com/syncany/syncany-plugin-s3). For this example, we'll install the FTP plugin:
+```
+$ sy plugin install ftp
+```
+
+**2. Initialize a local directory**
 
 ```
 $ sy init
-Choose a storage plugin. Available plugins are: ftp, local, webdav
+Choose a storage plugin. Available plugins are: ftp, local, webdav, s3, sftp
 Plugin: ftp
 
 Connection details for FTP connection:
 - Hostname: example.com
 - Username: ftpuser
 - Password (not displayed): 
-- Path: /repo-folder
+- Path: repo-folder
 - Port (optional, default is 21): 
 
 Password (min. 10 chars): (user enters repo password)
@@ -68,7 +74,7 @@ This sets up a new repository on the given remote storage and initializes the
 local folder. You can now use `sy connect` to connect to this repository
 from other clients.
 
-**2. Add files and synchronize**
+**3. Add files and synchronize**
 
 To let Syncany do everything automatically, simple use the `sy watch` command. 
 This command will synchronize your local files. 
@@ -84,7 +90,7 @@ $ sy up
 $ sy down
 ```
 
-**3. Connect other clients**   
+**4. Connect other clients**   
 To connect new clients to an existing repository, use the `sy connect` command.
 This will set up your local folder to sync with the chosen remote repository.
 
@@ -115,7 +121,7 @@ Break some hashes for us and [donate some Bitcoins](https://blockchain.info/addr
 Licensing, website and contact
 ------------------------------
 
-Syncany is licensed under the GPLv2 open source license. It is mainly developed by [Philipp C. Heckel](http://blog.philippheckel.com/). We are always looking for people to join or help out. Feel free to contact us:
+Syncany is licensed under the GPLv2 open source license. It is actively developed by [Philipp C. Heckel](http://blog.philippheckel.com/) and [many others](https://github.com/binwiederhier/syncany/graphs/contributors). We are always looking for people to join or help out. Feel free to contact us:
 
 - [Syncany website](https://www.syncany.org/), still with screenshots of the old interface
 - [Syncany wiki page](https://github.com/binwiederhier/syncany/wiki), **most important resource, and always updated**

core/syncany-cli/build.gradle

@@ -25,14 +25,23 @@ mainClassName = "org.syncany.Syncany"
 
 startScripts {
 	defaultJvmOpts = [ "-Xmx1024m", "-Dfile.encoding=utf-8" ]
-	classpath = files('$APP_HOME/lib/*')
 
 	doLast {
 		def winFile = file getWindowsScript()
 		def unixFile = file getUnixScript()
 
-		winFile.text = winFile.text.replaceAll("(set CLASSPATH=.+)", '$1;%AppData%\\\\syncany\\\\plugins\\\\*')
-		unixFile.text = unixFile.text.replaceAll("(CLASSPATH=\\\$APP_HOME.+)", '$1:~/.config/syncany/plugins/*')
+		winFile.text = winFile.text.replaceAll("(set CLASSPATH=.+)", 'set CLASSPATH=%APP_HOME%\\\\lib\\\\*;%AppData%\\\\Syncany\\\\plugins\\\\*')
+		unixFile.text = unixFile.text.replaceAll("(CLASSPATH=\\\$APP_HOME.+)", 'CLASSPATH=\\\$APP_HOME/lib/*:~/.config/syncany/plugins/*')
+
+		// Post Java process commands: Delayed plugin JAR file deletion (Windows only)
+		String winPurgeFileDeletionCommands = "@rem Delete plugin JARs\r\n"
+		winPurgeFileDeletionCommands += "SET PURGEFILE=%AppData%\\\\Syncany\\\\purgefile\r\n";
+		winPurgeFileDeletionCommands += "if exist %PURGEFILE% (\r\n";
+		winPurgeFileDeletionCommands += "  @for /f %%b in (%PURGEFILE%) do del /q \"%%b\" 2>NUL\r\n";
+		winPurgeFileDeletionCommands += "  del /q %PURGEFILE% 2>NUL\r\n";
+		winPurgeFileDeletionCommands += ")\r\n\r\n";
+
+		winFile.text = winFile.text.replaceAll("(:end)", "${winPurgeFileDeletionCommands}:end")
 	}
 }
 

core/syncany-cli/src/main/java/org/syncany/cli/AbstractInitCommand.java

@@ -17,13 +17,13 @@
  */
 package org.syncany.cli;
 
-import java.net.InetAddress;
+import java.math.BigInteger;
 import java.net.UnknownHostException;
+import java.security.SecureRandom;
 import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
-import java.util.Random;
 
 import joptsimple.OptionSet;
 import joptsimple.OptionSpec;
@@ -33,11 +33,11 @@
 import org.syncany.connection.plugins.Connection;
 import org.syncany.connection.plugins.Plugin;
 import org.syncany.connection.plugins.PluginOptionSpec;
-import org.syncany.connection.plugins.StorageTestResult;
 import org.syncany.connection.plugins.PluginOptionSpec.OptionValidationResult;
 import org.syncany.connection.plugins.PluginOptionSpecs;
 import org.syncany.connection.plugins.Plugins;
 import org.syncany.connection.plugins.StorageException;
+import org.syncany.connection.plugins.StorageTestResult;
 import org.syncany.operations.init.GenlinkOperationResult;
 import org.syncany.util.StringUtil;
 import org.syncany.util.StringUtil.StringJoinListener;
@@ -54,7 +54,7 @@ protected ConfigTO createConfigTO(ConnectionTO connectionTO) throws Exception {
 		ConfigTO configTO = new ConfigTO();
 
 		configTO.setDisplayName(getDefaultDisplayName());
-		configTO.setMachineName(getDefaultMachineName());
+		configTO.setMachineName(getRandomMachineName());
 		configTO.setMasterKey(null); 
 		configTO.setConnectionTO(connectionTO); // can be null
 
@@ -289,9 +289,9 @@ public String getString(Plugin plugin) {
 		return plugin;
 	}
 
-	protected String getDefaultMachineName() throws UnknownHostException {
-		return new String(InetAddress.getLocalHost().getHostName() + System.getProperty("user.name") + Math.abs(new Random().nextInt())).replaceAll(
-				"[^a-zA-Z0-9]", "");
+	protected String getRandomMachineName() {
+		String randomStr = new BigInteger(128, new SecureRandom()).toString(32);
+		return (randomStr.length() > 16) ? randomStr.substring(0, 16) : randomStr;
 	}
 
 	protected String getDefaultDisplayName() throws UnknownHostException {

core/syncany-cli/src/main/java/org/syncany/cli/CommandLineClient.java

@@ -26,9 +26,7 @@
 import java.io.InputStream;
 import java.io.OutputStream;
 import java.io.PrintStream;
-import java.text.SimpleDateFormat;
 import java.util.ArrayList;
-import java.util.Date;
 import java.util.List;
 import java.util.logging.ConsoleHandler;
 import java.util.logging.FileHandler;
@@ -38,7 +36,6 @@
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
 
-import joptsimple.OptionException;
 import joptsimple.OptionParser;
 import joptsimple.OptionSet;
 import joptsimple.OptionSpec;
@@ -64,6 +61,10 @@
 public class CommandLineClient extends Client {
 	private static final Logger logger = Logger.getLogger(CommandLineClient.class.getSimpleName());
 
+	private static final String LOG_FILE_PATTERN = "syncany.log";
+	private static final int LOG_FILE_COUNT = 4;
+	private static final int LOG_FILE_LIMIT = 25000000; // 25 MB
+
 	private static final Pattern HELP_TEXT_RESOURCE_PATTERN = Pattern.compile("\\%RESOURCE:([^%]+)\\%");
 	private static final String HELP_TEXT_HELP_SKEL_RESOURCE = "/help/help.skel";
 	private static final String HELP_TEXT_USAGE_SKEL_RESOURCE = "/help/usage.skel";
@@ -110,14 +111,14 @@ public int start() throws Exception {
 
 			// Evaluate options
 			// WARNING: Do not re-order unless you know what you are doing!
-			//initHelpOption(options, optionHelp, options.nonOptionArguments());
 			initConfigOption(options, optionLocalDir);
 			initLogOption(options, optionLog, optionLogLevel, optionLogPrint, optionDebug);
 
 			// Run!
 			return runCommand(options, optionHelp, options.nonOptionArguments());
 		}
-		catch (OptionException e) {
+		catch (Exception e) {
+			logger.log(Level.SEVERE, "Exception while initializing or running command.", e);
 			return showErrorAndExit(e.getMessage());
 		}
 	}	
@@ -165,11 +166,11 @@ private void initLogHandlers(OptionSet options, OptionSpec<String> optionLog, Op
 			}			
 		}
 		else if (config != null && config.getLogDir().exists()) {
-			logFilePattern = config.getLogDir()+File.separator+new SimpleDateFormat("yyMMdd").format(new Date())+".log";
+			logFilePattern = config.getLogDir() + File.separator + LOG_FILE_PATTERN;
 		}
 
 		if (logFilePattern != null) {	
-			Handler fileLogHandler = new FileHandler(logFilePattern, true);			
+			Handler fileLogHandler = new FileHandler(logFilePattern, LOG_FILE_LIMIT, LOG_FILE_COUNT, true);			
 			fileLogHandler.setFormatter(new LogFormatter());
 
 			Logging.addGlobalHandler(fileLogHandler);
@@ -221,12 +222,12 @@ private int runCommand(OptionSet options, OptionSpec<Void> optionHelp, List<?> n
 		Command command = CommandFactory.getInstance(commandName);
 
 		if (command == null) {			
-			showErrorAndExit("Given command is unknown: "+commandName);			
+			return showErrorAndExit("Given command is unknown: "+commandName);			
 		}
 
 		// Potentially show help
 		if (options.has(optionHelp)) {
-			showCommandHelpAndExit(commandName);
+			return showCommandHelpAndExit(commandName);
 		}
 
 		// Init command
@@ -237,12 +238,12 @@ private int runCommand(OptionSet options, OptionSpec<Void> optionHelp, List<?> n
 		// Pre-init operations
 		if (command.getRequiredCommandScope() == INITIALIZED_LOCALDIR) { 
 			if (config == null) {
-				showErrorAndExit("No repository found in path. Use 'init' command to create one.");			
+				return showErrorAndExit("No repository found in path, or configured plugin not installed. Use 'sy init' to create one.");			
 			}			
 		}
 		else if (command.getRequiredCommandScope() == UNINITIALIZED_LOCALDIR) {
 			if (config != null) {
-				showErrorAndExit("Repository found in path. Command can only be used outside a repository.");			
+				return showErrorAndExit("Repository found in path. Command can only be used outside a repository.");			
 			}
 		}
 
@@ -253,10 +254,8 @@ else if (command.getRequiredCommandScope() == UNINITIALIZED_LOCALDIR) {
 		}
 		catch (Exception e) {
 			logger.log(Level.SEVERE, "Command "+ commandName+" FAILED. ", e);
-			showErrorAndExit(e.getMessage());
+			return showErrorAndExit(e.getMessage());
 		}	
-
-		return -1; // Never reached!
 	}
 
 	private void showUsageAndExit() throws IOException {
@@ -267,12 +266,12 @@ private void showHelpAndExit() throws IOException {
 		printHelpTextAndExit(HELP_TEXT_HELP_SKEL_RESOURCE);
 	}
 
-	private void showCommandHelpAndExit(String commandName) throws IOException {
+	private int showCommandHelpAndExit(String commandName) throws IOException {
 		String helpTextResource = HELP_TEXT_CMD_SKEL_RESOURCE.replace(HELP_VAR_CMD, commandName);
-		printHelpTextAndExit(helpTextResource);		
+		return printHelpTextAndExit(helpTextResource);		
 	}
 
-	private void printHelpTextAndExit(String helpTextResource) throws IOException {
+	private int printHelpTextAndExit(String helpTextResource) throws IOException {
 		InputStream helpTextInputStream = CommandLineClient.class.getResourceAsStream(helpTextResource);
 
 		if (helpTextInputStream == null) {
@@ -286,6 +285,8 @@ private void printHelpTextAndExit(String helpTextResource) throws IOException {
 
 		out.close();		
 		System.exit(0);
+
+		return -1; // Never reached
 	}
 
 	private String replaceVariables(String line) throws IOException {
@@ -355,7 +356,7 @@ private int showErrorAndExit(String errorMessage) {
 		out.close();	
 		System.exit(0);
 
-		return 0;
+		return -1; // Never reached
 	}
 
 }