Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rclonesync does not sync empty directories #63

Open
tliero opened this issue Sep 21, 2020 · 7 comments
Open

rclonesync does not sync empty directories #63

tliero opened this issue Sep 21, 2020 · 7 comments
Labels
limitation Known problem with no planned path for solution

Comments

@tliero
Copy link

tliero commented Sep 21, 2020

When syncing a local directory with a (Google Drive) remote, empty directories that exist either on Path1 or Path2 are not being created on the other. While for "end user files" this behavior might be acceptable, especially folder structures that belong to programs might expect empty directories to be in place.

Edit: Adding --rclone-args --create-empty-src-dirs as last parameter unfortunately does not solve the issue, as --create-empty-src-dirs is not a global rclone parameter. This results in rclone lsl returning an error.

@cjnaz
Copy link
Owner

cjnaz commented Sep 22, 2020

Edit: Adding --rclone-args --create-empty-src-dirs as last parameter unfortunately does not solve the issue, as --create-empty-src-dirs is not a global rclone parameter. This results in rclone lsl returning an error.

That's unfortunate. I wonder if any --rclone-args can be discarded for the rclone LSLs.

@tliero
Copy link
Author

tliero commented Sep 23, 2020

I guess global flags would still be expected to be passed on to all subcommands. Subcommand-specific flags like --create-empty-src-dirs seem to be the rare exception for rclone. So either rclonesync would have to recognize specific flags or there needs to be a way to pass on flags to sub-commands only, e.g. --rclone-copy-args. But that would create a problem with the catch-all structure of the command line and make it impossible to pass global and specific args at the same time.
Maybe it would actually be the easiest to just recognize the few subcommand-specific args.

@cjnaz
Copy link
Owner

cjnaz commented Sep 24, 2020

The problem is rooted in rclonesync specifically and only working on files. File differences are found and propagated to the other side. A parallel algorithm would have to be implemented to check for directory differences (specifically empty directory adds and deletes) and propagate those to the other side.

A not-very-robust workaround is to do a an rclone sync from your master path to the other with --create-empty-src-dirs. A --filter "- *" with --create-empty-src-dirs may cause rclone to touch only empty directories, and thus not force the need for an rclonesync --first-sync to follow. Any deleted empty directories would have to be manually deleted on the other side as directories themselves are not tracked.

I'll leave this open for the record. Its a limitation.

@cjnaz cjnaz added the limitation Known problem with no planned path for solution label Sep 24, 2020
@cjnaz
Copy link
Owner

cjnaz commented Sep 25, 2020

This sequence may get you close to what you need

1) rclonesync <path1> <path2>
2) rclone copy <path1> <path2> --filter "+ */" --filter "- **" --create-empty-src-dirs
3) rclone copy <path2> <path1> --filter "+ */" --filter "- **" --create-empty-src-dirs

The filtering in line 2 first includes all directories in path1, and then excludes everything else in path1. The empty directories in path1 are created on path2.
The filtering on line 3 does the same for path2 to path1.

The limitation is that empty directories only accumulate, and must be manually deleted on both sides. You could use rlconesync's --remove-empty-directories switch to clean them out of both paths, or rclone's rmdirs command.

@bloatmode
Copy link

bloatmode commented Mar 14, 2022

Is there any way to automatically delete empty directories on the remote that aren't present on local storage? I ended up with a ton of them and I think I'm getting throttled when doing rclone lsl on the remote because it takes more than an hour.

Update: I managed to delete them using the web interface of the cloud service, sorting by content size. It would still be nice if the program could delete empty directories on one path when they're deleted on the other.

@cjnaz
Copy link
Owner

cjnaz commented Mar 15, 2022

Did you try the --remove-empty-directories switch?

@bloatmode
Copy link

Thanks! Sorry for bumping this issue, I found it before reading the documentation when searching for my problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
limitation Known problem with no planned path for solution
Projects
None yet
Development

No branches or pull requests

3 participants