file error when there is no internet

[luislozoya]

When coping from google drive to the computer if the internet if off it will retry 3 times, then it will report a fail, but after that when i checked the folder of the copied files i notice one with 0 size then when running first sync again it skips the file and even worse as it does a sync after that it overwrites the correct file in google drive with the 0 size file

suggestion fix is to:
-remove the failed file

[cjnaz]

Ouch. I agree, it shouldn't do that. Would you please post a verbose log for when the 0 size file is created? This will help a lot in isolating the problem and fix.

[luislozoya]

hi here is the log

2020/08/04 16:00:58 INFO :
Transferred: 70.163M / 417.356 MBytes, 17%, 27.217 kBytes/s, ETA 3h37m42s
Transferred: 0 / 1, 0%
Elapsed time: 43m59.8s
Transferring:

•                             TT T01-08.mkv:  0% /347.193M, 0/s, -
  

2020/08/04 16:01:58 INFO :
Transferred: 70.163M / 417.356 MBytes, 17%, 26.611 kBytes/s, ETA 3h42m39s
Transferred: 0 / 1, 0%
Elapsed time: 44m59.8s
Transferring:

•                             TT T01-08.mkv:  0% /347.193M, 0/s, -
  

2020/08/04 16:02:58 INFO :
Transferred: 70.163M / 417.356 MBytes, 17%, 26.033 kBytes/s, ETA 3h47m36s
Transferred: 0 / 1, 0%
Elapsed time: 45m59.8s
Transferring:

•                             TT T01-08.mkv:  0% /347.193M, 0/s, -
  

2020/08/04 16:03:58 INFO :
Transferred: 70.163M / 417.356 MBytes, 17%, 25.479 kBytes/s, ETA 3h52m33s
Transferred: 0 / 1, 0%
Elapsed time: 46m59.8s
Transferring:

•                             TT T01-08.mkv:  0% /347.193M, 0/s, -
  

2020/08/04 16:04:58 INFO :
Transferred: 70.163M / 417.356 MBytes, 17%, 24.948 kBytes/s, ETA 3h57m30s
Transferred: 0 / 1, 0%
Elapsed time: 47m59.8s
Transferring:

•                             TT T01-08.mkv:  0% /347.193M, 0/s, -
  

2020/08/04 16:05:58 INFO :
Transferred: 70.163M / 417.356 MBytes, 17%, 24.439 kBytes/s, ETA 4h2m27s
Transferred: 0 / 1, 0%
Elapsed time: 48m59.8s
Transferring:

•                             TT T01-08.mkv:  0% /347.193M, 0/s, -
  

2020/08/04 16:06:00 ERROR : TT T01-08.mkv: Failed to copy: multpart copy: failed to open source: open file failed: Get https://www.googleapis.com/drive/v3/files/1BEbDzqatVHKjWEIorIGYzA3GxTUq3bjW?alt=media: dial tcp: lookup www.googleapis.com on 10.255.255.2:53: write udp 192.168.0.106:42540->10.255.255.2:53: write: operation not permitted
2020/08/04 16:06:00 ERROR : Attempt 1/3 failed with 1 errors and: multpart copy: failed to open source: open file failed: Get https://www.googleapis.com/drive/v3/files/1BEbDzqatVHKjWEIorIGYzA3GxTUq3bjW?alt=media: dial tcp: lookup www.googleapis.com on 10.255.255.2:53: write udp 192.168.0.106:42540->10.255.255.2:53: write: operation not permitted
2020/08/04 16:06:58 INFO :
Transferred: 70.163M / 70.163 MBytes, 100%, 24.420 kBytes/s, ETA 0s
Elapsed time: 49m2.1s

2020/08/04 16:07:39 ERROR : Attempt 2/3 failed with 1 errors and: couldn't list directory: Get https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%2715KW9nKs3kbWe1J23RxGsmr5cr0OU3FqZ%27+in+parents%29+and+%28name%3D%27TT+T01-08.mkv%27%29+and+mimeType%21%3D%27application%2Fvnd.google-apps.folder%27&supportsAllDrives=true: dial tcp: lookup www.googleapis.com on 10.255.255.2:53: write udp 192.168.0.106:60725->10.255.255.2:53: write: operation not permitted
2020/08/04 16:07:58 INFO :
Transferred: 70.163M / 70.163 MBytes, 100%, 24.420 kBytes/s, ETA 0s
Elapsed time: 49m2.1s

2020/08/04 16:08:58 INFO :
Transferred: 70.163M / 70.163 MBytes, 100%, 24.420 kBytes/s, ETA 0s
Elapsed time: 49m2.1s

2020/08/04 16:09:58 INFO :
Transferred: 70.163M / 70.163 MBytes, 100%, 24.420 kBytes/s, ETA 0s
Elapsed time: 49m2.1s

2020/08/04 16:10:25 ERROR : Attempt 3/3 failed with 1 errors and: couldn't list directory: Get https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%2715KW9nKs3kbWe1J23RxGsmr5cr0OU3FqZ%27+in+parents%29+and+%28name%3D%27TT+T01-08.mkv%27%29+and+mimeType%21%3D%27application%2Fvnd.google-apps.folder%27&supportsAllDrives=true: dial tcp: lookup www.googleapis.com on 10.255.255.2:53: write udp 192.168.0.106:48458->10.255.255.2:53: write: operation not permitted
2020/08/04 16:10:25 Failed to copyto: couldn't list directory: Get https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%29%2CnextPageToken%2CincompleteSearch&includeItemsFromAllDrives=true&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+%28%2715KW9nKs3kbWe1J23RxGsmr5cr0OU3FqZ%27+in+parents%29+and+%28name%3D%27TT+T01-08.mkv%27%29+and+mimeType%21%3D%27application%2Fvnd.google-apps.folder%27&supportsAllDrives=true: dial tcp: lookup www.googleapis.com on 10.255.255.2:53: write udp 192.168.0.106:48458->10.255.255.2:53: write: operation not permitted
2020/08/04 16:14:33 Source doesn't exist or is a directory and destination is a file
2020/08/04 16:18:45 Source doesn't exist or is a directory and destination is a file
2020/08/04 16:23:13 Source doesn't exist or is a directory and destination is a file

[cjnaz]

I've posted a question in the rclone forum RE temporary file support during copies/syncs. I understand that some remotes don't support server-side renames, so the temp file method can't work in all cases. I'm considering a minor revamp of rclonesync to use filtering for bulk copy operations, which make it messier for rclonesync to initially push a file with a temp name and then rename it after successful transfer. (Thinking out loud)

[cjnaz]

As noted in rclone issue #4513, this is a known issue with rclone. rclonesync is not the best place to fix this up. I'll leave this issue open for now. Perhaps it'll get fixed on rclone sometime soon.

[cjnaz]

Check out V3.0, just posted. This version aborts nicely when a run is done while there is no internet. Running again should work nicely.
The core problem of this thread is a corrupted file on local/ftp/sftp remotes if the internet dies or the rclonesync or rclone process dies in the middle of a file copy. This is a problem for rclone to solve, as there is no way for rclonesync to gracefully and reliably deal with it. I'll leave this issue open.

[luislozoya]

Check out V3.0, just posted. This version aborts nicely when a run is done while there is no internet. Running again should work nicely.
The core problem of this thread is a corrupted file on local/ftp/sftp remotes if the internet dies or the rclonesync or rclone process dies in the middle of a file copy. This is a problem for rclone to solve, as there is no way for rclonesync to gracefully and reliably deal with it. I'll leave this issue open.

cool thanks ill check it out.