Duplicating files on Linux

[qyx268]

Hi developers,

Thanks for the great app first! I can't easily find anything related in the existing Issues or Discussions. So here am I. If I've missed any, could you please direct me to the post? Thanks!

I'm trying to understand why on my Ubuntu desktop, files keep getting duplicated even when no one is using it. I have been away from the desktop these days and I was using the computer which is a Mac in the office to prepare some powerpoint slides. The file name is slides.pptx. As you can see below, I got multiple duplicates with file names being slides-homePC-N.pptx where homePC is the hostname of my Ubuntu desktop. I can understand this might happen when the same file is being edited on two OneDrive client ends. But I was not using the Ubuntu desktop while all duplicates were created on this end. Do you know what might have caused this and whether this is a solution for it. My configuration mostly follows the default except for log_dir and my onedrive version is v2.4.25-1+np2+4.1. Thanks for your time.

-rw------- 1 yqin yqin 69469486 Oct 23 02:59 slides-homePC-2.pptx
-rw------- 1 yqin yqin 69462147 Oct 23 03:37 slides-homePC-3.pptx
-rw------- 1 yqin yqin 71465670 Oct 24 00:10 slides-homePC-4.pptx
-rw------- 1 yqin yqin 71523573 Oct 24 00:32 slides-homePC-5.pptx
-rw------- 1 yqin yqin 69415693 Oct 23 02:42 slides-homePC.pptx
-rw------- 1 yqin yqin 71529694 Oct 24 00:33 slides.pptx

Cheers,
Yuxiang

[abraunegg]

@qyx268

onedrive version is v2.4.25-1+np2+4.1.

This is your issue.

Significant changes in how file duplicates were detected and created was implemented in v2.5.x.

You are also using an unsupported client version. Please read: https://github.com/abraunegg/onedrive?tab=readme-ov-file#supported-application-version

[qyx268]

@abraunegg Thanks, that's already something for me to start with. I've updated to the latest version. i will report if similar issues happen again.

[qyx268]

Hey @abraunegg, I have updated onedrive to v2.5.2-1+np1+1.1, but keep seeing similar (if not the same) issue. This time, the duplication happened to the notebooks of OneNote, which are synced through OneDrive.

The screenshot shows what I see in OneNote at the moment, where you can see my homepc (the ubuntu desktop using this repo) has duplicated so many notebooks... Below, I show a couple of places in the log where duplications of Conferences.one happens. My configuration has not changed since the update and mostly follows the default except for log_dir. Any idea how to fix it? Perhaps I'm not configuring it properly?

[2024-Nov-06 15:42:23.9145280] Starting a sync with Microsoft OneDrive
[2024-Nov-06 15:42:25.7443854] Syncing changes from Microsoft OneDrive ...
[2024-Nov-06 15:42:28.0884457] Changed local items to upload to Microsoft OneDrive: 1
[2024-Nov-06 15:42:28.7082626] Skipping uploading this item as a locally modified file, will upload as a new file (online file already exists and is newer): Documents/Yuxiang's Notebook/Conferences.one
[2024-Nov-06 15:42:33.5780114] Uploading new file: Documents/Yuxiang's Notebook/Conferences-homePC.one ... done
[2024-Nov-06 15:42:33.5813665] Online file validation unable to be performed: input JSON whilst valid did not contain data which could be validated
[2024-Nov-06 15:42:33.5813914] WARNING: Skipping upload integrity check for: Documents/Yuxiang's Notebook/Conferences-homePC.one
[2024-Nov-06 15:42:35.5829625] Moving Documents/Yuxiang's Notebook/Conferences.one to Documents/Yuxiang's Notebook/Conferences-homePC.one
[2024-Nov-06 15:42:35.5840514] Moved local item was not in-sync with local database - uploading as new item
[2024-Nov-06 15:42:36.4047235] Sync with Microsoft OneDrive is complete
[2024-Nov-06 15:47:36.4185239] Attempting to contact Microsoft OneDrive Login Service
[2024-Nov-06 15:47:37.7467304] Successfully reached Microsoft OneDrive Login Service
BLABLA
[2024-Nov-06 16:03:06.7673152] Changed local items to upload to Microsoft OneDrive: 1
[2024-Nov-06 16:03:07.2550386] Skipping uploading this item as a locally modified file, will upload as a new file (online file already exists and is newer): Documents/Yuxiang's Notebook/Conferences-homePC.one
[2024-Nov-06 16:03:11.5224638] Uploading new file: Documents/Yuxiang's Notebook/Conferences-homePC-homePC.one ... done
[2024-Nov-06 16:03:11.5238795] Online file validation unable to be performed: input JSON whilst valid did not contain data which could be validated
[2024-Nov-06 16:03:11.5238865] WARNING: Skipping upload integrity check for: Documents/Yuxiang's Notebook/Conferences-homePC-homePC.one
[2024-Nov-06 16:03:13.4908886] Moving Documents/Yuxiang's Notebook/Conferences-homePC.one to Documents/Yuxiang's Notebook/Conferences-homePC-homePC.one
[2024-Nov-06 16:03:13.4915155] Moved local item was not in-sync with local database - uploading as new item
[2024-Nov-06 16:03:14.3508592]
BLABLA
[2024-Nov-06 16:18:28.9998847] Number of items to download from Microsoft OneDrive: 1
[2024-Nov-06 16:18:31.7558583] ERROR: File download hash mismatch. Increase logging verbosity to determine why.
[2024-Nov-06 16:18:31.7559099] INFO: Potentially add --disable-download-validation to work around this issue but downloaded data integrity cannot be guaranteed.
[2024-Nov-06 16:18:31.7559316] Removing local file Documents/Yuxiang's Notebook/Conferences-homePC.one due to failed integrity checks
[2024-Nov-06 16:18:31.7564044] Downloading file: Documents/Yuxiang's Notebook/Conferences-homePC.one ... failed!
[2024-Nov-06 16:18:33.9418791]
[2024-Nov-06 16:18:33.9419242] Failed items to download to/from Microsoft OneDrive: 1
[2024-Nov-06 16:18:33.9419465] Failed to download: Documents/Yuxiang's Notebook/Conferences-homePC.one
[2024-Nov-06 16:18:33.9651098]
[2024-Nov-06 16:18:33.9651365] Sync with Microsoft OneDrive has completed, however there are items that failed to sync.
[2024-Nov-06 16:18:33.9651444] To fix any download failures you may need to perform a --resync to ensure this system is correctly synced with your Microsoft OneDrive Account
[2024-Nov-06 16:18:33.9651512]
BLABLA
[2024-Nov-06 16:54:16.0437397] Removing local file Documents/Yuxiang's Notebook/Conferences-homePC.one due to failed integrity checks
[2024-Nov-06 16:54:16.0439173] Downloading file: Documents/Yuxiang's Notebook/Conferences-homePC.one ... failed!
[2024-Nov-06 16:54:16.8008251] ERROR: File download hash mismatch. Increase logging verbosity to determine why.
[2024-Nov-06 16:54:16.8008497] INFO: Potentially add --disable-download-validation to work around this issue but downloaded data integrity cannot be guaranteed.
[2024-Nov-06 16:54:16.8008551] Removing local file Documents/Yuxiang's Notebook/Conferences.one due to failed integrity checks
[2024-Nov-06 16:54:16.8010962] Downloading file: Documents/Yuxiang's Notebook/Conferences.one ... failed!
[2024-Nov-06 16:54:17.4477305] ERROR: File download hash mismatch. Increase logging verbosity to determine why.
[2024-Nov-06 16:54:17.4477386] INFO: Potentially add --disable-download-validation to work around this issue but downloaded data integrity cannot be guaranteed.
BLABLA
[2024-Nov-06 17:04:33.7961271] Changed local items to upload to Microsoft OneDrive: 1
[2024-Nov-06 17:04:34.2743705] Skipping uploading this item as a locally modified file, will upload as a new file (online file already exists and is newer): Documents/Yuxiang's Notebook/Conferences-homePC-homePC.one
[2024-Nov-06 17:04:38.6284071] Uploading new file: Documents/Yuxiang's Notebook/Conferences-homePC-homePC-homePC.one ... done
[2024-Nov-06 17:04:38.6318277] Online file validation unable to be performed: input JSON whilst valid did not contain data which could be validated
[2024-Nov-06 17:04:38.6318711] WARNING: Skipping upload integrity check for: Documents/Yuxiang's Notebook/Conferences-homePC-homePC-homePC.one
[2024-Nov-06 17:04:40.8189617] Moving Documents/Yuxiang's Notebook/Conferences-homePC-homePC.one to Documents/Yuxiang's Notebook/Conferences-homePC-homePC-homePC.one
[2024-Nov-06 17:04:40.8201617] Moved local item was not in-sync with local database - uploading as new item
[2024-Nov-06 17:04:41.4040801]

There are more in the log which are basically recursion of creating more Conferences-homePC-homePC-...-homePC.one

[abraunegg]

@qyx268
Microsoft OneNote files are not supported for a variety of reasons.

This bug was fixed post v2.5.2

[qyx268]

ok...

[abraunegg]

References for reading:

• Bug: A database statement execution error occurred: NOT NULL constraint failed: item.type #2938
• A database statement execution error occurred: NOT NULL constraint failed: item.type #2936
• Make onedrive stop syncing the Notebooks (of OneNote) #2940
• ODP|ODB: OneDrive API query for driveItem details fails to respond with data despite being requested in select statement OneDrive/onedrive-api-docs#1879

[qyx268]

thank you! I've found some of these.