Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync progress calculation is incorrect (or could be improved) #610

Closed
achou11 opened this issue Aug 15, 2024 · 1 comment · Fixed by #619
Closed

Sync progress calculation is incorrect (or could be improved) #610

achou11 opened this issue Aug 15, 2024 · 1 comment · Fixed by #619
Assignees
Labels
bug Something isn't working

Comments

@achou11
Copy link
Member

achou11 commented Aug 15, 2024

Related to #550

From Gregor on Slack:

I believe I know what is happening. There are two stages in sync:
"initial sync", which include membership records, device info, project settings, and presets, fields and icons.
"data sync", which includes observations, tracks and photos.

  1. stage 1 ("initial sync") happens without user intervention in the background, regardless of what screen the user is on.
  2. stage 2 ("data sync") happens when 2 or more users have pressed the "sync" button (enabled data sync).

When designing this, I don't think we had fully considered the amount of data in the "initial sync" stage when two devices are syncing for the first time, because > there are a lot of presets, fields and icons to sync. A couple of things are contributing to the observed behaviour:

  1. The progress calculation in the frontend code shows progress across > both initial sync and data sync. The progress of initial sync is hidden from a device that has not yet pressed "sync" (e.g. enabled data sync), but the device that > has enabled data sync (by pressing "sync") will see this progress. If initial sync is still happening, then the device that first presses sync will see this > progress, giving the impression that "sync" is happening, even though it's the initial sync only.

  2. The way progress is calculated, specifically [maxDataSyncCount here](https://github.com/digidem/comapeo-mobile/blob/main/src/frontend/hooks/useSyncState.> ts#L131-L138) is not calculated when the user turns on data sync, rather it is continuously updated to the maximum observed value. This means that the progress is > counting background "initial sync" that has happened before the user has pressed "sync". This means that even if "initial sync" is complete when the user presses > "sync", it shows that already complete progress as part of the sync progress. E.g. if the "initial sync" included 90 records, and "data sync" has 10 records, then if > initial sync is complete, and the user presses "sync", they will see sync progress as 90% done.

Suggested solutions:

For (2), progress should start at 0 when the user first presses "sync". This requires an update to [this code](https://github.com/digidem/comapeo-mobile/blob/main/src/> frontend/hooks/useSyncState.ts#L131-L138) to set maxDataSyncCount at the moment the user enables data sync.

For (1) I am not sure how we expose this to the user - this is a UX issue. Do we expose this "initial sync" as a separate progress bar, that all devices see? Do we > hide it from the users completely? I think this requires further discussion. It would be good to start gathering metrics about how long this "initial sync" is taking, > and how frequent it is that a user presses "sync" before initial sync is complete, because it may be so rare that this is a non-issue. Certainly after presets, fields > and icons have synced, very little additional data is added to the database that is included in the "initial sync" stage - only 2 records (membership and deviceInfo) > for each device that has been added or removed from the project.

@achou11 achou11 added the bug Something isn't working label Aug 15, 2024
@achou11 achou11 self-assigned this Aug 15, 2024
@achou11 achou11 changed the title Sync progress calculation is incorrect (or could be improved Sync progress calculation is incorrect (or could be improved) Aug 15, 2024
@achou11
Copy link
Member Author

achou11 commented Aug 15, 2024

The progress calculation in the frontend code shows progress across > both initial sync and data sync.

Don't think this is the case? Based on the implementation, it's only doing calculations based on the data sync (no initial).

const newSyncCount = state.data.want + state.data.wanted;
this.#maxDataSyncCount =
this.#maxDataSyncCount === null
? newSyncCount
: Math.max(this.#maxDataSyncCount, newSyncCount);

const currentCount = this.#state.data.want + this.#state.data.wanted;

Although the progress probably should include the initial too, I think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant