A question re using volsync with syncthing to provide a remote copy

[carukc]

Hello,

First let me thank you for the project. I think that volsync is an incredibly useful tool. I was about to try to dodo somehting manually with syncthing when I discovered volsync.

I do however have a couple questions about how to best use it. I need to do a rapid sync to a remote location and from what I have read, my experiments so far and after reading the docs for the docuwiki example, it looks like I have to setup a local sync with my application volume and then add the remote as a peer. Is this the only way to make this work? We have >100GB of files that need to be synced to a remote location and it seems to me that this approach would require that I maintain a local source volume what syncs both with the remote and the pod volume (and thus would require that I provision an extra 100GB local volume).

Is it possible to sidecar volsync so that the pod/app volume is the same as the volsync volume? This way I only need a single 100GB volume locally, which I can sync to the remote location. In other words, can the volsync/syncthing and app volumes be the same?

Thanks
Chris

[JohnStrunk]

When using Syncthing, we recommend that you do, in fact, sync the application's actual PVC, not a snapshot/clone. That way, you get the latest data sent across as soon as it's written (well, as soon as syncthing notices).

When you configure the ReplicationSource(s), make sure you use copyMethod: Direct. This will avoid taking snapshots/clones and have VolSync just use the app's PVC as the source.

Thanks for asking about this... in a quick check of the docs, it looks like we never mention setting this. 🤦‍♂️

[carukc]

Thanks John, Yes.. much better to sync the actual files. I have tried this configuration setting but it is generating an error:

I followed the convention that was used in rsync example here ,https://github.com/backube/volsync/blob/dad17ec5fbe6413038ba07c21faf68c4cc6603be/examples/rsync/volsync_v1alpha1_replicationsource.yaml , as follows:

apiVersion: volsync.backube/v1alpha1
kind: ReplicationSource
metadata:
  name: sync-data-development-builds
spec:
  sourcePVC: data-development-builds
  syncthing:
    serviceType: ClusterIP
    copyMethod: Direct
    configStorageClassName: longhorn
    # List of peers that this ReplicationSource should connect to.
    peers:
      # The Syncthing ID of the peer.
    - ID: ###
      # The address of the peer - this will be used as a data connection.
      address: tcp://10.42.249.16:22000
      # Whether or not the peer should introduce this ReplicationSource to other peers.
      introducer: false

but I am getting the error:

Error from server (BadRequest): error when creating "vol-syncthing.yml": ReplicationSource in version "v1alpha1" cannot be handled as a ReplicationSource: strict decoding error: unknown field "spec.syncthing.copyMethod

Thanks
Chris

[JohnStrunk]

Move copyMethod to be a sub-field of spec instead of syncthing:

apiVersion: volsync.backube/v1alpha1
kind: ReplicationSource
metadata:
  name: sync-data-development-builds
spec:
  sourcePVC: data-development-builds
  copyMethod: Direct
  syncthing:
    serviceType: ClusterIP
    configStorageClassName: longhorn
    # List of peers that this ReplicationSource should connect to.
    peers:
      # The Syncthing ID of the peer.
    - ID: ###
      # The address of the peer - this will be used as a data connection.
      address: tcp://10.42.249.16:22000
      # Whether or not the peer should introduce this ReplicationSource to other peers.
      introducer: false

[carukc]

I did try that before putting the parameter under synching, but assumed that this was wrong because it did not work. This is why I copied how rsync was done. Just in case the issue was the version of volsync I updated the operator before trying again, using helm upgrade, but still no joy:

Error from server (BadRequest): error when creating "vol-syncthing.yml": ReplicationSource in version "v1alpha1" cannot be handled as a ReplicationSource: strict decoding error: unknown field "spec.copyMethod"

[tesshuflower]

Something was bothering me about this one - was almost going to create an issue to update the docs and mention direct mode...

But - I think for Syncthing it's going to essentially use Direct CopyMode by default - taking a bit of a look at the code it doesn't use the volumehandler at all for the sourcePVC (aka dataPVC in the snippet below), and will just check that the sourcePVC exists (and then use it directly) - so I believe using CopyMode: Direct is not necessary and in fact ignored by the Syncthing mover.

volsync/controllers/mover/syncthing/mover.go

Lines 287 to 300 in dad17ec

	func (m *Mover) ensureDataPVC(ctx context.Context) (*corev1.PersistentVolumeClaim, error) {
	// check if the data PVC exists, error if it doesn't
	dataPVC := &corev1.PersistentVolumeClaim{
	ObjectMeta: metav1.ObjectMeta{
	Name: *m.dataPVCName,
	Namespace: m.owner.GetNamespace(),
	},
	}
	if err := m.client.Get(ctx, client.ObjectKeyFromObject(dataPVC), dataPVC); err != nil {
	return nil, err
	}
	return dataPVC, nil
	}

[carukc]

Thanks @tesshuflower , with 'copyMethod: Direct' removed it seems to start but I'm now seeing:

Message:  Get "https://volsync-sync-data-development-builds-api.development-builds:8384/rest/config": dial tcp 10.43.215.242:8384: connect: connection refused

So it never entered the READY state:

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS                  IMAGES                                                                                                  SELECTOR
volsync-sync-data-development-builds     0/1     1            0           34m     syncthing                   quay.io/backube/volsync:0.7.1                                                                           app.kubernetes.io/component=syncthing-mover,app.kubernetes.io/name=sync-data-development-builds,app.kubernetes.io/part-of=volsync

10.43.215.242 is the vol sync api service:

service/volsync-sync-data-development-builds-api    ClusterIP   10.43.215.242   <none>        8384/TCP

as a result I do not seem to be getting a DeviceID, so I can not connect the remote (full) SyncThing instance. Status on the ReplicationSource is:

status:
  conditions:
  - lastTransitionTime: "2023-10-27T01:04:55Z"
    message: 'Get "https://volsync-sync-data-development-builds-api.development-builds:8384/rest/config":
      dial tcp 10.43.215.242:8384: connect: connection refused'
    reason: Error
    status: "False"
    type: Synchronizing
  lastSyncStartTime: "2023-10-27T01:04:51Z"
  syncthing: {}

[tesshuflower]

@carukc Could you get the logs from the volsync syncthing mover pod? It should be the pod with a name starting with something like: volsync-sync-data-development-builds.

[carukc]

Hi @tesshuflower , Apologies for the delayed reply. I managed to get it all working over the weekend. I discovered that the issue was with the persistent volume that was being used. I had tried to change the storage class without first removing the old PV. so startup was failing.

Once I found the correct logs, everything made sense.

So we now have the synching mover running well with a full synching instance.

Next I hope to get rsync and the other movers.

Thanks also to @JohnStrunk for the kind suggestions. You guys are stars!

Warmest Regards
Chris