parsyncfp2 is not starting

[cmonty14]

Hello,
I'm trying to transfer files from Ceph device (source) to NFS share (target).
Source and target are mounted on the same host.

This is the directory tree of the source:

# tree -L 1 /mnt/rbd/
/mnt/rbd/
├── PBA
├── PBH
└── VCT

It looks like parsyncfp2 is not starting using this command:

# parsyncfp2 -v 1 --NP=8 --chunksize=8M --startdir='/mnt/rbd' VCT /mnt/backupVCT/
parsyncfp2 version 2.56 (Sargasso Apocalypse)
Mar 20, 2023
by Harry Mangalam <[email protected]>

parsyncfp2 is a Perl script that wraps the near-miraculous Tridgell/
Mackerras 'rsync' to provide load balancing and parallel operation across
network connections to increase the amount of bandwidth it can use.
The 'fp' variant uses 'fpart' to bypass the need for a full recursive
descent of the dir trees before the actual transfer starts.
Versions >2 allow multihost sends to increase bandwidth saturation
WARNING: Do NOT use any '--delete*' options in the rsync options ('--ro').
parsyncfp2 is distributed under the Gnu Public License (GPL) v3.

Can you please advise how to fix this?

[hjmangalam]

Looking into this. I went back and forth in deciding whether to support pfp2 on mounted filesystems bc rsync itself performs so badly in these situations. I'll try to resolve the issues to allow it with a WARN and then let you wait for it to complete. If it's too gnarly, I may whiff on it tho since it's not what rsync/pfp2 does well. Also, is your Ceph set up as an object FS or a posix FS? If the former, it may just not work at all since the underlying object store probably doesn't support the kind of block reads that rsync requires. (?) Does naked rsync work to actually sync 2 files in your situation? harry

…

On Tue, Mar 21, 2023 at 6:14 AM Thomas ***@***.***> wrote: Hello, I'm trying to transfer files from Ceph device (source) to NFS share (target). Source and target are mounted on the same host. This is the directory tree of the source: # tree -L 1 /mnt/rbd/ /mnt/rbd/ ├── PBA ├── PBH └── VCT It looks like parsyncfp2 is not starting using this command: # parsyncfp2 -v 1 --NP=8 --chunksize=8M --startdir='/mnt/rbd' VCT /mnt/backupVCT/ parsyncfp2 version 2.56 (Sargasso Apocalypse) Mar 20, 2023 by Harry Mangalam ***@***.***> parsyncfp2 is a Perl script that wraps the near-miraculous Tridgell/ Mackerras 'rsync' to provide load balancing and parallel operation across network connections to increase the amount of bandwidth it can use. The 'fp' variant uses 'fpart' to bypass the need for a full recursive descent of the dir trees before the actual transfer starts. Versions >2 allow multihost sends to increase bandwidth saturation WARNING: Do NOT use any '--delete*' options in the rsync options ('--ro'). parsyncfp2 is distributed under the Gnu Public License (GPL) v3. Can you please advise how to fix this? — Reply to this email directly, view it on GitHub <#9>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASF3Y6IHNZKYPOAQERPFTDW5GSSTANCNFSM6AAAAAAWCNSRY4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- Harry Mangalam

[cmonty14]

Actually rsync works w/o problems.

[hjmangalam]

OK , then you're the test case.. :) Configuring a system to test on. Harry

…

On Tue, Mar 21, 2023 at 8:56 AM Thomas ***@***.***> wrote: Looking into this. I went back and forth in deciding whether to support pfp2 on mounted filesystems bc rsync itself performs so badly in these situations. I'll try to resolve the issues to allow it with a WARN and then let you wait for it to complete. If it's too gnarly, I may whiff on it tho since it's not what rsync/pfp2 does well. Also, is your Ceph set up as an object FS or a posix FS? If the former, it may just not work at all since the underlying object store probably doesn't support the kind of block reads that rsync requires. (?) Does naked rsync work to actually sync 2 files in your situation? harry … <#m_-5670668789800388889_> On Tue, Mar 21, 2023 at 6:14 AM Thomas *@*.*> wrote: Hello, I'm trying to transfer files from Ceph device (source) to NFS share (target). Source and target are mounted on the same host. This is the directory tree of the source: # tree -L 1 /mnt/rbd/ /mnt/rbd/ ├── PBA ├── PBH └── VCT It looks like parsyncfp2 is not starting using this command: # parsyncfp2 -v 1 --NP=8 --chunksize=8M --startdir='/mnt/rbd' VCT /mnt/backupVCT/ parsyncfp2 version 2.56 (Sargasso Apocalypse) Mar 20, 2023 by Harry Mangalam @.*> parsyncfp2 is a Perl script that wraps the near-miraculous Tridgell/ Mackerras 'rsync' to provide load balancing and parallel operation across network connections to increase the amount of bandwidth it can use. The 'fp' variant uses 'fpart' to bypass the need for a full recursive descent of the dir trees before the actual transfer starts. Versions >2 allow multihost sends to increase bandwidth saturation WARNING: Do NOT use any '--delete*' options in the rsync options ('--ro'). parsyncfp2 is distributed under the Gnu Public License (GPL) v3. Can you please advise how to fix this? — Reply to this email directly, view it on GitHub <#9 <#9>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASF3Y6IHNZKYPOAQERPFTDW5GSSTANCNFSM6AAAAAAWCNSRY4 . You are receiving this because you are subscribed to this thread.Message ID: *@*.***> -- Harry Mangalam Actually rsync works w/o problems. — Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASF3Y35KZZ2OKICHWCYB33W5HFUDANCNFSM6AAAAAAWCNSRY4> . You are receiving this because you commented.Message ID: ***@***.***>

-- Harry Mangalam

[hjmangalam]

Just pushed a version that works for me.. hjm

…

On Tue, Mar 21, 2023 at 8:56 AM Thomas ***@***.***> wrote: Looking into this. I went back and forth in deciding whether to support pfp2 on mounted filesystems bc rsync itself performs so badly in these situations. I'll try to resolve the issues to allow it with a WARN and then let you wait for it to complete. If it's too gnarly, I may whiff on it tho since it's not what rsync/pfp2 does well. Also, is your Ceph set up as an object FS or a posix FS? If the former, it may just not work at all since the underlying object store probably doesn't support the kind of block reads that rsync requires. (?) Does naked rsync work to actually sync 2 files in your situation? harry … <#m_-5670668789800388889_> On Tue, Mar 21, 2023 at 6:14 AM Thomas *@*.*> wrote: Hello, I'm trying to transfer files from Ceph device (source) to NFS share (target). Source and target are mounted on the same host. This is the directory tree of the source: # tree -L 1 /mnt/rbd/ /mnt/rbd/ ├── PBA ├── PBH └── VCT It looks like parsyncfp2 is not starting using this command: # parsyncfp2 -v 1 --NP=8 --chunksize=8M --startdir='/mnt/rbd' VCT /mnt/backupVCT/ parsyncfp2 version 2.56 (Sargasso Apocalypse) Mar 20, 2023 by Harry Mangalam @.*> parsyncfp2 is a Perl script that wraps the near-miraculous Tridgell/ Mackerras 'rsync' to provide load balancing and parallel operation across network connections to increase the amount of bandwidth it can use. The 'fp' variant uses 'fpart' to bypass the need for a full recursive descent of the dir trees before the actual transfer starts. Versions >2 allow multihost sends to increase bandwidth saturation WARNING: Do NOT use any '--delete*' options in the rsync options ('--ro'). parsyncfp2 is distributed under the Gnu Public License (GPL) v3. Can you please advise how to fix this? — Reply to this email directly, view it on GitHub <#9 <#9>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASF3Y6IHNZKYPOAQERPFTDW5GSSTANCNFSM6AAAAAAWCNSRY4 . You are receiving this because you are subscribed to this thread.Message ID: *@*.***> -- Harry Mangalam Actually rsync works w/o problems. — Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASF3Y35KZZ2OKICHWCYB33W5HFUDANCNFSM6AAAAAAWCNSRY4> . You are receiving this because you commented.Message ID: ***@***.***>

-- Harry Mangalam

[hjmangalam]

Will probably not work as well with you if you don't have parallel FSs on both sides, but with using pfp2 to rsync across 2 GPFS mounts, I see ~6x increase in speed (1st time transfer) with NP=12 (decreasing effectiveness above 12). that's more than I expected. Thanks for asking me to look at it again. Harry On Tue, Mar 21, 2023 at 11:48 AM Harry Mangalam ***@***.***> wrote:

…

Just pushed a version that works for me.. hjm On Tue, Mar 21, 2023 at 8:56 AM Thomas ***@***.***> wrote: > Looking into this. I went back and forth in deciding whether to support > pfp2 on mounted filesystems bc rsync itself performs so badly in these > situations. I'll try to resolve the issues to allow it with a WARN and then > let you wait for it to complete. If it's too gnarly, I may whiff on it tho > since it's not what rsync/pfp2 does well. Also, is your Ceph set up as an > object FS or a posix FS? If the former, it may just not work at all since > the underlying object store probably doesn't support the kind of block > reads that rsync requires. (?) Does naked rsync work to actually sync 2 > files in your situation? harry > … <#m_-6823775414837348546_m_-5670668789800388889_> > On Tue, Mar 21, 2023 at 6:14 AM Thomas *@*.*> wrote: Hello, I'm trying > to transfer files from Ceph device (source) to NFS share (target). Source > and target are mounted on the same host. This is the directory tree of the > source: # tree -L 1 /mnt/rbd/ /mnt/rbd/ ├── PBA ├── PBH └── VCT It looks > like parsyncfp2 is not starting using this command: # parsyncfp2 -v 1 > --NP=8 --chunksize=8M --startdir='/mnt/rbd' VCT /mnt/backupVCT/ parsyncfp2 > version 2.56 (Sargasso Apocalypse) Mar 20, 2023 by Harry Mangalam @.*> > parsyncfp2 is a Perl script that wraps the near-miraculous Tridgell/ > Mackerras 'rsync' to provide load balancing and parallel operation across > network connections to increase the amount of bandwidth it can use. The > 'fp' variant uses 'fpart' to bypass the need for a full recursive descent > of the dir trees before the actual transfer starts. Versions >2 allow > multihost sends to increase bandwidth saturation WARNING: Do NOT use any > '--delete*' options in the rsync options ('--ro'). parsyncfp2 is > distributed under the Gnu Public License (GPL) v3. Can you please advise > how to fix this? — Reply to this email directly, view it on GitHub <#9 > <#9>>, or unsubscribe > https://github.com/notifications/unsubscribe-auth/AASF3Y6IHNZKYPOAQERPFTDW5GSSTANCNFSM6AAAAAAWCNSRY4 > . You are receiving this because you are subscribed to this thread.Message > ID: *@*.***> > -- Harry Mangalam > > Actually rsync works w/o problems. > > — > Reply to this email directly, view it on GitHub > <#9 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AASF3Y35KZZ2OKICHWCYB33W5HFUDANCNFSM6AAAAAAWCNSRY4> > . > You are receiving this because you commented.Message ID: > ***@***.***> > -- Harry Mangalam

-- Harry Mangalam