rsync cannot find the target path #18
Comments
|
或许他与这个issues相关 |
|
I added the --debug option and got more detailed information. It is clear that the command in line 1770 of the code does not have a given target path. Core issues：
All output：(pytorch) root@notebook-lisirui-cpu1-c7ewc7-notebook-0:~# parsyncfp2 --NP=1 --interface=eth0 --verbose=3 --debug --startdir=/root one /root/two
204 : **DEBUG PROGRESS**
306 : **DEBUG PROGRESS**
notebook-lisirui-cpu1-c7ewc7-notebook-0 DEBUG: [356]: FPARTSIZE set to: [10G]
FPARTSIZE_N set to [10737418240]
Press [ENTER] to continue.
403 : **DEBUG PROGRESS**
notebook-lisirui-cpu1-c7ewc7-notebook-0 WARN: The --NP value is only [1]; to take advantage of pfp2, this should be
set to at least 4. Continuing inefficiently..
587 : **DEBUG PROGRESS**
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: Using [eth0] to send data and to monitor
690 : **DEBUG PROGRESS**
notebook-lisirui-cpu1-c7ewc7-notebook-0 DEBUG: [706]: Using network interface [eth0] with connection quality [ 10000Mb/s]
Press [ENTER] to continue.
notebook-lisirui-cpu1-c7ewc7-notebook-0 DEBUG: [730]: 1m load is [2.54] and the 1m Load:#CPU ratio is [0.0264583333333333] ( [96] CPU cores).
OK to continue.
Press [ENTER] to continue.
notebook-lisirui-cpu1-c7ewc7-notebook-0 WARN: About to remove all the old cached chunk, log,
and PID files from [/root/.pfp2/fpcache]
and the previous rsync log files from [/root/.pfp2].
Enter ^C to stop this.
Press [ENTER] to continue.
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: [17] files from the previous run have been cleared.
Re-initializing dirs and configuring fpart.. This may take a few secs.
786 : **DEBUG PROGRESS**
notebook-lisirui-cpu1-c7ewc7-notebook-0 DEBUG: [795]: Composing the new fpart target dirtmp in a loop.
Press [ENTER] to continue.
notebook-lisirui-cpu1-c7ewc7-notebook-0 DEBUG: [827]: Looping to add the fpart target: ['/root/one' ]
Press [ENTER] to continue.
901 : **DEBUG PROGRESS**
notebook-lisirui-cpu1-c7ewc7-notebook-0 DEBUG: [977]: fpart cmd:
[(fpart -vvv -L -W 'mv $FPART_PARTFILENAME /root/.pfp2/fpcache' -z -s 10737418240 -o /root/.pfp2/fpcache/hold/f '/root/one' 2> /root/.pfp2/fpcache/FPART.log.11.18.04_2024-05-08 & fpid=$!; fpr='r'; while [ ! -z "$fpr" ]; do sleep 1; fpr=`ps -h -p $fpid`; done; touch /root/.pfp2/fpcache/FPART_DONE ) & ]
Press [ENTER] to continue.
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: fpart launched with command:
[(fpart -vvv -L -W 'mv $FPART_PARTFILENAME /root/.pfp2/fpcache' -z -s 10737418240 -o /root/.pfp2/fpcache/hold/f '/root/one' 2> /root/.pfp2/fpcache/FPART.log.11.18.04_2024-05-08 & fpid=$!; fpr='r'; while [ ! -z "$fpr" ]; do sleep 1; fpr=`ps -h -p $fpid`; done; touch /root/.pfp2/fpcache/FPART_DONE ) & ]
998 : **DEBUG PROGRESS**
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: Waiting [2]s for first chunk files to be written.
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: [/root/.pfp2/fpcache/f.1] visible.
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: Waiting [4]s for first chunk files to be written.
1108 : **DEBUG PROGRESS**
1184 : **DEBUG PROGRESS**
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: Starting the 1st [1] rsyncs ..
1279 : **DEBUG PROGRESS**
1468 : **DEBUG PROGRESS**
1468 : **DEBUG PROGRESS**
notebook-lisirui-cpu1-c7ewc7-notebook-0 DEBUG: [1581]: sPIDs string = []
Press [ENTER] to continue.
1591 : **DEBUG PROGRESS**
notebook-lisirui-cpu1-c7ewc7-notebook-0 DEBUG: [1770]: Starting [cd && rsync --bwlimit=0 -as --log-file=/root/.pfp2/rsync-log-11.18.04_2024-05-08_1 --files-from=/root/.pfp2/fpcache/f.1 '/root' & echo "${!}" >> /root/.pfp2/rsync-PIDs-11.18.04_2024-05-08]
Press [ENTER] to continue.
1776 : **DEBUG PROGRESS**
notebook-lisirui-cpu1-c7ewc7-notebook-0 DEBUG: [1779]: Starting [cd && rsync --bwlimit=0 -as --log-file=/root/.pfp2/rsync-log-11.18.04_2024-05-08_1 --files-from=/root/.pfp2/fpcache/f.1 '/root' & echo "${!}" >> /root/.pfp2/rsync-PIDs-11.18.04_2024-05-08]
Press [ENTER] to continue. |
|
HI, Thanks for reporting this.
I'm travelling and away from my main rig, but from looking at the versions
involved and what you're trying to do (rsync in a local filesystem), the
problem is that you're using a recent version of fpart (1.51) with an older
version of pfp2. The newer version of fpart starts numbering the chunk
files at 1 instead of 0, so pfp2 is waiting for f.0 to appear, which will
never happen. I believe that you have to go back to a version of fpart
before 1.5 for it to start numbering at 0. More recent versions of pfp2
check compatibility of fpart.
You can pull an old version of fpart here:
https://github.com/martymac/fpart/releases/tag/fpart-1.4.0
The second possible problem is that pfp2 was not meant for rsyncing across
local file systems and I thought that it actually refused to do this (but I
do not see the refusal text in your listing). The current version will do
this, tho not particularly well for many reasons. Using pfp2 with an NP=1
and syncing across a local file system will definitely be slower than just
using native rsync - is there another reason for trying that?
hjm
…On Tue, May 7, 2024 at 8:00 PM whoami ***@***.***> wrote:
Issue:
Parsyncfp2 encounters an error when parsing the rsync command. From the
logs, it seems that there is a problem with the target address parsed on
line 1770 of the code, but I'm unable to fix it.
System Version:
- Ubuntu VERSION="22.04.4 LTS (Jammy Jellyfish)"
- fpart v1.5.1
- parsyncfp2 version 2.59
- rsync version 3.2.7, protocol version 31
Command Run:
parsyncfp2 --NP=1 --interface=eth0 --verbose=3 --startdir=/root one
/root/two
Output:
notebook-lisirui-cpu1-c7ewc7-notebook-0 WARN: The --NP value is only [1]; to take advantage of pfp2, this should be
set to at least 4. Continuing inefficiently..
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: Using [eth0] to send data and to monitor
notebook-lisirui-cpu1-c7ewc7-notebook-0 WARN: About to remove all the old cached chunk, log,
and PID files from [/root/.pfp2/fpcache]
and the previous rsync log files from [/root/.pfp2].
Enter ^C to stop this.
Press [ENTER] to continue.
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: [9] files from the previous run have been cleared.
Re-initializing dirs and configuring fpart.. This may take a few secs.
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: fpart launched with command:
[(fpart -vvv -L -W 'mv $FPART_PARTFILENAME /root/.pfp2/fpcache' -z -s 10737418240 -o /root/.pfp2/fpcache/hold/f '/root/one' 2> /root/.pfp2/fpcache/FPART.log.10.51.44_2024-05-08 & fpid=$!; fpr='r'; while [ ! -z "$fpr" ]; do sleep 1; fpr=`ps -h -p $fpid`; done; touch /root/.pfp2/fpcache/FPART_DONE ) & ]
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: Waiting [2]s for first chunk files to be written.
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: [/root/.pfp2/fpcache/f.1] visible.
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: Waiting [4]s for first chunk files to be written.
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: Starting the 1st [1] rsyncs ..
[notebook-lisirui-cpu1-c7ewc7-notebook-0] : Starting [cd && rsync --bwlimit=0 -as --log-file=/root/.pfp2/rsync-log-10.51.44_2024-0rsync version 3.2.7 protocol version 31.1 '/root' & echo "${!}" >> /root/.pfp2/rsync-PIDs-10.51.44_2024-05-08]
Copyright (C) 1996-2022 by Andrew Tridgell, Wayne Davison, and others.
Web site: https://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, symlinks, symtimes, hardlinks, hardlink-specials,
hardlink-symlinks, IPv6, atimes, batchfiles, inplace, append, ACLs,
xattrs, optional secluded-args, iconv, prealloc, stop-at, no crtimes
Optimizations:
SIMD-roll, no asm-roll, openssl-crypto, no asm-MD5
Checksum list:
xxh128 xxh3 xxh64 (xxhash) md5 md4 sha1 none
Compress list:
zstd lz4 zlibx zlib none
Daemon auth list:
sha512 sha256 sha1 md5 md4
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
rsync is a file transfer program capable of efficient remote update
via a fast differencing algorithm.
...
...
...
Use "rsync --daemon --help" to see the daemon-mode command-line options.
Please see the rsync(1) and rsyncd.conf(5) manpages for full documentation.
See https://rsync.samba.org/ for updates, bug reports, and answers
rsync error: syntax or usage error (code 1) at options.c(2433) [client=3.2.7]
notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: All my rsyncs issued; waiting for them to finish.
(stats) ERROR on input, no usable data. Check what you're feeding me.notebook-lisirui-cpu1-c7ewc7-notebook-0 INFO: (Reminder) If you suspect errors, check the rsync logs. [/root/.pfp2/rsync-log-10.51.44_2024-05-08_*] and the fpart log: [/root/.pfp2/fpcache/FPART.log.10.51.44_2024-05-08] Your fpcache files were written in [/root/.pfp2/fpcache] and take up [40K /root/.pfp2]. You rsync'ed about [ bytes = 0.00000 MB] via all [1] rsyncs on this host.
If you used the multihost mode, the whole pfp2 log is here:
[/root/.pfp2/pfp-log-10.51.44_2024-05-08]
(mostly text but identifies as 'binary' due to text coloration strings):
If you used the singlehost mode and want to log the output, capture it
by prefixing the pfp2 command with 'script' ('man script' for details).
The host-specific command used for this parsyncfp2 run (in case the host
dies midway or for debugging purposes) was:
[/usr/bin/parsyncfp2 --NP=1 --interface=eth0 --verbose=3 --startdir=/root one /root/two]
—
Reply to this email directly, view it on GitHub
<#18>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AASF3Y3U3S2UMVU7B3LKL2DZBGIM5AVCNFSM6AAAAABHMDVFWWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI4DINJXG43DGNQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
--
Harry Mangalam
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Issue:
Parsyncfp2 encounters an error when parsing the rsync command. From the logs, it seems that there is a problem with the target address parsed on line 1770 of the code, but I'm unable to fix it.
System Version:
Command Run:
parsyncfp2 --NP=1 --interface=eth0 --verbose=3 --startdir=/root one /root/twoOutput:
The text was updated successfully, but these errors were encountered: