Plotman seems stuck on phase 4? Not writing to DST? #508
-
|
So plotman has been running for around 2 days. But I look in the DST drives like mnt/HDD01 which are enabled as FARM directories in Chia GUI Client. They are also in plotman.yaml. But it seems like plotman is unable to move them from TMP drives to DST drives. Why is this? Also. I am both plotting and farming on the same machine... so my Archive is not Rsync'd to anywhere and is currently disabled. I just want to fill my 12 DST drives on the same machine as plotman doing the plotting with NVMEs. What do I need to do in order to free up plotman to move these files to DST? I dont see any errors in the logs, just tons of phase 4:0 as seen in the image. Thanks, |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 7 replies
-
|
Archival is still appropriate for local disks. The |
Beta Was this translation helpful? Give feedback.
-
|
How do I setup archive to work with local disks? Every guide I’ve seen assumes a separate farmer. I was able to solve the DST writing issue going as root and making those directories CHMOD 777. |
Beta Was this translation helpful? Give feedback.
-
|
seem like your plot couldnt get out fast enough. Congestion. My guess, your hdd connection is usb2 |
Beta Was this translation helpful? Give feedback.
-
Hey bud. I'm sata6 on all drives (or nvme depending). It turned out to be the permissions. I needed CHMOD 777 for that folder so since it was only logs. |
Beta Was this translation helpful? Give feedback.
-
|
I've seen this a few times. It's like it dies on the copy from tmp to dst. When I dug in I saw that the tmp file was the right size so I killed the chia process and renamed the plot's tmp file to be .plot and moved it to the farm and it was happy with it. Probably some IO timeouts or something I'm guessing. |
Beta Was this translation helpful? Give feedback.
-
|
If we had the source code for create_plot_disk() we could fix these problems, sadly 'chia' is a black-box None of these problems have anything to do with 'plotman' as its just a passive tool that collects information from state of machine, and chia-logs. Most important thing is the NVME has a 1/2 life of about 2-4 weeks, I normally see most of my NVME's DIE at 25% of TBW ( tera bytes written ) see 'sudo nvme /dev/xxx',this is the kind of thing that would be useful on 'plot-man' to alert people when their nvme drives near terminal failure, as 'chia create plots' has NO MESSAGING of any kind, we have to treat the patient as a black-box, but we know the NVME's drives are killed at a certain level of usage. What I find is that while the NVME drive will fail -t that at the point of early failure they can be used to -2 ( temp2 ), which just writes out linear 108gb files, which is fine even with a drive near failure and post failure ( -temp1 ) The problem with -temp ( temp 1 directory ) is that the 100's of 1,000's of random read/writes only has to have an error of 1 in 4+ billion to cause the 'mis-match' at the end of plot creation, which causes the 'create plot' to terminate. Other than this 'mis-match' error, the other symptom of a DYING NVME is they'll hang, burn cpu cycles, but never increment file-size, you will see stuck sizes of 24gb, 42gb, 74gb in the -temp1 folders of the plots So plotman needs to track all the temp-files for size, have a tolerance of say 10min, if size doesn't change, then send email to client that plot has failed (hung) and change-out that NVME, and use it as a -2 temp-dir. |
Beta Was this translation helpful? Give feedback.
-
Turned out it was some permissions on the MNT/HDD0X folders that i was using in DST. One person has suggested I setup a local archive, to allow archive to do some of it's magic. But the only option I see is RSYNC, and why/how would I use that to speak to local machine and move files from the NVME temps. Also wouldnt that mean I need to use one of my HDDs as a DST drive? Or burn an NVME doing buffering? That seems strange instead of just making each HDD a final resting place and using all available NVME to plot.. |
Beta Was this translation helpful? Give feedback.
-
|
same as you use remote archive. You can set the host to localhost. rsync is perfectly happy hitting local folders or hitting local host. |
Beta Was this translation helpful? Give feedback.
-
|
This is the official repo for plot creation, good luck. |
Beta Was this translation helpful? Give feedback.
Archival is still appropriate for local disks. The
dstlayer is not meant as a final resting place, just a buffer. The plot processes are chia processes and plotman is not involved in them once they are started so you will have to check the log files and... maybe permissions on the mounted disks?