-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Logical Volume is in inconsistent state after running Windows chkdsk. #29
Comments
Can you create small UDF partition with this problem and provide dump of it? |
The only HDD I have is 1TB. How do I provide a dump? Sorry, not exactly Linux savvy here. I've been trying to get a working UDF disk working for about 6 hours with multiple attempts made in Linux and in Windows 10. |
As you are using GParted, just create e.g. 10MB partition on it, not need to do full 1TB. And once you have it in that "non-working" state create dump via dd command (sudo dd if=/dev/your_partition of=/path/to/file_dump) |
Alright. Here if chkdsk's output(if it helps anyway): The type of the file system is UDF. CHKDSK is verifying ICBs ... Windows has made corrections to the file system. 9983 KB total disk space. 512 bytes in each allocation unit. and uploaded is the dump. Disk is a 512 emulated(512 logical, 4096 phyiscal) drive. Don't know if that matters or not. |
udfinfo is right about inconsistent state. Logical Volume Integrity Descriptor is in Open state. And therefore fixing UDF filesystem is needed prior to using it. Seems that you hit a bug in windows chkdsk which just damaged UDF filesystem. @smagnani is working on fsck.udf so maybe would know about this problem or could look how to integrate fix procedure for these problems. |
Is there anyone at MS that could be contacted to check into this? I know at least one Microsoft employee on Reddit that I could try contacting but IDK but kind of response i'd actually get... |
The dump is from after chkdsk has "repaired" the partition? --Steve |
Also it might help to know versions of software components involved: Windows 7/10/?, which Linux kernel/distro, which GParted version. |
Pali, from udfinfo output it does not look like the program checks to see whether the requested partition is currently mounted. That could explain a "udfinfo: Warning: Logical Volume is in inconsistent state" message where one isn't expected. |
Thanks smag. Just for the giggles, I tried mounting and unmounting the disk from Gnome Disks right after using Gparted and you are correct: the warning only ever shows when it's mounted but NOT when it isn't. Gparted doesn't mount the disk after formatting either and that dump was made directly after using Gparted without mounting. Here is the log after Gparted but WITHOUT mounting it: I'm using Arch Linux which currently has Gparted V1 and I am using 5.2.8-zen1-1-zen kernel. |
Can you clarify on how "fragile" UDF is? I want this to be a shared drive between Windows and Linux and really don't want to manually unmount it every time... |
Same output behavior even after running chkdsk, warning shows when mounted but not when mounted: |
Pali, one thing I see in comparing the dumps is that chkdsk does not expect a terminating block following the 12 blocks of the VRS. That's why it complains about the Unallocated Space Descriptor. When it "repairs" the partition the terminating block becomes marked as unallocated volume space. |
About fragility...I think you're seeing some of it - although I'd like to get to the bottom of it because you're seeing interoperatbility problems I haven't. It's late here so for now let me just point you to some recent threads: |
Last for tonight...Ty, what version of Windows are you using? Can you right-click on chkdsk.exe (I think it's in C:\Windows\System32) and find its version property? |
Right, sorry for forgetting that. File version is 10.0.18956.1000. Windows version is Windows 10 Pro for Workstations(insider preview). Build is 18956.rs_prerelease.190803-1414. |
I should also bring up that files made from within Linux are marked as read only in Windows. I assumed this was normal but the second link seems to describe this as an issue... edit: and making them writable is persistent and is only needed once. |
That is truth. But any other file disk utility (e.g. e2fsprogs) do not work correctly when using them on mounted file system. I will probably add some checks if partition is mounted also for read-only operations. Or at least print warning that output data may be bogus. |
Ty - My chkdsk is 10.0.17134.1. I can get it to produce the same output yours does, so at least Microsoft hasn't introduced new bugs in your insider preview. I think the "errors in Space Bitmap Descriptor at block 0" in your chkdsk output is a result of the issue described here: That one can be worked around with a tweak to mkudffs. Pali, I remember said you were having trouble getting github to pull from my fork; can you just apply the change to your repo as a patch instead? Do you need me to mail you a .patch file? Interestingly, there are no reports of Unallocated Space Descriptor errors unless you run chkdsk with /f. So that's apparently a bug in chkdsk, which is disappointing since it's really the only game in town at the moment. I wonder if chkdsk would like it better if mkudffs terminated the VRS with an actual Terminating Descriptor instead of a block of zeroes. If so we could work around that too. |
Never mind. The standard is pretty clear, the block following the VRS has to be all zero (or unrecorded, which doesn't apply for HD media). So I guess the takeaway here is, always run chkdsk without the '/f' option first, and only re-run with repairs enabled if something is actually reported as wrong in the check-only pass. Ty, can you clarify what you mean by "files made from within Linux are marked as read only in Windows"? Are files individually read-only in Windows, but the volume as a whole is writable? Or does Windows treat the volume as a whole as read-only? I've seen a lot of the latter, usually when the udfs.sys driver thinks the bitmap is missing or unusable, but I can't say I've seen the former. |
Individual folders and files. The media as a whole is writable. |
And are you able to change permissions for those individual folders and files either on Linux via chmod or on Windows via properties dialog? |
In Windows, yes. In Linux they are always writable. |
Ty - nice catch. I never noticed that this was occurring, probably because I work much more in Linux than Windows. This looks like an issue we might be able to fix in the Linux UDF driver. The problem is that UDF defines a slightly different set of basic file permissions than Linux - specifically, UDF has "delete" and "change attribute" permissions for each access class (user/group/other). There are no equivalents for these in Linux. When the Linux UDF driver creates a file (or directory), no UDF delete or change attribute permissions are granted. It seems the lack of delete permission is what causes Windows to mark an item read-only when its permissions otherwise indicate that it should be read-write. If anyone (U or G or O) had delete permission the item would not be marked read-only. For the record, I can't see that the UDF change attribute permission has any effect in Windows. |
Thanks. I was just booting back and forth to make sure that both Linux and Windows where playing nice together with the UDF drive and ran into it. So, with everything said, can UDF be used safety at all on a portable HDD? If not, is there rough guesses as to when it'll be in a usable state? I'd really like to offload my shared Windows games(Proton) and project files to it. Currently I'm using an NTFS drive but Linux corrupts it so often that I need to redownload games and repair it in Windows ...it's becoming a real bother. No other cross-platform FS(Fat32/64) that i'm aware of besides UDF has a rich feature set. FYI, Windows sometimes runs chkdsk on startup... does it actually do anything that hurts the data at all? |
I sent a patch upstream for the read-only issue, we'll see how it goes: Whether UDF can be used "safely" is a tricky question to answer. It depends on your use case and risk tolerance. My personal feeling is that UDF works best in read-only filesystems or those that are lightly updated. The more heavily the filesystem is updated, or the more UDF features it makes use of, the less I'd recommend it. From what you've said I think the same might be true for NTFS. In part because UDF is not journaled, it's also a poor choice for users who don't want to dismount the filesystem gracefully. Reminds me of the Bad Old Days of SVR4 where filesystems were so very easy to corrupt. The lack of a viable open-source fscker for UDF is a huge problem, especially given reports of filesystem damage caused by the only alternative (Windows chkdsk). I've personally lost an entire USB stick of files to a chkdsk 'repair'. The loss may have been provoked by corruption from some (now fixed) Linux UDF driver bugs, but at this point I would never attempt a chkdsk repair on a personal filesystem without making a backup first. Having spent a good bit of time reviewing and trying to improve some open-source attempts at a UDF fscker, I can say that the richness of the standard makes development of a complete checker daunting. To avoid creating damage in places where none existed, any checker has to refuse to repair a filesystem that makes use of UDF features it doesn't support. Filesystem drivers probably need to adopt more of a "first do no harm" strategy for similar reasons. Earlier you mentioned a Microsoft contact - it may help move the ball forward if we can bring recently identified bugs to Microsoft's attention: |
I think I just answered my own question: `The type of the file system is UDF. CHKDSK is verifying ICBs ... Windows has checked the file system and found problems. 976760575 KB total disk space.
1953521150 total allocation units on disk. C:\Users\young>chkdsk /f D: CHKDSK is verifying ICBs ... Windows has made corrections to the file system. 976760575 KB total disk space.
1953521150 total allocation units on disk. With only three games on a freshly formatted drive from Linux to Windows this all happened. The Microsoft contact is just someone who posts the Insider previews on the Windows subreddit. I don't have MS on speed dial sadly, but i'll give it a shot. |
Sorry. I wish it were ready for prime-time, I really do. If you're able to get something like this to happen on a smaller filesystem, or can provide a recipe for how you populated this one, I can try to reconstruct the crashed airplane to see if there might be other Linux driver bugs we haven't found yet. I've never seen reports of "Error in allocation descriptors" before, never mind such extensive ones. There was a driver bug I fixed awhile back that could cause problems on large filesystems (https://lkml.org/lkml/2017/10/12/327) but I don't think bad allocation descriptors was one of the symptoms, and it doesn't look like your partition is quite large enough. |
Check added for udfinfo and udflabel. |
Running chkdsk on a UDF partition formatted using GParted results in a working and usable UDF disk... however, running chkdsk anytime after results in an "Logical Volume is in inconsistent state" warning when checking the drives info via udfinfo.
Disk is a Seagate backup plus 1TB USB 3.0 drive.
The text was updated successfully, but these errors were encountered: