fd can get stuck when ran onto the whole FS from the root

[Porkepix]

The ran command is just fd foobar /. It gets stuck and never end. I sadly was unable to understand what's causing it.
I can investigate to give answers if you have some ideas about what else could I check.

Below is the partition's setup:

# lsblk
NAME           MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda              8:0    0 223,6G  0 disk  
├─sda1           8:1    0   512M  0 part  /boot/efi
├─sda2           8:2    0   200M  0 part  
│ └─cryptboot  254:3    0   198M  0 crypt /boot
└─sda3           8:3    0 222,9G  0 part  
  └─lvm        254:0    0 222,9G  0 crypt 
    ├─vg0-swap 254:1    0     8G  0 lvm   [SWAP]
    └─vg0-root 254:2    0 214,9G  0 lvm   /

EDIT: Issue is reproducing everytime on this Archlinux setup. Couldn't reproduce it on another Archlinux with unencrypted system.

[sharkdp]

I can reproduce this on my PC. This seems to be a permission problem with files in /proc and /sys.

Could you try to run

fd -E /proc -E /sys foobar /

where -E (--exclude) excludes these two folders?

(Note: you might have to exclude some other mount points as well)

[Porkepix]

Indeed this does work. Strange that my personal server doesn't have the same issue though.

[Porkepix]

Oh, and the issue was present even when running fd as root. So I'm not sure about the permission idea.

[sharkdp]

Oh, and the issue was present even when running fd as root. So I'm not sure about the permission idea.

I see, interesting. Something else must be going on inside /proc and/or /sys. If anybody has an idea on how to fix this, I'd be glad for any hints.

[Porkepix]

Excluding only proc is enough, so problem comes from something inside /proc.
343 items inside /proc makes a lot of things to test though.

[Porkepix]

Even more interesting.
fd -E "/proc/[0-9]*" foobar / is succeeding.
fd -E "/proc/[0-9]*" foobar /proc is failing.

[sharkdp]

Wait, this also seems to change from time to time. Right now, fd foobar / works perfectly fine on my machine(?!?).

[sharkdp]

I suspect this could be caused by infinite recursion within /proc (see here or here).

Quote:

Avoiding symbolic links may not be sufficient for avoiding "infinite" recursion in /proc. On the bright side, "infinite" is bounded by PATH_MAX...

Unfortunately, I cannot reproduce the problem at the moment. Could you try

fd --max-depth 50 foobar /

?

[sharkdp]

I'm going to close this (for now), as there is no further feedback. Feel free to comment here if this should be re-opened.

The workaround for now is to explicitly exclude /proc via

fd -E /proc ...

[Porkepix]

Sorry, didn't knew you was waiting for input: it was added with an edit, so I wasn't notified: original comment was not asking for input. Please post a new comment in such situation.

I tested what you asked for: didn't solved the issue, so I guess this isn't infinite recursion.
And while it hang like that, CPU is using every single core at 100%.

I guess we could then reopen it?

EDIT: Even if I'm the opener I can't reopen myself.

[Porkepix]

By the way, this was on Linux computers.
I'm currently running it on an old mac. I guess it's gonna take long (old, bad shaped HDD, no SSD in it), so I'll let it run, but I think that when I'll be back this evening computer will be brozen with out of memory: currently running for like 10 minutes, CPU is only around 10%, but memory constantly increase. It started low and currently is already at 140MB.

EDIT: CPU lowered to 1.5-2%, but it still continue.

EDIT2: Might not be the bug on the mac, 30 minutes later, it found a file with "foobar" in the name (!) at the 9th level of depth. So I'll see this evening if it ended correctly, the HDD is kinda slow, and probably fd is slowed down by I/O on disk rather than CPU like other SSD-using computers.
However, is it normal for the memory to constantly increase over the time?