Locking

[egandro]

Is there some file locking implemented?

E.g. I run a k8s cluster with multiple workers and multiple parallel instances of the workers.

The $HOME/.n8n directory is shared via NFS over the cluster.

Q: What happens if multiple workers do a write operation to the same JSON file? Is there some locking implemented?

Thx.

[andrigamerita]

Hi @egandro, a file locking mechanism is indeed implemented, for both reads and writes. However, I have only been able to test (and confirm) its effectiveness in the context of a single n8n instance, with of course both multiple workflows and multiple branches of a workflow trying to access the same file at the same time.

I have never tried a configuration with a k8s cluster, or really any multiple n8n instances sharing files but being otherwise independent. I should test this case, but I believe that no issues should ever arise (assuming your file system is working properly), since I implemented file locking in a way that is bound only to the file system where the JSON file resides, and the system time.

It works like this: a process tries to acquire a lock by creating a so-called lock directory, and if it succeeds then it acquires the lock and takes care of deleting it when it finishes, and refreshing it if takes more than a few seconds to complete; if it can't acquire the lock, it waits and keeps trying if it sees the lock is being is being refreshed, otherwise it considers it stale and forcibly acquires it after another few seconds (to prevent any crashed process from causing a ghost lock).

One thing I'm now not sure of is what happens if the n8n instances run on a different system time: if the time is radically different, then issues are guaranteed to arise, but I don't know what happens when the difference is just of timezone, I never tested it.

I will try to do the tests I can and report it here or clarify things better in the README.