Can't lock the .nextflow/history file

[RongrongZhao]

hi, I'm running my pipeline on cluster. I submitted in array jobs using qsub. Most jobs ran successfully but a few failed with the same error: can't lock file .nextflow/history. I'm not sure why this just happened to a few jobs ,these jobs are actually the same, just with different input file. Does anyone know how to solve this problem?

Many thanks!

[abhi18av]

Hi @RongrongZhao ,

Would it be possible for you to share the .nextflow.log file - it might help in understand what might be happening.

[RongrongZhao]

Hi, sorry for late reply. Here is the relevant part in the .nextflow.log:
May-05 14:25:42.482 [main] DEBUG nextflow.cli.Launcher - $> nextflow run main.nf --path /exports/eddie/scratch/s1616612/Annotation_pipeline_4.10/split_data/xfe
May-05 14:25:42.502 [main] DEBUG nextflow.cli.Launcher - $> nextflow run main.nf --path /exports/eddie/scratch/s1616612/Annotation_pipeline_4.10/split_data/xew
May-05 14:25:43.670 [main] ERROR nextflow.cli.Launcher - @unknown
java.lang.IllegalStateException: Can't lock file: /exports/eddie/scratch/s1616612/Annotation_pipeline_4.10/.nextflow/history -- Nextflow needs to run in a file system that supports file locks
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:80)
at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:74)
at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callConstructor(ConstructorSite.java:84)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:59)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:237)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:249)
at nextflow.util.HistoryFile.withFileLock(HistoryFile.groovy:433)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:190)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:58)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:51)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:156)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:168)
at nextflow.util.HistoryFile.generateNextName(HistoryFile.groovy:461)
at nextflow.cli.CmdRun.checkRunName(CmdRun.groovy:284)
at nextflow.cli.CmdRun.run(CmdRun.groovy:240)
at nextflow.cli.Launcher.run(Launcher.groovy:460)
at nextflow.cli.Launcher.main(Launcher.groovy:642)

[pditommaso]

It's written

java.lang.IllegalStateException: Can't lock file: /exports/eddie/scratch/s1616612/Annotation_pipeline_4.10/.nextflow/history -- Nextflow needs to run in a file system that supports file locks

[abhi18av]

Hi @RongrongZhao

Can't lock file: /exports/eddie/scratch/s1616612/Annotation_pipeline_4.10/.nextflow/history -- Nextflow needs to run in a file system that supports file locks

These lines here gives the main clue, the filesystem being used at your cluster doesn't support file locks - perhaps you need to get in touch with your cluster admins.

There are a couple of related threads which might help you in the discussion
#1310
#1544

Update: Oh, I see I was a couple minutes late 😅

[RongrongZhao]

Thanks for your reply! I submitted array jobs using qsub and there were 25 jobs in total. Why did this problem occur in only two jobs, while the other jobs were all finished successfully?

[Zruto]

Hello, I also encountered a similar problem. In a batch of tasks, several errors occasionally appeared. Did you finally find the reason?

[mjuilk]

Hello, I also encountered a similar problem. In a batch of tasks, several errors occasionally appeared. Did you finally find the reason?

Maybe this helps somebody who looks this up in the future, but my problem was that I was trying to run nextflow in the $DATA folder of the HPC cluster which uses a different file system from $HOME. When I ran in $HOME, it worked perfectly.