FLOCK on linux with kernel 5.5 on samba share fails

[LocutusOfBorg]

Hello @druzus,
I found your commit 30d0f5b when debugging a session of failures on our SMB-based network implementation.

We use harbour to lock/write dbf/cdx on SMB3.1.1 network mount share.

However after linux 5.5 (commit d0677992d2af3d65f1c1c21de3323d09d4891537) we got a regression on the shared lock, looking like as your statement on

      * src/rtl/filesys.c
        * do not convert exclusive lock to shared one when file is open in
          readonly mode on POSIX system using flock() for DENY_* flag emulation.
          Now HB_USE_BSDLOCKS is enabled only for Linux and it's documented that
          this implementation allow to use shared and exclusive locks regardless of
          file open mode.

is not true anymore.

I did create a simple c program, using old and new kernel to see the differences, and I'm attaching them for reference to this bug report

#include <stdio.h> 
#include <errno.h> 
#include <fcntl.h> 
#include <sys/file.h>

int main()
{
            struct flock64 lock_info;
            int hFileHandle;
	    // OPEN
	    int ret;
            hFileHandle = openat(AT_FDCWD, "/home/user/net/smb.dbf", O_RDWR);
	    printf("\nhFileHandle %d\n", hFileHandle);
            lock_info.l_type   = F_RDLCK;
            lock_info.l_start  = 0x7fffffffUL;
            lock_info.l_len    = 0x1UL;
            lock_info.l_whence = SEEK_SET;   /* start from the beginning of the file */
            lock_info.l_pid    = 0;
	    ret = flock(hFileHandle, LOCK_EX | LOCK_NB);
	    //ret = fcntl( hFileHandle, ( 0 ) ? F_SETLKW64: F_SETLK64, &lock_info );
	    if (ret == -1)
		    printf("fcntl errno: %d\n", errno);
	
	    // LOCK
            lock_info.l_type   = F_WRLCK;
            lock_info.l_start  = 2147483645;
            lock_info.l_len    = 1;
            lock_info.l_whence = SEEK_SET;   /* start from the beginning of the file */
            lock_info.l_pid    = 0;
	    printf("\nhFileHandle %d\n", hFileHandle);
            ret = fcntl( hFileHandle, ( 0 ) ? F_SETLKW64: F_SETLK64, &lock_info );
	    if (ret == -1)
		    printf("fcntl errno: %d\n", errno);
}

logs.tar.gz

with flock/fcntl I get on old kernel:

hFileHandle 3

hFileHandle 3

while with new kernel >5.5 (6.8 on Ubuntu 24.04 in my case, but I did install all the 5.4 and all 5.5 available on launchpad)
with flock:

hFileHandle 3

hFileHandle 3
fcntl errno: 13

with fcntl:

hFileHandle 3

hFileHandle 3

To unblock the current situation, I found that this simple patch fixes the issue for me:

diff --git a/include/hbapifs.h b/include/hbapifs.h
index 828af0e714..ae860e0220 100644
--- a/include/hbapifs.h
+++ b/include/hbapifs.h
@@ -229,16 +229,6 @@ extern HB_EXPORT int        hb_fsCanWrite    ( HB_FHANDLE hFileHandle, HB_MAXINT
 #  define HB_SHARELOCK_SIZE         0x1UL
 #  if defined( HB_USE_BSDLOCKS_OFF )
 #     undef HB_USE_BSDLOCKS
-#  elif defined( HB_OS_LINUX ) && \
-        ! defined( __WATCOMC__ ) && ! defined( HB_USE_BSDLOCKS )
-      /* default usage of BSD locks in *BSD systems for emulating
-       * MS-DOS/Windows DENY_* flags has been disabled because tests
-       * on FreeBSD 6.2 and macOS shows that this implementation
-       * can create self deadlock when used simultaneously with
-       * POSIX locks - thanks to Phil and Lorenzo for locating the
-       * problem and tests [druzus]
-       */
-#     define HB_USE_BSDLOCKS
 #  endif
 #endif

I'm attaching logs and pcaps with all the 4 cases (old/new kernel and old/new lock mechanism).

I'm however wondering if we should just throw away the HB_USE_BSDLOCKS code since it is unused with this patch.

Note: samba server is configured with "veto oplocks" for dbf/cdx files, to allow record locking granularity

Note2: I'm quoting flock manpage

CIFS details
       Up to Linux 5.4, flock() is not propagated over SMB.  A file with
       such locks will not appear locked for remote clients.

       Since Linux 5.5, flock() locks are emulated with SMB byte-range
       locks on the entire file.  Similarly to NFS, this means that
       [fcntl(2)](https://man7.org/linux/man-pages/man2/fcntl.2.html) and flock() locks interact with one another.  Another
       important side-effect is that the locks are not advisory anymore:
       any IO on a locked file will always fail with EACCES when done
       from a separate file descriptor.  This difference originates from
       the design of locks in the SMB protocol, which provides mandatory
       locking semantics.

       Remote and mandatory locking semantics may vary with SMB protocol,
       mount options and server type.  See mount.cifs(8) for additional
       information.

I'm not expert on samba protocol, nor on harbour code, so please forgive me if the analysis looked uncorrect (and this is why I didn't open yet a PR)

Gianfranco

[druzus]

Hi Gianfranco,

Linux does not support DENY_* flags. We can emulate
some part of this DOS/Win functionality using POSIX
locks or BSD locks. In most of other *nixes BSD locks
are emulated by the kernel as POSIX locks and this
may be the source of deadlock so I disabled them in
Harbour by default. In Linux kernels POSIX and BSD
locks are independent and they do not interact each
other. It means that we have two independent lock
systems just like in DOS and Windows so the emulation
it's much closer to original and does not create series
of bad side effects when such emulation is done using
POSIX locks. So I decided to do that by default. Anyhow
it may create problems when the files are located on
remote servers and Linux has to propagate locks to
the file server. How it's done on the client side and
how they are interpreted by server depends on used
protocol (NFS, SMB, ...), its capability and given
implementation. Very often above two different
locking systems can be mapped on the client or
server side to chosen one and create problem with
deadlocks. For this reason I added to Harbour
compile time macro: HB_USE_BSDLOCKS_OFF
If you want to access files from remote resources then
just simply recompile Harbour with
export HB_USER_CFLAGS=HB_USE_BSDLOCKS_OFF
and if you find other *nix system which just like Linux
uses independent BSD and POSIX locks implementation
and this will be server for your clients accessing local
files then you can enable BSD locks rebuilding Harbour
with:
export HB_USER_CFLAGS=HB_USE_BSDLOCKS

best regards,
Przemek

[LocutusOfBorg]

Hello @druzus thanks for the fast reply!

I have to start by saying that I opened a pull request to address this problem: #389
For the other people who might face similar issue, and to remember in the future about this issue (and what I did to solve it), I didn't export just a variable, but I kicked out the old and now deprecated code (if nobody uses that lock mechanism anymore).

That said, thanks for the reply, what worries me is that the kernel broke a default behavior used by applications since years, and this is a really sad thing to do. I agree that the kernel changing such behavior is not something we should have, and I'm pretty sure @torvalds would agree with me.

Unfortunately this kernel change makes the locking mechanism completely broken and useless, locking an entire DB for a one line change is not feasible, and this change makes performance loss something unacceptable.

Recompiling harbour with a default change is ok for me, I already created a deb, and installed on all my production network, together with the kernel bump.

But I still ask you if you can consider changing this default, because it took days of work, for a non harbour engineer as me, I had to go down to the rabbit hole, do a git bisect of kernel code (yes, this happened because a newer kernel was backported on an Ubuntu LTS, so breakage happened on LTS servers, something you don't expect to happen). It took days to understand the regression was coming on kernel side because rebooting is not something that can be done easily on production servers, and it took even more time to understand why and how the change was made, and debugging of SMB protocol with wireshark was at the end the only option I had to figure it out, as well as creating a simple c file with simple glibc calls to rule out another glibc/kernel bug.

So, for me in order to save other people the same nightmare, I hope

1. if you consider BSDLOCKS to be dropped everywhere now
2. if you consider checking for kernel at runtime and throwing some big message related to samba/linux 5.5
3. if you consider keeping this bug open, so other people might know in advance by going to github what is the issue
4. if you consider my pull request :) https://github.com/harbour/core/pull/389/files

thanks a lot for your time, it was appreciated

[druzus]

A false assumption produces a false conclusion.
To clarify.
The changes in Linux kernel didn't broke anything. Just simply before this modification BSD locks where ignored in case of samba resources. Command like flock (man flock) and C code using flock() function (man 2 flock) didn't work with samba remote filesystems (did you know that all scripts using flock command on files from remote SMB resources were working without any synchronization?)
In case of Harbour Linux builds compiled without HB_USE_BSDLOCKS_OFF macro the emulation of DENY* flags in file open/create operations was completely ignored and didn't work at all.
Now these locks aren't ignored but are propagated to samba server using the SMB API so they have to be translated to standard DOS/Windows file range locks. It means that now BSD locks have a chance to work.
It would be nice to have an option to configure how this translation works, i.e. flock() locks the whole file or locks some region at given offset. In the second case mapping flock() calls to 1 byte at offset 0x7fffffff will give the same behavior as Harbour does when is compiled without BSD locks. I chosen this address because it was used by samba server to emulate DENY modes. I do not know if it's still true. You may check it in samba source code.

The problem with flock() emulation isn't new. It is known for very long time. People using NFS resources also have it and in this case Linux kernel has propagated BSD locks to client side for more then 20 years so they have to use Harbour Linux builds compiled HB_USE_BSDLOCKS_OFF from beginning.

So why HB_USE_BSDLOCKS_OFF is not default in Harbour Linux builds?
Because most of Harbour Linux applications are executed on the server side and access files from local disks what gives many times better performance then accessing files from network resources. For them BSD locks works nicely and these locks are bound with file descriptor not file inode+pid like standard fcntl() POSIX locks so they do not have bad side effects which POSIX locks inherited from the ill specification (i.e. close() releases all locks of calling process even if the file is still open by other file description so if you execute function which open and close DBF file which is used by your code all your locks on this file suddenly disappear. To fix POSIX fcntl() locks bad side effects Linux kernel from 3.15 supports new non POSIX locks called "Open file description locks" (F_OFD_* fcntl() operations). I'm sill looking for spare time to implement them in Harbour.
Anyhow I'll think about runtime function to enable/disable BSD locks for DENY modes emulation.

[LocutusOfBorg]

Hello, is samba locking implemented here?
https://gitlab.com/samba-team/samba/-/blob/master/source3/locking/posix.c?ref_type=heads

Btw thanks for the detailed answer, and yes, enabling/disabling BSD locks for DENY modes will for sure fix my issue in a better way!

[manuel-rubio]

@LocutusOfBorg, if this issue isn't an issue anymore, please close it and do the same with the PR. Thank you both for this discussion, it's worth it for learning the insights about how Samba and the Linux kernel work.

[alcz]

If someone else reaches here having issues, it's also worth pointing at Harbour's NETIO: https://github.com/harbour/core/tree/master/contrib/hbnetio
That may be a less hassle replacement for remote locking issues.

[manuel-rubio]

If someone else reaches here having issues, it's also worth pointing at Harbour's NETIO: https://github.com/harbour/core/tree/master/contrib/hbnetio That may be a less hassle replacement for remote locking issues.

@alcz great suggestion, I think it could even be included in the documentation.

[prgwiz]

I would agree. MS keeps changing the protocols for file sharing and Netio has been very good. Postgres is even better ;-)

…

On Tue, Oct 28, 2025 at 7:31 AM Manuel Rubio ***@***.***> wrote: *manuel-rubio* left a comment (harbour/core#388) <#388 (comment)> If someone else reaches here having issues, it's also worth pointing at Harbour's NETIO: https://github.com/harbour/core/tree/master/contrib/hbnetio That may be a less hassle replacement for remote locking issues. @alcz <https://github.com/alcz> great suggestion, I think it could even be included in the documentation. — Reply to this email directly, view it on GitHub <#388 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABTQJT2R7AYLD2Q6ZD3WBZD3Z5HYLAVCNFSM6AAAAACDWJGTVKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTINJVHE4TQMJYGI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>