cmdheight=0: NULL dereference in grid_del_lines()

[fidgetingbits]

Problem

I'm trying to setup some lua tests for a neovim plugin following this.

This resulted in me running into a NULL pointer deref:

(gdb) bt
#0  0x00000000005d3cbb in grid_del_lines ()
#1  0x0000000000671cb0 in msg_scroll_up ()
#2  0x00000000006761ba in msg_start ()
#3  0x0000000000674ef4 in msg_attr_keep ()
#4  0x00000000006754e9 in emsg_multiline ()
#5  0x0000000000676dbd in semsg_multiline ()
#6  0x0000000000623eb0 in nlua_error ()
#7  0x000000000062ddcd in nlua_exec_file ()
#8  0x0000000000459a93 in main ()
(gdb) 
(gdb) x/i $pc
=> 0x5d3cbb <grid_del_lines+587>:       mov    (%rdx),%rdi
(gdb) i r
rax            0x0                 0
rbx            0x1                 1
rcx            0x1                 1
rdx            0x0                 0
rsi            0x0                 0
rdi            0x966a80            9857664
rbp            0x1                 0x1
rsp            0x7ffedab7b660      0x7ffedab7b660
r8             0x0                 0
r9             0x0                 0
r10            0x0                 0
r11            0x1                 1
r12            0x0                 0
r13            0x0                 0
r14            0xffffffff          4294967295
r15            0x966a80            9857664
rip            0x5d3cbb            0x5d3cbb <grid_del_lines+587>
(gdb) x/20i $pc-25
   0x5d3ca2 <grid_del_lines+562>:       pop    %rbp
   0x5d3ca3 <grid_del_lines+563>:       pop    %r14
   0x5d3ca5 <grid_del_lines+565>:       pop    %r15
   0x5d3ca7 <grid_del_lines+567>:       ret
   0x5d3ca8 <grid_del_lines+568>:       nopl   0x0(%rax,%rax,1)
   0x5d3cb0 <grid_del_lines+576>:       mov    0x20(%r15),%rsi
   0x5d3cb4 <grid_del_lines+580>:       movslq %r10d,%rdx
   0x5d3cb7 <grid_del_lines+583>:       lea    (%rsi,%rdx,8),%rdx
=> 0x5d3cbb <grid_del_lines+587>:       mov    (%rdx),%rdi
   0x5d3cbe <grid_del_lines+590>:       cmp    %r11d,%r14d
   0x5d3cc1 <grid_del_lines+593>:       jl     0x5d3ce9 <grid_del_lines+633>
   0x5d3cc3 <grid_del_lines+595>:       cmp    $0x1,%ebp
   0x5d3cc6 <grid_del_lines+598>:       jne    0x5d3fb0 <grid_del_lines+1344>
   0x5d3ccc <grid_del_lines+604>:       nopl   0x0(%rax)
   0x5d3cd0 <grid_del_lines+608>:       movslq %ebx,%rdx
   0x5d3cd3 <grid_del_lines+611>:       lea    (%rsi,%rdx,8),%rdx
   0x5d3cd7 <grid_del_lines+615>:       mov    (%rdx),%rcx
   0x5d3cda <grid_del_lines+618>:       mov    %rcx,(%rsi,%rax,8)
   0x5d3cde <grid_del_lines+622>:       movslq %ebx,%rax
   0x5d3ce1 <grid_del_lines+625>:       add    $0x1,%ebx

There has been at least 2 different cases where this function has had crashes in the past maybe #20316 and #10031, but they are both pre 0.10.0.

There is one other open issue #22278 that has grid_del_lines in the stacktrace, but is not the same issue as it's not the crashing function.

Steps to reproduce

I'm running busted --run unit with the following setup:

❯ bat -p .busted
return {
    _all = {
        lua = './test/nvim-shim.sh'
    },
    unit = {
        ROOT = {'./test/unit/'},
    },
}

❯ bat -p ./test/nvim-shim.sh
#!/usr/bin/env bash
set -euo pipefail

export XDG_CONFIG_HOME='test/xdg/config/'
export XDG_STATE_HOME='test/xdg/local/state/'
export XDG_DATA_HOME='test/xdg/local/share/'

mkdir -p ${XDG_CONFIG_HOME} ${XDG_STATE_HOME} ${XDG_DATA_HOME}
mkdir -p ${XDG_DATA_HOME}/nvim/site/pack/testing/start || true
# XXX - Expects to be in cursorless.nvim, so we should force cd here
ln -sf "${PWD}" ${XDG_DATA_HOME}/nvim/site/pack/testing/start/cursorless.nvim

NVIM='nvim'
if type -a ${NVIM} | grep 'is an alias'; then
  NVIM=$(type -p nvim)
fi
# shellcheck disable=SC2068
echo $@
${NVIM} --cmd 'set loadplugins' -l $@
exit_code=$?
rm ${XDG_DATA_HOME}/nvim/site/pack/testing/start/cursorless.nvim 2>/dev/null || true

exit $exit_code

❯ bat -p test/unit/buffer_selection_spec.lua
describe("cursorless buffer selection", function()
  describe("Basic selection", function()
    before_each(function()
      local pos = vim.api.nvim_win_get_cursor(0)[2]
      local line = vim.api.nvim_get_current_line()
      local nline = line:sub(0, pos) .. "hello" .. line:sub(pos + 1)
      vim.api.nvim_set_current_line(nline)
    end)

    it("Can read visible lines", function()
      print("foo")
      assert.is_true(true)
    end)
  end)
end)

If I have print("foo") in the above buffer_selection_spec.lua file:

❯ busted --run unit
/nix/store/w1wjplk0znvid1y1kljz1a1nqm2dc24q-luajit2.1-busted-2.2.0-1/busted-2.2.0-1-rocks/busted/2.2.0-1/bin/busted --ignore-lua --run unit
./test/nvim-shim.sh: line 19: 1640715 Segmentation fault      (core dumped) ${NVIM} --cmd 'set loadplugins' -l $@
error: Recipe `test` failed on line 5 with exit code 1

If I comment out the print:

❯ busted --run unit
/nix/store/w1wjplk0znvid1y1kljz1a1nqm2dc24q-luajit2.1-busted-2.2.0-1/busted-2.2.0-1-rocks/busted/2.2.0-1/bin/busted --ignore-lua --run unit
●
1 success / 0 failures / 0 errors / 0 pending : 0.000675 seconds

Expected behavior

Don't crash.

Neovim version (nvim -v)

❯ /home/aa/.nix-profile/bin/nvim --version NVIM v0.10.0 Build type: Release LuaJIT 2.1.1693350652

Vim (not Nvim) behaves the same?

don't know, not sure how to reproduce given my repro relies on lua test

Operating system/version

nixos 24.05

Terminal name/version

wezterm

$TERM environment variable

xterm-256color

Installation

nixpkgs

[zeertzjq]

Can you reproduce this using a Debug build and provide a backtrace with line numbers?

[zeertzjq]

I cannot reproduce this crash with Nvim 0.10.0 and busted 2.2.0 on Arch Linux:

/usr/lib/luarocks/rocks-5.4/busted/2.2.0-1/bin/busted --ignore-lua --run unit
foo●
1 success / 0 failures / 0 errors / 0 pending : 0.000427 seconds

[fidgetingbits]

Interestingly I can't reproduce it with a debug version of 0.10.0 built from nixpkgs unstable:

busted/2.2.0-1/bin/busted --ignore-lua --run unit
foo●
1 success / 0 failures / 0 errors / 0 pending : 0.000478 seconds

I tested with neovim from 24.05, which is 0.9.5, and it also doesn't repro.

Then I remembered the neovim binary I use on my system is built using nixvim, so is actually passing in additional commands in a wrapper, which is including my whole init script (which I was trying to avoid with the busted tests, so happy this bug made me realize lol):

the wrapper is adding:

--cmd \"lua vim.g.node_host_prog='/nix/store/n94b5rwrmhimk467ijgpzncdck48bk2z-neovim-0.10.0/bin/nvim-node';vim.g.loaded_perl_provider=0;vim.g.loaded_python_provider=0;vim.g.python3_host_prog='/nix/store/n94b5rwrmhimk467ijgpzncdck48bk2z-neovim-0.10.0/bin/nvim-python3';vim.g.ruby_host_prog='/nix/store/n94b5rwrmhimk467ijgpzncdck48bk2z-neovim-0.10.0/bin/nvim-ruby'\" --cmd \"set packpath^=/nix/store/2ks1w6jpw6fxdpca3a5yyxq291cx54fc-vim-pack-dir\" --cmd \"set rtp^=/nix/store/2ks1w6jpw6fxdpca3a5yyxq291cx54fc-vim-pack-dir\" -u /nix/store/ywydszddljnsg6s48vhdlfpf5nj80a5h-init.lua"

So that's gonna make a simple repro a bit harder, but I'll take a look afterwards.

(gdb) bt
#0  0x00005556dc1d212f in grid_del_lines (grid=grid@entry=0x5556dc491228 <msg_grid>, row=row@entry=0, line_count=line_count@entry=1, end=0, col=col@entry=0, width=0) at /build/source/src/nvim/grid.c:1056
#1  0x00005556dc24dddb in msg_scroll_up (may_throttle=<optimized out>, zerocmd=<optimized out>) at /build/source/src/nvim/message.c:2364
#2  0x00005556dc24b170 in msg_start () at /build/source/src/nvim/message.c:1447
#3  0x00005556dc24a912 in msg_attr_keep (s=s@entry=0x5556dd5f0b90 "Error detected while processing /nix/store/ywydszddljnsg6s48vhdlfpf5nj80a5h-init.lua:", attr=attr@entry=1, keep=false, multiline=false)
    at /build/source/src/nvim/message.c:341
#4  0x00005556dc24bafd in msg (s=0x5556dd5f0b90 "Error detected while processing /nix/store/ywydszddljnsg6s48vhdlfpf5nj80a5h-init.lua:", attr=1) at /build/source/src/nvim/message.c:246
#5  msg_source (attr=attr@entry=1) at /build/source/src/nvim/message.c:606
#6  0x00005556dc24c20b in emsg_multiline (
    s=s@entry=0x5556dc4a3d10 <semsg_multiline[errbuf]> "E5113: Error while calling lua chunk: vim/_editor.lua:0: /nix/store/ywydszddljnsg6s48vhdlfpf5nj80a5h-init.lua..nvim_exec2() called at /nix/store/ywydszddljnsg6s48vhdlfpf
5nj80a5h-init.lua:0: Vim(colors"..., multiline=<optimized out>) at /build/source/src/nvim/message.c:752
#7  0x00005556dc24c81f in semsg_multiline (fmt=fmt@entry=0x5556dc3b05a5 "E5113: Error while calling lua chunk: %.*s") at /build/source/src/nvim/message.c:807
#8  0x00005556dc20e8f1 in nlua_error (lstate=lstate@entry=0x7fd586d5f380, msg=0x5556dc3b05a5 "E5113: Error while calling lua chunk: %.*s") at /build/source/src/nvim/lua/executor.c:156
#9  0x00005556dc20f3ef in nlua_exec_file (path=path@entry=0x5556dd5e6ae0 "/nix/store/ywydszddljnsg6s48vhdlfpf5nj80a5h-init.lua") at /build/source/src/nvim/lua/executor.c:1854
#10 0x00005556dc2e3ad4 in do_source (fname=fname@entry=0x7ffd2d12f930 "/nix/store/ywydszddljnsg6s48vhdlfpf5nj80a5h-init.lua", check_other=check_other@entry=0, is_vimrc=is_vimrc@entry=0, ret_sid=ret_sid@entry=0x0)
    at /build/source/src/nvim/runtime.c:2223
#11 0x00005556dc21c19a in source_startup_scripts (parmp=0x7ffd2d12d7f8) at /build/source/src/nvim/main.c:2098
#12 main (argc=<optimized out>, argv=<optimized out>) at /build/source/src/nvim/main.c:465
(gdb) x/i $pc
=> 0x5556dc1d212f <grid_del_lines+287>: mov    (%rax,%rcx,8),%r15d
(gdb) i r
rax            0x0                 0
rbx            0x0                 0
rcx            0x0                 0
rdx            0x0                 0
rsi            0x0                 0
rdi            0xffffffff          4294967295
rbp            0x0                 0x0
rsp            0x7ffd2d12d1a0      0x7ffd2d12d1a0
r8             0x0                 0
r9             0x0                 0
r10            0x7                 7
r11            0x5556dd5e2720      93831569483552
r12            0x5556dc491228      93831551324712
r13            0x1                 1
r14            0x0                 0
r15            0x5556dc49e2e4      93831551378148
rip            0x5556dc1d212f      0x5556dc1d212f <grid_del_lines+287>
eflags         0x10246             [ PF ZF IF RF ]
cs             0x33                51
ss             0x2b                43
ds             0x0                 0
es             0x0                 0
fs             0x0                 0
gs             0x0                 0
fs_base        0x7fd586d88b80      140555067100032
gs_base        0x0  
(gdb) p/x *grid
$2 = {handle = 0x3, chars = 0x0, attrs = 0x0, vcols = 0x0, line_offset = 0x0, dirty_col = 0x0, rows = 0x0, cols = 0x0, valid = 0x0, throttled = 0x0, row_offset = 0x0, col_offset = 0x0, target = 0x0, blending = 0x0, 
  focusable = 0x1, zindex = 0x0, comp_row = 0x0, comp_col = 0x0, comp_width = 0x0, comp_height = 0x0, comp_index = 0x0, comp_disabled = 0x0}
(gdb) list
1051          j -= line_count;
1052          grid_clear_line(grid, grid->line_offset[j] + (size_t)col, width, false);
1053        } else {
1054          // whole width, moving the line pointers is faster
1055          j = row + i;
1056          temp = (unsigned)grid->line_offset[j];
1057          while ((j += line_count) <= end - 1) {
1058            grid->line_offset[j - line_count] = grid->line_offset[j];
1059          }
1060          grid->line_offset[j - line_count] = temp;

Crash is on temp = (unsigned)grid->line_offset[j];, which the above shows grid->line_offset is NULL.

[zeertzjq]

Hmm, I think this is as simple as this:

printf 'vim.o.cmdheight = 0\nprint("foo")\n' | nvim --clean -l -

[fidgetingbits]

Yep that does it.