2.7.0: incorrect color with terminal background image

[i0ntempest]

In version 2.7.0 with Apple Terminal, background color seems to be broken, which can be seen if background image or transparency is enabled:

Debug log:

Please post the following report at <https://github.com/walles/moor/issues>,
or e-mail it to johan.walles@gmail.com.

Version      : 2.7.0
LANG         : en_US.UTF-8
TERM         : xterm-256color
MOOR         : 
EDITOR       : /opt/local/bin/nano
TERM_PROGRAM : Apple_Terminal

LESS_TERMCAP_md : ESC[1;31m
LESS_TERMCAP_us : ESC[1;4;32m
LESS_TERMCAP_so : ESC[01;33m

GOOS    : darwin
GOARCH  : arm64
Compiler: gc
NumCPU  : 16

Stdin  is a terminal: true
Stdout is a terminal: true

time="Oct 25 20:07:08.051573" level=debug msg="File is assumed to be uncompressed: /Users/Admin/Documents/Scripts/auto-updatedb.sh"
time="Oct 25 20:07:08.054620" level=debug msg="File is assumed to be uncompressed: /Users/Admin/Documents/Scripts/auto-updatedb.sh"
time="Oct 25 20:07:08.054629" level=info msg="ttyin terminal state: &{state:{termios:{Iflag:26624 Oflag:2 Cflag:19200 Lflag:536870979 Cc:[4 255 255 127 23 21 18 255 3 28 26 25 17 19 22 15 1 0 20 255] Ispeed:9600 Ospeed:9600}}}"
time="Oct 25 20:07:08.054634" level=info msg="ttyout terminal state: &{state:{termios:{Iflag:26624 Oflag:2 Cflag:19200 Lflag:536870979 Cc:[4 255 255 127 23 21 18 255 3 28 26 25 17 19 22 15 1 0 20 255] Ispeed:9600 Ospeed:9600}}}"
time="Oct 25 20:07:08.054637" level=debug msg="Counted 13 lines in 7.375µs at 567ns/line"
time="Oct 25 20:07:08.054649" level=info msg="No known terminal with arrow keys emulation detected, assuming mouse tracking is needed"
time="Oct 25 20:07:08.054657" level=info msg="Stream read in 14.541µs, have 13 lines"
time="Oct 25 20:07:08.054674" level=info msg="Entering Twin main loop..."
time="Oct 25 20:07:08.055056" level=debug msg="Terminal background color detected as #808080 after 397.625µs"
time="Oct 25 20:07:08.060302" level=debug msg="Using style <tango>"
time="Oct 25 20:07:08.060306" level=info msg="Pager starting"
time="Oct 25 20:07:08.060453" level=info msg="Entering pager main loop..."
time="Oct 25 20:07:08.061599" level=debug msg="highlightFromMemory() took 6.937291ms"
time="Oct 25 20:07:08.061602" level=debug msg="Tailing file /Users/Admin/Documents/Scripts/auto-updatedb.sh"
time="Oct 25 20:07:23.861751" level=info msg="ttyin read error, twin giving up: EOF"

Apparantly everything between 2.1.1 and 2.7 is broken. 2.1.1 works fine:

Debug log:

Please post the following report at <https://github.com/walles/moor/issues>,
or e-mail it to johan.walles@gmail.com.

Version      : 2.1.1
LANG         : en_US.UTF-8
TERM         : xterm-256color
MOOR         : 
EDITOR       : /opt/local/bin/nano
TERM_PROGRAM : Apple_Terminal

GOOS    : darwin
GOARCH  : arm64
Compiler: gc
NumCPU  : 16

Stdin  is a terminal: true
Stdout is a terminal: true

time="Oct 25 20:08:39.968928" level=debug msg="File is assumed to be uncompressed: /Users/Admin/Documents/Scripts/auto-updatedb.sh"
time="Oct 25 20:08:39.971657" level=debug msg="File is assumed to be uncompressed: /Users/Admin/Documents/Scripts/auto-updatedb.sh"
time="Oct 25 20:08:39.971668" level=debug msg="Counted 13 lines in 3.292µs at 253ns/line"
time="Oct 25 20:08:39.971674" level=info msg="Stream read in 3.375µs, have 13 lines"
time="Oct 25 20:08:39.971776" level=info msg="ttyin terminal state: &{state:{termios:{Iflag:26624 Oflag:2 Cflag:19200 Lflag:536870979 Cc:[4 255 255 127 23 21 18 255 3 28 26 25 17 19 22 15 1 0 20 255] Ispeed:9600 Ospeed:9600}}}"
time="Oct 25 20:08:39.971782" level=info msg="ttyout terminal state: &{state:{termios:{Iflag:26624 Oflag:2 Cflag:19200 Lflag:536870979 Cc:[4 255 255 127 23 21 18 255 3 28 26 25 17 19 22 15 1 0 20 255] Ispeed:9600 Ospeed:9600}}}"
time="Oct 25 20:08:39.971799" level=info msg="No known terminal with arrow keys emulation detected, assuming mouse tracking is needed"
time="Oct 25 20:08:39.971854" level=info msg="Entering Twin main loop..."
time="Oct 25 20:08:39.972390" level=debug msg="Terminal background color detected as #808080 after 568.041µs"
time="Oct 25 20:08:39.972394" level=debug msg="Using style <tango>"
time="Oct 25 20:08:39.972396" level=info msg="Pager starting"
time="Oct 25 20:08:39.972415" level=info msg="Entering pager main loop..."
time="Oct 25 20:08:39.973374" level=debug msg="highlightFromMemory() took 1.69775ms"
time="Oct 25 20:08:39.973377" level=debug msg="Tailing file /Users/Admin/Documents/Scripts/auto-updatedb.sh"

[walles]

Workaround

Set your MOOR environment variable to --terminal-fg.

Fix

moor needs to be able to tell that the terminal has a background image, and I don't know if there's a way to query for that.

If you have any ideas I'm all ears.

One thing that's happening though is that Apple's Terminal claims to have a background color:

time="Oct 25 20:08:39.972390" level=debug msg="Terminal background color detected as #808080 after 568.041µs"

Since there is no background color (but an image), this response from the terminal is not really true.

AFAIU, what the terminal should be doing is to not respond at all to moor's background query (since there is no background color).

Which would enable moor to adapt.

So maybe this needs to be brought up with Apple?

Notes

Just tested with iTerm2 and it works the same way.

[walles]

Posted a question here: https://stackoverflow.com/questions/79799431/how-to-determine-that-the-terminal-has-a-background-image-not-just-color

[i0ntempest]

Thanks, with the workaround moor is now usable. But there's still a few spots with incorrect background:
Section headers when using moor as MANPAGER:

And line numbers:

Indeed seems like an Apple problem. Apple Terminal has some limitations compared to other third party terminals, I'm kinda considering switching just haven't made my mind yet.

[walles]

Did you set your own section headers using LESS_TERMCAP_* or are these from moor?

You can do moor --help to find out.

If you set them yourself you should consider not setting a background for them.

[i0ntempest]

Those are from my .profile:

export LESS_TERMCAP_mb=$'\e[1;31m'
export LESS_TERMCAP_md=$'\e[1;31m'
export LESS_TERMCAP_me=$'\e[0m'
export LESS_TERMCAP_se=$'\e[0m'
export LESS_TERMCAP_so=$'\e[01;33m'
export LESS_TERMCAP_ue=$'\e[0m'
export LESS_TERMCAP_us=$'\e[1;4;32m'

I don't think any of these sets background. moor --help gives:

And if I remove them man page would look like this:

[walles]

I just released v2.7.1 with updates to --terminal-fg, so your man pages should now look good with that option:
https://github.com/walles/moor/releases/

On an unrelated note, moor automatically hides line numbers for man pages (and git output), so no need for --no-linenumbers in your MANPAGER environment variable. Also, man pages come pre wrapped, so I don't think --wrap has much effect in there.

You could consider dropping MANPAGER and setting these instead:

PAGER=moor
MOOR=--wrap

Also maybe consider --reformat for automatic JSON reformatting if that's something you're interested in, I find it super nice!

[i0ntempest]

Yup, man pages now look good.
As for the line number though, I'm not sure why but they aren't auromatically removed when using in man pages so I have to use that arg.

[walles]

As for the line number though, I'm not sure why but they aren't auromatically removed when using in man pages

Strange, I wonder why this code wouldn't work in your case?

moor/cmd/moor/moor.go

Lines 330 to 362 in 83d6d54

	// On man pages, disable line numbers by default.
	//
	// Before paging, "man" first checks the terminal width and formats the man page
	// to fit that width.
	//
	// Then, if moor adds line numbers, the rightmost part of the man page won't be
	// visible.
	//
	// So we try to detect showing man pages, and in that case disable line numbers
	// so that the rightmost part of the page is visible by default.
	func noLineNumbersDefault() bool {
	if os.Getenv("MANPATH") != "" {
	// Set by "man" on macOS, skip line numbers in this case
	return true
	}
	if os.Getenv("MAN_PN") != "" {
	// Set by "man" on Ubuntu 22.04.4 when I tested it inside of Docker,
	// skip line numbers in this case
	return true
	}
	if os.Getenv("GIT_EXEC_PATH") != "" {
	// Set by "git".
	// Neither logs nor diffs are helped by line numbers, turn them off by
	// default.
	return true
	}
	// Default to not skipping line numbers
	return false
	}

[i0ntempest]

Not sure, probably because I'm using mandoc from MacPorts

[walles]

Not sure, probably because I'm using mandoc from MacPorts

I want to detect that thing if it's easy enough.

Can you try doing this and sending me any lines with MAN in them?

Note that piping through grep won't work, you'll have to copy the result from the terminal and filter it afterwards.

PAGER=env MANPAGER=env man man