Skip to content

NetBSD

Jump to bottom Edit New page
Henryk Paluch edited this page Aug 29, 2024 · 50 revisions

NetBSD notes

Here are random notes on NetBSD OS - tested 9.3/amd64

NetBSD is BSD Unix where main priority is portability. To avoid delaying release on popular platforms caused by less active platforms - they use tiered release management:

Unique NetBSD Features:

You can see tests of RUMP kernel in action on CVS web:

Also NetBSD 9.3 seems to be most in par with Linux and its nouveau(4) drm/kms driver. My card is GT 218 made by Zotac (nice passive card without fan!). Here is info:

dmesg -t | fgrep GT2 | tail -1

nouveau0: info: NVIDIA GT218 (0a8280b1)

Also note that on PCI it is identified as GT210 card:

pcictl pci0 list | fgrep VGA

006:00:0: NVIDIA GeForce 210 (VGA display, revision 0xa2)

But acceleration does not work under 9.3-stable. However 10-Beta is fine (see text at the end of article)

In other BSDs the situation is not good:

  • on FreeBSD there is proprietary nvidia driver support (so it includes even 3D acceleration) - see https://wiki.freebsd.org/Graphics#NVIDIA_graphics. But nVidia dropped GT218 card support from versions above 340. So it is only matter of time, when this driver will stop working with latest Xorg servers. This is another reminder why closed-source proprietary drivers are awful - they may drop your card support any time and you can do nothing about it.
  • on OpenBSD there is similar driver called nv see https://man.openbsd.org/nv.4, but it is somehow extremely slow (when moving Window - which is clearly 2D operation, so it should be fast). Same as reported here: https://marc.info/?l=openbsd-misc&m=120982696625080&w=2 on this thread from 2008, but not resolution. I need to retest it under different Window manager (old Window managers seems to work much better - possible that Xfce uses 3D API even for Window movement).

Well some recommend Radeon from ATI (now AMD), but I remember very well that around 2007 even Radeon used binary blobs under Linux (and it broke badly with every new Linux kernel release)

Another highlights:

  • NetBSD 10RC1 installs perfectly under Hyper-V 2012R2 in UEFI (Generation 2) mode - please see NetBSD UEFI for details.

Issues:

  • under KVM, you have to use old i440fx machine type. When new Q35 is used, NetBSD will be unable to find any kind of virtual disk (both VirtIO BLK and/or VirtIO SCSI).

  • tar hangs up when trying to backup Floppy device under /dev - because kernel attempts to access it. Workaround is to insert formatted floppy in driver while tar is running... This problem is observed on both NetBSD 9.3 and 10 RC3.

    Later (PR 55815) is interesting reading - there is a bit heated debate what is right and what is wrong...

  • CVS "stable" branch is not what you may think: It is NOT branch where only security and bugfixes are imported. Actually there ARE all these three: security fixes, bug fixes and "minor features". If you install world from latest "stable" branch you will get this warning (from /usr/src/etc/motd.stable):

    This system is running a development snapshot of a stable branch of the NetBSD operating system, which will eventually lead to a new formal release. This snapshot may contain bugs or other unresolved issues and is not yet considered release quality. Please bear this in mind and use the system with care.

    Please see http://www.netbsd.org/releases/release-map.html

    However when there are security fixes you have to use "stable" branch (there is no minor patch branch anymore despite above release-map) and follow instructions how to cherry-pick only fixes and build install them. See for example, kernel security fix: https://ftp.netbsd.org/pub/NetBSD/security/advisories/NetBSD-SA2022-004.txt.asc

Problems with NetBSD 10 RC3

I experience several problems with NetBSD 10 RC3:

  1. panic right on reboot (need serial cable to log it)
  2. tar hangs up when backing up floppy devices (kernel somehow starts accessing floppy and blocks tar and later timeouts)
  3. DRM nouveau no longer works on my NVidia GT 218 (some weird error)

I will try to collect more information when available...

Verifying installation sets

I don't know why but there is no handy command to verify integrity of installed files (like FreeBSD's freebsd-update IDS command). Althouth there are some checks when cron or you run sh /etc/security they are not verifying installation sets in /etc/mtree/set.*.

Thus I wrote simple shell script check-sets.sh with contents:

!/bin/sh
set -eu
err_count=0

for i in /etc/mtree/set.*
do
        echo "Checking set '$i' ..."
        mtree -e -p / -f $i && echo OK || {
                echo "FAILED" >&2
                err_count=$(( err_count + 1 ))
        }
done

echo "$err_count sets failed."
if [ $err_count -eq  0 ]
then
        echo "OK: all sets validated successfully"
        exit 0
else
        echo "ERROR: Total $err_count set validation failed" >&2
        exit 1
fi

And run it. What is quite disturbing that it found one unexpected error:

/root/check-sets.sh

hecking set '/etc/mtree/set.base' ...
usr/lib/i386/libkadm5srv.so.15.0: 
        sha256 (0xa6bc0eba3aa08db46f668ad10e6002feb62273b23ce020c86ec8b8f394680d2d, 0x5a99994d8af8c44d6a558267ea1a0e0fe3906b0bc3902be1656272b4347f5399)
./var/run/named missing
FAILED

Trying manually:

sum -a SHA256 /usr/lib/i386/libkadm5srv.so.15.0

SHA256 (/usr/lib/i386/libkadm5srv.so.15.0) = 5a99994d8af8c44d6a558267ea1a0e0fe3906b0bc3902be1656272b4347f5399

Script to reproduce:

cd
curl -fLO http://cdn.netbsd.org/pub/NetBSD/NetBSD-9.3/amd64/binary/sets/base.tar.xz
curl -fLO http://cdn.netbsd.org/pub/NetBSD/NetBSD-9.3/amd64/binary/sets/MD5
grep -w base.tar.xz MD5 > MD5.base
# content of MD5.base: MD5 (base.tar.xz) = 1ca2d3cc3d057eb6880da34a2b8012dc
md5 -c MD5.base
mkdir -p unpack
doas tar xpf base.tar.xz -C unpack
cd unpack && doas mtree -e -f etc/mtree/set.base

So 1st hash is expected and 2nd is real.

Commands for Linux users

History command in Bourne shell:

# verify that you are using Bourne shell (/bin/sh)
$ echo $SHELL

/bin/sh

$ fc -l

# will dump your command history

When command-line editing does not work:

  • when your Bourne shell does not support command-line editing it is because it was not enabled in /etc/shrc - it normally works for login-shell only (shell started with -)

  • in such case you need to invoke manually:

    # enables command-line editing
set -o emacs
# enables bash-like tab expansion
set -o tabcomplete

  • or become root by appending -S to doas, for example:

    doas -S

Listing pci devices (like lspci in Linux):

# must be root or in group 'wheel' to access /dev/pci
pcictl pci0 list

Listing process tree: ps axd

To see systat(1) like in OpenBSD you have to use vmstat argument, i.e.:

systat vmstat
  • Note that systat has vim like commands, for example :drives will show list of displayed drives. :drives cd0 wd1 will replace list of displayed drives with cd0 and wd1

Or you can try systat pigs (what name?) to see CPU intensive processes...

For top command I recommend -c (show command args, same as under Linux) and -1 - show perc/cpu utilization.

Tiny fonts on console

Unfortunately NetBSD imported Linux DRM that overtakes console and sets ridiculously high console resolution (sic!).

You have to options:

  1. Extract and load larger fonts: https://www.unitedbsd.com/d/311-wsfont-changing-the-default-netbsd-console-font
  2. disable DRM to keep Text console (if you don't need X11)

Solution 1: tested but you have to extract and build 4 files from above web page. But probably best solution...

Solution 2: disabling DRM:

The solution is described here:

In my case DRM is provided as nouveau* (NVIDIA Graphics) - to disable it append to /boot.cfg:

# Disable Direct Rendering Manager (DRM) drivers
userconf=disable i915drmkms*
userconf=disable nouveau*
userconf=disable radeon*

This should do the trick.

Side-effect: Recent X server (that depends on DRM) will not start.

Please also see:

Security

I disable both Postfix and Inetd. Append to /etc/rc.conf:

postfix=NO
inetd=NO

And restart NetBSD using init 6

Custom firewall - quickstart:

  • WARNING! In case of NetBSD 10+ (tested Beta) you have to rename all blacklist* to blocklist* (which is semantically nonsense, if someone insists on renaming it should be denylist).

  • first copy this configuration (blacklistd is used from example npf.conf):

    cp /usr/share/examples/blacklist/blacklistd.conf /etc/

  • now I used /usr/share/examples/npf/host-npf.conf as template for /etc/npf.conf

  • here is my copy of /etc/npf.conf - you will need at least change network card name (nfe0) on both lines:

    $wired_if = "nfe0"
$wired_addrs= ifaddrs(nfe0)

alg "icmp"

procedure "log" {
    # Send log events to npflog0, see npfd(8)
    log: npflog0
}

group "wired" on $wired_if {
    # Placeholder for blacklistd (configuration separate) to add blocked hosts
    ruleset "blacklistd"

    # Allow SSH on wired interface and log all connection attempts
    #pass stateful in on $wired_if proto tcp to $wired_addrs port ssh apply "log"
    # Skip logging
    pass stateful in on $wired_if proto tcp to $wired_addrs port ssh
}

group default {
    # Default deny, otherwise last matching rule wins
    block all apply "log"

    # Don't block loopback
    pass on lo0 all

    # Allow incoming DHCP server responses
    pass in family inet4 proto udp from any port bootps to any port bootpc
    pass in family inet6 proto udp from any to any port "dhcpv6-client"

    # Allow IPv6 ICMP
    pass family inet6 proto ipv6-icmp all

    # Allow incoming IPv4 pings
    pass in family inet4 proto icmp icmp-type echo all

    # Allow being tracerouted
    pass in proto udp to any port 33434-33600

    # Allow incoming mDNS traffic from neighbours
    pass in proto udp to any port mdns

    # Allow all outbound traffic
    pass stateful out all
}

  • now validate your firewall configuration

    npfctl validate

  • now you need to append 3 lines at the end of /etc/rc.conf:

    # NPF firewall
npf=YES
npfd=YES
blacklistd=YES

  • and finally reboot

  • I had small problem that on reboot I see dhcp6_sendmessage: IFNAME0: sendmsg: Network is unreachable

  • but it seems to proceed - interface gets both IPv4 and IPV6 address(es) assigned

  • to see firewall log use command like this (from man npfd):

    doas tcpdump -n -e -tttt -r /var/log/npflog0.pcap

  • to watch firewall log in real-time use command like this (from man npfd):

    doas tcpdump -n -e -ttt -i npflog0

Custom hostname

Default hostname is localhost. To change it to netbsd use this command:

echo netbsd > /etc/myname

And reboot using reboot command. This file is picked up in /etc/rc.d/network startup script.

DHCP client is sensible and it overwrites local hostname from DHCP server only if it is not set or if it is localhost.

Installing binary packages

At first copy and paste this uncommented line from /root/.profile (as root):

export PKG_PATH="http://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/$(uname -p)/$(uname -r|cut -f '1 2' -d.|cut -f 1 -d_)/All"

Above line will use 9 as base, I prefer to use 9.3 as base using:

# uname -r returns '9.3_STABLE' and we change it with sed to '9.3'
export PKG_PATH="http://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/$(uname -p)/$(uname -r|sed 's/_.*//')/All"

Then install pkgin using:

pkg_add pkgin

WARNING! Above PKG_PATH variable is used only for pkg_add command. However pkgin commands below use repository path in /usr/pkg/etc/pkgin/repositories.conf To have latest packages for NetBSD 9.3 I decided to change

  • from: https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/$arch/9.0/All
  • to: https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/$arch/9.3/All

And run:

# update package database
pkgin up
# if you want to upgrade existing packages try:
pkgin ug

Then you can use these commands to search and install vim:

pkgin se vim
pkgin in vim

Typically I install those packages using pkgin install ...:

  • curl
  • doas (sudo alternative). After installation create /usr/pkg/etc/doas.conf with contents: permit nopass :wheel this will permit group wheel to become root without password
  • git
  • lynx best text-only browser without JS nonsense. BTW: while google.com refuses to server Firefox with disabled JS, it will happily serve Lynx without JS(!) at all.
  • mc Midnight Commander
  • mercurial alternative to Git - can be used to fetch NetBSD repository using command hg
  • vim best editor ever

NOTE: tmux - is ALREADY INCLUDED in base system - virtual terminal - you can detach any time (using Ctrl-b and d) and attach even from multiple clients at same time (using tmux attach command). Often I run one tmux on local console and another one remotely - so it is easily visible what is going on.

Also we have to install trusted CA list (empty after NetBSD install):

# as root
pkgin install mozilla-rootcerts
# this one is needed for openssl apps - for example lynx
pkgin install mozilla-rootcerts-openssl

If you want to be scared you can try:

pkg_admin fetch-pkg-vulnerabilities
pkg_admin audit

Building system using sysbuid/sysupgrade

Removed - I prefer to use /usr/src/build.sh directly.

Keeping system up-to-date

Expecting that you have installed doas package (sudo replacement)

Optional: speedup with journaling:

doas mount -u -o log,noatime /

Note that there are too distinct guides in NetBSD manual:

  1. Chapter 33. Crosscompiling NetBSD with build.sh - when binaries will be installed to other target machine.
  2. Chapter 35. Updating an existing system from sources - to update system where we build.

We will follow later option.

I will start with stable tarball (it is updated daily and it is fastest option): Notice that it is under NetBSD-release-10 dir - which is confusing, one would expect NetBSD-stable-10...

r=`uname -r |  awk -F . '{ print $1 }'`
echo $r
# 10
mkdir ~/$r-stable
cd ~/$r-stable
curl -fLO http://cdn.netbsd.org/pub/NetBSD/NetBSD-release-$r/tar_files/src.tar.gz
curl -fLO http://cdn.netbsd.org/pub/NetBSD/NetBSD-release-$r/tar_files/src.tar.gz.MD5
# only if plan build/update X-Window:
curl -fLO http://cdn.netbsd.org/pub/NetBSD/NetBSD-release-$r/tar_files/xsrc.tar.gz
curl -fLO http://cdn.netbsd.org/pub/NetBSD/NetBSD-release-$r/tar_files/xsrc.tar.gz.MD5

NOTE: If your transfer crashes you can add -C - to tell curl to continue download, for example:

curl -fLO -C - http://cdn.netbsd.org/pub/NetBSD/NetBSD-release-10/tar_files/xsrc.tar.gz

To check checksums we need to make little dirty trick (fake tar_files directory):

ln -s . tar_files
md5 -c src.tar.gz.MD5
md5 -c xsrc.tar.gz.MD5

Now I will use /usr/src and /usr/xsrc as non privileged user:

doas rm -rf /usr/src
doas tar xpzf src.tar.gz -C /usr
doas chown -R $USER:wsrc /usr/src
# same for X-Window
doas rm -rf /usr/xsrc
doas tar xpzf xsrc.tar.gz -C /usr
doas chown -R $USER:wsrc /usr/xsrc
# not documented but needed
doas mkdir /usr/obj /usr/tools
doas chown $USER:wsrc /usr/obj /usr/tools

If you want to update existing /usr/src tree on exiting branch (called "sticky tag") using CVS you can follow 32.4.2. Fetching a NetBSD stable branch using:

cd /usr/src
cvs -z3 update -dP
# and for X
cd /usr/xsrc
cvs -z3 update -dP

To verify which branch you use you can do this:

cat /usr/src/CVS/Tag | sed 's/^T//'

netbsd-9

cat /usr/xsrc/CVS/Tag | sed 's/^T//'

netbsd-9

(CVS branch is implemented like RCS sticky tag)

To see what changed you need to have some TAG from history or use DATE (CVS does not have Commit-ID because each file has independent revision sequence):

cd /usr/src
cvs -z3 -q diff -u  -D 2022-11-12 -D today | \
  tee ~/netbsd-2022-11-12-`date '+%Y-%m-%d'`.diff

If you want to see what's going on CVS source of NetBSD you can watch:

Now we will follow guide:

  1. Build userland (if you have more than 2 cores you can adjust -j 2 parameter for your environment):

    ./build.sh -O ../obj -T ../tools -U -j 2 distribution

    It took 4 hours and 28 minutes on old 2 core machine. Mainly because also toolchain is build.

  2. Build kernel

    ./build.sh -O ../obj -T ../tools -U -j 2 kernel=GENERIC

    It took 22 minutes to build kernel on old 2 core machine.

  3. Install kernel

    doas mv /netbsd /netbsd.old
doas mv /usr/obj/sys/arch/amd64/compile/GENERIC/netbsd /
# ALERT! Booting new kernel
doas reboot

  4. install userland (binaries, libraries)

    # ALERT! Overwriting / system !
doas ./build.sh -O ../obj -T ../tools -U install=/

  5. updating configuration files

    doas /usr/sbin/postinstall -s /usr/src check
# ALERT! Updating configuration files including /etc/
doas /usr/sbin/postinstall -s /usr/src fix
doas /usr/sbin/etcupdate -s /usr/src

    WARNING! Ensure that these two lines stays in /etc/ttys!

    console "/usr/libexec/getty Pc"         wsvt25  off secure
constty "/usr/libexec/getty Pc"         wsvt25  on secure

    Do not allow to overwrite it with "vt100" - in such case full screen apps (mc) will be only black and white and all extended keys will stop working...

  6. final reboot

    reboot

Various commands

To list all listening processes on TCP and UDP, use this command:

$ sockstat -l46

USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
root     dhcpcd     325   10 24/58  *.*                   *.*
root     dhcpcd     325   11 udp6   *.dhcpv6-client       *.*
root     dhcpcd     325   12 udp    *.bootpc              *.*
root     sshd       500    3 tcp6   *.ssh                 *.*
root     sshd       500    4 tcp    *.ssh                 *.*

Cross-building from Linux

Experiment: cross-build NetBSD/amd64 on Debian 11 Linux, following Chapter 33. Crosscompiling NetBSD with build.sh

First we will try build without X-Window (X-Window requires 2 additional arguments: -X /path_to/xsrc and -x - enable building X11)

On your (Debian 11) Linux try:

sudo apt-get install curl cvs build-essential tmux zlib1g-dev
mkdir ~/netbsd-9-stable
cd ~/netbsd-9-stable
# takes around 2minutes on backbone (Azure Debian 11 VM in Germany)
# VM type: Standard D8s v3 (8 vcpus, 32 GiB memory)), OS Disk: Premium SSD LRS, 64GB
curl -fLO http://cdn.netbsd.org/pub/NetBSD/NetBSD-release-9/tar_files/src.tar.gz
tar xzf src.tar.gz
# unpacked src takes round 3GB
cd src
./build.sh list-arch | fgrep amd64
# recommended: run in tmux
# line below (toolchain build) took 7 min on 8-core, and needed 0.5GB under ~/obj
./build.sh -U -O ~/obj -j`nproc` -m amd64 -a x86_64 tools
# building system (userland) around 55 minutes, will create ~/obj/releasedir sets + boot.iso
# requires around total 8GB under ~/obj
./build.sh -U -u -j`nproc` -O ~/obj -m amd64 -a x86_64 release
# building USB pendrive IMG - it takes just around 5 minutes and it uses
# around 18GB space in ~/obj
./build.sh -U -u -j`nproc` -O ~/obj -m amd64 -a x86_64 install-image
# building ISO image:
./build.sh -U -u -j`nproc` -O ~/obj -m amd64 -a x86_64 iso-image

There should be now:

  • result of install-image target: USB pendrive image both unpacked and gzipped: 
    ~/obj/distrib/amd64/installimage-bios/NetBSD-9.3_STABLE-amd64-bios-install.img (1.8GB)
~/obj/distrib/amd64/installimage-bios/NetBSD-9.3_STABLE-amd64-bios-install.img.gz (290GB)
  • result of iso-image target: 
    ~/obj/distrib/amd64/cdroms/installcd/NetBSD-9.3_STABLE-amd64.iso (404MB)
# you can compress it and keep original (with -k) - will save around 33%
gzip -9vk NetBSD-9.3_STABLE-amd64.iso
~/obj/distrib/amd64/cdroms/installcd/NetBSD-9.3_STABLE-amd64.iso.gz (270MB)

GPT + BIOS details

I was very naive and expected that I can install all 3 BSDs (FreeBSD, NetBSD and OpenBSD) to single disk with GPT+BIOS partitioning. But I quickly found that it is not that way:

  • OpenBSD has very limited GPT support (actually only GPT+UEFI) - as of 2023

    • However we can boot it directly from Linux/GRUB2 - see OpenBSD

  • NetBSD uses its own mbr_bpt bootloader and is capable to seek and boot GPT NetBSD partition only

  • Linux GRUB uses its own way (dedicated "BIOS boot partition").

  • FreeBSD like Linux adds its own bootable partition (that depends on its own MBR bootloader)

How GPT+BIOS boot works, really useful:

The problem is that NetBSD's gpt_mbr expects:

  • only one "bootable" partition (more than 1 causes error)
  • which hat to have at offset 48decimal (ent_attr): 
    /* sys/arch/i386/stand/mbr/gpt_mbr  */
testb   $0x04,48(%di)
jz      not_this
  • which corresponds to: 
    // sys/sys/disklabel_gpt.h
#define GPT_ENT_ATTR_LEGACY_BIOS_BOOTABLE (1ULL << 2)
   /* legacy BIOS boot partition */

How Linux GRUB boots from GPT+BIOS (very different way!):

Here is when I installed FreeBSD on GPT+BIOS after NetBSD, but does not know yet, how to boot FreeBSD:

gpt show wd0

      start       size  index  contents
          0          1         PMBR
          1          1         Pri GPT header
          2         32         Pri GPT table
         34       2014         Unused
       2048  196608000      1  GPT part - NetBSD FFSv1/FFSv2
  196610048   16777216      2  GPT part - NetBSD swap
  213387264       1024      3  GPT part - 83bd6b9d-7f41-11dc-be0b-001560b84f0f
  213388288  209715200      4  GPT part - FreeBSD UFS/UFS2
  423103488   33554432      5  GPT part - FreeBSD swap
  456657920  520115215         Unused
  976773135         32         Sec GPT table
  976773167          1         Sec GPT header

Issues

Kernel boot freeze with 1TB SATA disk Seagate

NetBSD 9.3 does not work with 1TB Seagate (other disks on same SATA controller - nVidia MCP55 work fine). It just hangs with:

viaide2: BSY never cleared, status 0x80

NOTE: after 5 minutes or so, kernel will ignore this disk and continue boot. However without HDD it is not much use...

It seems that I'm not alone:

Also others posts (unclear which NetBSD versions are affected):

Latest news!

I think that I found fix! See

--- /root/sata_subr.c	2023-05-13 07:38:30.264874166 +0000
+++ /usr/src/sys/dev/ata/sata_subr.c	2023-05-13 07:40:24.075628047 +0000
@@ -93,8 +93,10 @@
 	 * the device. It doesn't hurt for other devices.
 	 */
 	bus_space_write_4(sata_t, scontrol_r, 0, 0);
-	scontrol = SControl_IPM_NONE | SControl_SPD_ANY | SControl_DET_INIT;
+	scontrol = SControl_IPM_NONE | SControl_SPD_G1 | SControl_DET_INIT;
 	bus_space_write_4(sata_t, scontrol_r, 0, scontrol);
+	aprint_normal("%s port %d: enforcing SATA1 speed at 1.5Gb/s\n",
+		    device_xname(chp->ch_atac->atac_dev), chp->ch_channel);
 
 	ata_delay(chp, 50, "sataup", flags);
 	scontrol &= ~SControl_DET_INIT;

And boot this kernel - it will force to use slowest SATA-1 speed 1.5Gb/s instead of auto-negotiation (SATA-II 3.0Gb/s in case of this HDD).

How to enter ddb -- in-kernel debugger

  • Good News! DDB in-kernel Debugger is enabled in GENERIC kernel! So just boot-up standard installation ISO: NetBSD-9.3-amd64.iso

  • or even better use boot -d to stop in DDB right after boot (see below). You can also increase message verbosity with -v or even more with -x

  • once it freezes on message BSY never cleared - our time has come. Here is output from null-modem cable connection (see text below):

    viaide2 at pci0 dev 5 function 1: NVIDIA MCP55 Serial ATA Controller (rev. 0xa2)
viaide2: using ioapic0 pin 21 for native-PCI interrupt
atabus3 at viaide2 channel 0
atabus4 at viaide2 channel 1
viaide2 port 1: device present, speed: 3.0Gb/s
viaide2: BSY never cleared, status 0x80

  • just press Ctrl-Alt-ESC to invoke DDB

  • you should see prompt db{0}>

  • now enter command show all procs

  • press SPACE to get next page. Notice our suspect called atarst:

    PID    LID S CPU     FLAGS       STRUCT LWP *               NAME WAIT
0       35 3   1       200   ffff88d0d467c1a0            atabus4 atarst

  • notice LWP number 35 and more important address: ffff88d0d467c1a0

  • to show stacktrace for this LWP you have to type trace/a ffff88d0d467c1a0

  • now you should see nice stacktrace like this:

    trace/a ffff88d0d467c1a0
trace: pid 0 lid 35 at 0xffffca80aead2dc0
sleepq_block() at netbsd:sleepq_block+0xbb
kpause() at netbsd:kpause+0x10b
__wdcwait_reset() at netbsd:__wdcwait_reset+0x162
wdcreset() at netbsd:wdcreset+0x7d
wdc_sataprobe() at netbsd:wdc_sataprobe+0x17a
atabusconfig() at netbsd:atabusconfig+0x65
atabus_thread() at netbsd:atabus_thread+0x6d

And this is so far all. I need to

So far we can train in building NetBSD kernel:

Using null-modem cable to capture console messages:

  • on Client (so far openSUSE LEAP 15.4) I installed

    sudo zypper in minicom
# give yourself access to USB CDC serial device:
sudo /usr/sbin/usermod -G dialout -a $USER
# relogin or run in current shell
newgrp dialout

  • running Client on RS-232 (using USB CDC<->RS232 + null-modem cable connected directly to COM1 port on Target (where NetBSD will boot)

    minicom -C logs/netbsd-`date '+%s'`.log -b 115200 -8 -D /dev/ttyUSB0
    • remember to use Ctrl-a followed by z (without holding Ctrl to get Minicom help.

  • now on Target: boot FreeBSD from CD this way:

  • at boot prompt select 3 to enter boot menu

  • press ? to see help.

  • now enter:

    consdev com0,115200

  • immediately your console should be redirected from VGA to 1st serial port (known as COM1 under DOS/Windows... and you should see output like:

    >> NetBSD/x86 BIOS Boot, Revision 5.11 (Thu Aug  4 15:30:37 UTC 2022) (from NetBSD 9.3)
>> Memory: 639/3111744 k

As first task we want to just grab all console messages until lockup, so enter:

dev cd0a:
boot

Waiting some time, until kernel freezes, then we exit minicom by pressing Ctrl-a followed by x (or followed by z for Help). Now we should have full kernel log in logs/netbsd-TIMESTAMP.log

Now we will try to use ddb(4) -- in-kernel debugger:

  • reset Target computer and again go to boot menu in NetBSD CD.

  • again start Minicom on Client using already shown command

  • again enter on Target: consdev com0,9600

  • and type (on Minicom) to break at debugger on kernel startup:

    boot -d

  • after a while you should see DDB prompt:

    Stopped in pid 0.1 (system) at  netbsd:breakpoint+0x5:  leave
db{0}>

  • now we will try to enable debug messages by modifying various kernel variables:

    write wdcdebug_atapi_mask ff
write atadebug_mask ff

  • and continue boot by issuing c and ENTER

  • we should see more detailed messages

  • trying again and dumping stuck code (-v is for verbose messages):

    boot -dv

  • in DDB issue again these 3 commands:

    write wdcdebug_atapi_mask ff
write atadebug_mask ff
c

  • after freeze on BSY never cleared issue Ctrl-f to send break and stop at DDB again:

    show procs

  • notice these lines:

    PID    LID S CPU     FLAGS       STRUCT LWP *               NAME WAIT
0       35 3   1       200   ffff88d0d467c1a0            atabus4 atarst

  • to get stacktrace try:

    trace/a ffff88d0d467c1a0
trace: pid 0 lid 35 at 0xffffca80aead2dc0
sleepq_block() at netbsd:sleepq_block+0xbb
kpause() at netbsd:kpause+0x10b
__wdcwait_reset() at netbsd:__wdcwait_reset+0x162
wdcreset() at netbsd:wdcreset+0x7d
wdc_sataprobe() at netbsd:wdc_sataprobe+0x17a
atabusconfig() at netbsd:atabusconfig+0x65
atabus_thread() at netbsd:atabus_thread+0x6d

  • NOTE: I have found that after around 5 minutes kernel will give up and continue boot but without detecting our HDD....

Here is recommended patch

FWIW, part of testing attempts to read from a failing disk i had written code to do similar with kernel options. i vaguely meant to try to convert it to using some sort of config flags so it's per-device capable, not all devices, that's why i didn't commit or send for review yet.

you'd use it for your need as:

options SATA_FORCE_SPD=SControl_SPD_G1

And the kernel patch from same source:

Index: sys/dev/ata/sata_subr.c
===================================================================
RCS file: /cvsroot/src/sys/dev/ata/sata_subr.c,v
retrieving revision 1.24
diff -p -u -r1.24 sata_subr.c
--- sys/dev/ata/sata_subr.c	21 Jun 2018 21:52:15 -0000	1.24
+++ sys/dev/ata/sata_subr.c	14 May 2023 17:12:15 -0000
@@ -93,7 +93,11 @@ sata_reset_interface(struct ata_channel 
 	 * the device. It doesn't hurt for other devices.
 	 */
 	bus_space_write_4(sata_t, scontrol_r, 0, 0);
+#ifdef SATA_FORCE_SPD
+	scontrol = SControl_IPM_NONE | SATA_FORCE_SPD | SControl_DET_INIT;
+#else
 	scontrol = SControl_IPM_NONE | SControl_SPD_ANY | SControl_DET_INIT;
+#endif
 	bus_space_write_4(sata_t, scontrol_r, 0, scontrol);
 
 	ata_delay(chp, 50, "sataup", flags);

Virtualization

Used this guide: https://www.netbsd.org/docs/guide/en/chap-virt.html#chap-virt-qemu

Unfortunately my CPU is not supported (as in case of FreeBSD and OpenBSD).

# modload nvmm

modload: nvmm: Not supported

# dmesg | tail -3 | sed 's/\[[^]]*\] *//'

NVMM: SVM-NP not supported
autoconfiguration error: nvmm: cpu not supported
WARNING: module error: modcmd(CMD_INIT) failed for `nvmm', error 86

Quirks

smartctl: device is busy

If you install smartctl with:

pkgin install smartmontools

If you follow manual page literally:

smartctl -a /dev/wd0d

Smartctl open device: /dev/wd0d failed: Device busy

Solution is offered here:

smartctl -a /dev/rwd0d

NetBSD on Raspberry PI

I own nearly 10 years old Raspberry PI 1B. I want to try NetBSD here.

  • let's start here: https://wiki.netbsd.org/ports/evbarm/raspberry_pi/#index2h2

  • download SD card image from: http://nycdn.netbsd.org/pub/NetBSD-daily/netbsd-9/202306020140Z/evbarm-earmv6hf/binary/gzimg/rpi.img.gz

  • I ungzip this image, keeping original (with -k - keep):

    gzip -dk rpi.img.gz
ls -lh rpi.img

-rw-r--r-- 1 hpaluch users 1.2G Jun  3 08:50 rpi.img

  • as pointed out in official docs, at least 2GB SD card required tested on 4GB.

  • you can login on local HDMI console using root - first time you will be defined, but second time you can login as root without password

  • note: hf means "hardware float" - that CPU has hardware support for floating point math

  • first look:

    $ sysctl hw.machine hw.model hw.byteorder hw.machine_arch

hw.machine = evbarm
hw.model = raspberrypi,model-b
hw.byteorder = 1234
hw.machine_arch = earmv6hf

$ uname -a

NetBSD rpi 9.3_STABLE NetBSD 9.3_STABLE (RPI) #0: Sun May 28 10:23:12 UTC 2023 \
  [email protected]:/usr/src/sys/arch/evbarm/compile/RPI evbarm

$ dmesg -t | fgrep cpu

cpus0 at simplebus0
cpu0 at cpus0: 700 MHz ARM1176JZ-S r0p7 (ARM11J V6ZK core)
cpu0: DC enabled IC enabled WB enabled LABT
cpu0: 16KB/32B 4-way L1 VIPT Instruction cache
cpu0: 16KB/32B 4-way write-back-locking-C L1 VIPT Data cache
vfp0 at cpu0: VFP11, rounding, exceptions

$ dmesg -t | fgrep memory

total memory = 448 MB
avail memory = 434 MB

$ sysctl hw.disknames

hw.disknames = ld0

$ dmesg -t | fgrep ld0

ld0 at sdmmc0: <0x41:0x3432:SD4GB:0x10:0x21c004d2:0x084>
ld0: 3859 MB, 1960 cyl, 64 head, 63 sec, 512 bytes/sect x 7903232 sectors
ld0: 4-bit width, High-Speed/SDR25, 50.000 MHz
boot device: ld0
root on ld0a dumps on ld0b

$ doas gpt show ld0

GPT not found, displaying data from MBR.

    start     size  index  contents
        0        1         MBR
        1    32767         Unused
    32768   163840      1  MBR part 12 (active)
   196608  7706624      2  MBR part 169

$ doas disklabel ld0 | sed '1,/^$/d'

8 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a:   7706624    196608     4.2BSD      0     0     0  # (Cyl.     96 -   3858)
 c:   7903232         0     unused      0     0        # (Cyl.      0 -   3858)
 d:   7903232         0     unused      0     0        # (Cyl.      0 -   3858)
 e:    163840     32768      MSDOS                     # (Cyl.     16 -     95)

  • WARNING! There are contradicting reports regarding swap - from: https://wiki.netbsd.org/ports/evbarm/raspberry_pi/#index7h2

    Note that swap is after /boot and before /, and not contained in the NetBSD fdisk partition.

    But I can't see reserved space for swap (the ID=169 partition is immediately after FAT32 partition (ID=12) and no swap is seen in disklabel. Anyway SD cards are so slow and so unreliable that it probably does not matter...

  • added sync to /etc/fstab (noatime was already there)

NOTE: FAT32 partition (used for boot) is already mounted under /boot!

  • content of /boot/config.txt

    # UART settings, see https://www.raspberrypi.org/documentation/configuration/uart.md
enable_uart=1
force_turbo=0

  • content of /boot/cmdline.txt

    root=ld0a console=fb
#fb=1280x1024           # to select a mode, otherwise try EDID
#fb=disable             # to disable fb completely

  • once system looks fine I removed HDMI output (removing the console=fb from /boot/cmdline.txt

  • here is how I connect to RPi from Minicom on Linux:

    minicom -b 115200 -8 -D /dev/ttyUSB0
# for logging add: -C console.log

Cross building kernel for for RPi 1b.

  • according to: https://wiki.netbsd.org/ports/evbarm/raspberry_pi/#index2h2 the hardware platform use for RPi 1b is called earmv6hf

  • for Host system we have to obtain sources first using: https://www.netbsd.org/docs/guide/en/chap-fetch.html

  • use your favorite browser to see 9-stable sources on: https://cdn.netbsd.org/pub/NetBSD/NetBSD-release-9/tar_files/

  • now on your NetBSD 9.3 Host (tested amd64 machine) try:

    # Host NetBSD 9.3 system (amd64), non-privileged user
mkdir ~/9-stable
cd ~/9-stable/
curl -kfLO https://cdn.netbsd.org/pub/NetBSD/NetBSD-release-9/tar_files/src.tar.gz
tar xzf src.tar.gz
cd src

  • now we have to build toolchain following: https://www.netbsd.org/docs/guide/en/chap-build.html#chap-build-tools

    # again follow CPU table on https://wiki.netbsd.org/ports/evbarm/raspberry_pi/#index2h1
# for RPi 1b we are looking for 'earmv6hf

./build.sh list-arch | fgrep earmv6hf
MACHINE=evbarm          MACHINE_ARCH=earmv6hf   ALIAS=evbearmv6hf-el    ALIAS=evbarmv6hf-el
MACHINE=evbarm          MACHINE_ARCH=earmv6hfeb ALIAS=evbearmv6hf-eb    ALIAS=evbarmv6hf-eb

  • to build toolchain (cross-gcc to make ARM binaries for RPi 1.b) it is strongly recommended to start tmux session on host and issue suitable cross-build command:

    tmux
cd ~/9-stable/src
# the -j2 invokes parallel build for 2 core CPU (my case)
time ./build.sh -U -O ~/obj -j2 -m evbarm -a earmv6hf tools 
# it took 30minutes on 2 core Athlon, 2GHz per core, 8GB RAM, SSD disk

  • now we will try to just build same kernel (to verify that we can build it properly)

  • recall, how to find current kernel arch/config

    # run on RPi 1.b target

$ uname -a

NetBSD rpi 9.3_STABLE NetBSD 9.3_STABLE (RPI) #0: Sun May 28 10:23:12 UTC 2023 \
  [email protected]:/usr/src/sys/arch/evbarm/compile/RPI evbarm

  • to cross-build kernel follow https://www.netbsd.org/docs/guide/en/chap-build.html#chap-boot-cross-build-kernel

  • note: we will skip cloning and/or modifying kernel config for that 1st build

  • you can review used kernel config file $HOME/9-stable/src/sys/arch/evbarm/conf/RPI

  • and then cross-build RPI kernel using:

    cd ~/9-stable/src
time ./build.sh -U -u -j2 -O ~/obj -m evbarm -a earmv6hf kernel=RPI
# around 8 minutes

       Kernels built from RPI:
        $HOME/obj/sys/arch/evbarm/compile/RPI/netbsd

$ file ~/obj/sys/arch/evbarm/compile/RPI/netbsd

~/obj/sys/arch/evbarm/compile/RPI/netbsd: ELF 32-bit LSB executable, \
 ARM, EABI5 version 1 (SYSV), statically linked, for NetBSD 9.3, not stripped

  • WARNING! As pointed out on https://wiki.netbsd.org/ports/evbarm/raspberry_pi/#index16h2 for boot you need netbsd.img (image for Boardcom bootlader on RPi). It is not possible to boot ELF kernel called netbsd on RPi!

  • copy ~/obj/sys/arch/evbarm/compile/RPI/netbsd.img to your RPi target and do something like:

    # on RPi 1.b target:
cd /boot
cp /home/USER/netbsd.img kernel.img
sync  
cd / # allow umount of /boot

  • spoiler alert - in my case that netbsd.img had exactly same size as original kernel.img

  • and finally on RPi try reboot and watch your (serial) console...

  • verify kernel version - that FQDN is your build machine. In my case:

    # run on RPi

$ uname -v | fold -s -w 80

NetBSD 9.3_STABLE (RPI) #0: Sat Jun  3 13:34:17 UTC 2023 
[email protected]:/home/ssduser/obj/sys/arch/evbarm/compile/RPI

TODO:

  • I want to access LM75 (I2C temperature) sensors from NetBSD in similar way as from Raspbian as documented on: https://github.com/hpaluch/pi-lm75a

  • so far I have to study:

    • man iic - this looks good: iic* at bcmi2c? # evbarm

    • man lmtemp

  • in kernel config RPI there is interesting commented out examples:

    # 'DS3231 Raspberry Pi RTC Board Real Time Clock Module for Arduino'
# sold by linksprite.com
#dsrtc* at iic1 addr 0x68 flags 3231

  • and manual page for lmtemp(4) shows:

    mtemp0 at iic? addr 0x48 flags 0x0000

  • where flags from manual page are:

    Device    addr           flags
LM75      0x48 - 0x4f    0x0000
LM75A     0x48 - 0x4f    0x0000 ### This I have
DS75      0x48 - 0x4f    0x0001
LM77      0x48 - 0x4b    0x0002

  • so theoretically this in RPI config would work in my case:

    mtemp0 at iic1 addr 0x48 flags 0x0000

  • the only problem will be to find right I2C bus, because my kernel show 3 I2C busses:

    bsciic0 at simplebus1: Broadcom Serial Controller
iic0 at bsciic0: I2C bus
bsciic1 at simplebus1: Broadcom Serial Controller
iic1 at bsciic1: I2C bus
bsciic2 at simplebus1: Broadcom Serial Controller
iic2 at bsciic2: I2C bus

  • maybe look here: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html

  • I2C has to be enabled - normally with raspi-config - kernel module (applies for Linux only) but also multipurpose pin configuration - from https://github.com/fivdi/i2c-bus/blob/master/doc/raspberry-pi-i2c.md

  • add to /boot/config.txt

  • dtparam=i2c_arm=on

  • also here:

NetBSD as workstation

I'm evaluating all 3 BSDs as GUI Workstation (to escape from systemd and forced containerization that is happening on Linux).

Let's start with sound support:

  • https://www.netbsd.org/docs/guide/en/chap-audio.html#chap-audio-configuring-device

  • here is my Audio hardware:

    pcictl pci0 list | fgrep Audio

# onboard sound-card - MSI-7250 board:
000:06:1: NVIDIA nForce MCP55 High Definition Audio Controller (mixed mode multimedia, HD Audio 1.0, revision 0xa2)
# HDMI audio on graphics card (I never used it):
006:00:1: NVIDIA GeForce 210 High Definition Audio Controller (mixed mode multimedia, HD Audio 1.0, revision 0xa1)

  • in my case:

    audiocfg list

0: [*] audio0 @ hdafg0: Realtek ALC883
       playback: 2ch, 48000Hz
       record:   2ch, 48000Hz
...

  • now we can just follow guide and try:

    audiocfg test 0

  • it will play sound for 2 seconds on channel (not sure which one should be left and right

  • somewhow my mixer has too many outputs:

    mixerctl -av | fgrep -w outputs
outputs.master=248,248 volume delta=8
outputs.master2=0,0 volume delta=256
outputs.master2.mute=off  [ off on ]
outputs.master3=0,0 volume delta=256
outputs.master3.mute=off  [ off on ]
outputs.master4=248,248 volume delta=8
outputs.master5=0,0 volume delta=256
outputs.master5.mute=off  [ off on ]
outputs.master6=0,0 volume delta=256
outputs.master6.mute=off  [ off on ]
outputs.master7=248,248 volume delta=8
outputs.master8=0,0 volume delta=256
outputs.master8.mute=off  [ off on ]
outputs.master9=0,0 volume delta=256
outputs.master9.mute=off  [ off on ]
outputs.master10=248,248 volume delta=8
outputs.master11=0,0 volume delta=256
outputs.master11.mute=off  [ off on ]
outputs.master12=0,0 volume delta=256
outputs.master12.mute=off  [ off on ]
outputs.master13=0,0 volume delta=256
outputs.master13.mute=off  [ off on ]
outputs.master14=0,0 volume delta=256
outputs.master14.mute=off  [ off on ]
outputs.master15=0,0 volume delta=256
outputs.master15.mute=off  [ off on ]
outputs.master16=0,0 volume delta=256
outputs.master16.mute=off  [ off on ]
outputs.dacsel=DAC00,DAC01,DIG02  { DAC00 DAC01 DIG02 }

How to test free samples:

  • install Ogg audio + player:

    doas pkgin in sound-theme-freedesktop vorbis-tools

  • list samples:

    pkg_info -L sound-theme-freedesktop

  • and run:

    ogg123 /usr/pkg/share/sounds/freedesktop/stereo/*.oga

And here my favourite console audio apps:

doas pkgin in mpg123 streamripper

Example playing online Heavy metal radio:

mpg123 http://stream.antenne.de:80/heavy-metal

If you find it too loud you can try command like:

mixerctl -w outputs.master=140
# my range is 0,0 to 248,248

We will follow guide to set this volume on boot:

  • create file /etc/mixerctl.conf with content like:

    outputs.master=140,140

  • and set mixerctl=YES in /etc/rc.conf

Using X-Window on NetBSD

In my case I did not install X so I needed to add them later using:

  • run as root sysinst
  • select >c: Re-install sets or install additional sets
  • confirm Yes
  • select >a: current system
  • Deselect all existing sets - from a to e !!!
  • select >d: Custom installation
  • select >m: X11 sets and confirm all
  • then start install selecting x: Install selected sets
  • install from: I used b: HTTP (I was lazy to search for CD-ROM)
  • confirm Host and follow x: Get Distribution
  • after installation select Exit system
  • after installation reboot your system (just to be sure)

Also add few usful tools:

doas pkgin in firefox
doas pkgin in xmms xmix

If you see warning that some shared libraries under /usr/X11R7/lib are missing it means that you did not install X-Window sets. In such example follow instructions from begining of this chapter.

Finally login to local console and try startx

PROBLEM:

Unfortunately I had again problems with nVidia GT218 under NetBSD 9.3-RELEASE

  • seems to be resolved in NetBSD 10 beta (see chapter below)
  • there is accelerated driver (Nouveau) from Linux on both kernel and X11, but in /var/log/Xorg.0.log found:
(EE) NOUVEAU(0): Error creating GPU channel: -12
(EE) NOUVEAU(0): Error initialising acceleration.  Falling back to NoAccel
(**) NOUVEAU(0): [COPY] acceleration disabled

And also dmesg -t shows:

nouveau0: autoconfiguration error: error: user: nvif_object_map, -12
nouveau0: autoconfiguration error: error: user: channel failed to initialise, -12

The errors is reported in:

  • /usr/src/sys/external/bsd/drm2/dist/drm/nouveau/nvif/nouveau_nvif_object.c
int
nvif_object_map(struct nvif_object *object)
{
// ...

                ret = client->driver->map(client, args.map.tag,
                    args.map.handle, object->map.size, &object->map.handle,
                    &object->map.ptr);
               if (ret) {
                        printf("HP: P2b %s:%d ret=%d mapfn=0x%p size=0x%x, tag=0x%p, handle=0x%lx\n",
                                        __FILE__,__LINE__,ret,client->driver->map,
                                        args.map.length,args.map.tag, (unsigned long) args.map.handle);
                        nvif_object_unmap(object);
                        return -ENOMEM;
                }

Got messages:

HP: P2b ../../../../external/bsd/drm2/dist/drm/nouveau/nvif/nouveau_nvif_object.c:231
 ret=-35 mapfn=0x0xffffffff8076fb0c size=0x2000, tag=0x0xffffffff80e01340, handle=0xfdc06000
nouveau0: autoconfiguration error: error: user: nvif_object_map, -12
...

Please note that actual error code (-35) is later rewritten to (-12) ENOMEM.

Reported here by someone else:

The default Window manager is called ctwm. To add Firefox and xmms to menu we need to first found which configuration file it uses. Please ignore manual page (man ctwm) - there are bogus paths that don't exists on real system. Here is geek way how to find it:

strings /usr/X11R7/bin/ctwm | fgrep ctwmrc

/etc/X11/ctwm/system.ctwmrc
%s/.ctwmrc.%d

So let's copy system wide configuration and modify it:

cp /etc/X11/ctwm/system.ctwmrc ~/.ctwmrc

Here is my modification - adding Firefox and XMMS to main menu:

diff -u /etc/X11/ctwm/system.ctwmrc ~/.ctwmrc
--- /etc/X11/ctwm/system.ctwmrc 2022-08-04 15:30:37.000000000 +0000
+++ /home/ssduser/.ctwmrc       2023-06-10 13:47:55.013533958 +0000
@@ -207,7 +207,9 @@
     " Terminal"                        !"uxterm &"
     " Calculator"              !"xcalc &"
     " Clock"                   !"xclock -digital &"
-    " XEyes"                   !"xeyes &"
+    " Firefox"                 !"firefox &"
+    " XMMS"                    !"xmms &"
+    " Mixer"                   !"xmix &"
     ""                         f.separator
     " Applications"            f.menu "appmenu"
     ""                         f.separator

And Quit and startx again.

XMix (xmix) tips:

  • xmix used Athena Widget set, which has specific way how scroll-bars work:
    • the more down of scrollbar you click the biggger move it will be (however I was unable to observe this effect on xmix)
    • the diretion of scroll is determined by mouse buttons:
      • Left button scrolls up
      • Right mouse button scrolls down

CTWM tips:

  • to switch workspace, press Win key (known as mod4 key in .ctwmrc and Down arrow or Up arrow
  • to see "Applications" submenu you have to hover over list icon (shown at right after "Applications" keyword)
  • the Applications submenu is generated by command /usr/X11R7/libexec/ctwm_app_menu run from .ctwmrc
  • here is result in my example: 
    /usr/X11R7/libexec/ctwm_app_menu

menu "appmenu"
{
        "Applications"  f.title
        " Browser Web Browser" !"firefox &"
}

NFS Server

How to enable NFS server on NetBSD 9.3:

  • DISCLAIMER! Setup below is generally unsafe and suitable for safe isolated network only!

  • you hae to add to /etc/rc.conf:

    rpcbind=YES rpcbind_flags="-l"      # -l logs libwrap
# -N is required if your NFS client is behind NAT and thus
#    both IP and client ports are translated
mountd=YES   mountd_flags="-N" # NFS mount requests daemon
nfs_server=YES                 # enable server daemons

  • prepare directory for exports

    mkdir /backups
# allow everybody to create files, but only owner can remove them
chmod a+rwxt

  • now you need to add suitable entry into /etc/exports

    /backups -public

  • not restart NetBSD or start manually those services in specified order:

    /etc/rc.d/rpcbind start
/etc/rc.d/mountd start
/etc/rc.d/nfsd start

  • NOTE: if you use NPF firewall, temporarily disable it with:

    npfctl stop

  • from client verify with this command exported directories:

    showmount -e IP_OF_NFS_SERVER

  • and mount it from client:

    mkdir /mnt/nfs
mount -t nfs IP_OF_NFS_SERVER:/backups /mnt/nfs

  • try to create some file or directory.

Testing Beta

To get more recent nouveau(4) driver in both kernel & X, trying beta. Downloaded and booted:

Here is kernel ident

NetBSD 10.0_BETA (GENERIC) #0: Fri Jun 23 05:42:34 UTC 2023
        [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC

And here latest nVidia GT218 error message :-/

nouveau0 at pci6 dev 0 function 0: NVIDIA GeForce 210 (rev. 0xa2)
...
[drm] Initialized nouveau 1.3.1 20120801 for nouveau0 on minor 0
nouveaufb0: framebuffer at 0xc0006000, size 1440x900, depth 32, stride 5888
nouveau0: autoconfiguration error: error: DRM: core notifier timeout
nouveau0: autoconfiguration error: error: DRM: base-0: timeout
no data for est. mode 640x480x67

However X-Window seems to work properly(!) There are messages in Xorg.0.log confirming that 2D EXA acceleration is used, etc. And YT seems to be able to play video with usable FPS.

More info:

cd /usr/src/sys/
fgrep -rn 'core notifier timeout'

./external/bsd/drm2/dist/drm/nouveau/dispnv50/\
nouveau_dispnv50_disp.c:1924:		NV_ERROR(drm, "core notifier timeout\n");

exFAT support

I'm planning to setup big backup disk with small NetBSD 9.3 at the begin for recovery and administration and several bit exFAT partition (on GPT+BIOS scheme).

And NetBSD is preferred over Linux (yes!) because NetBSD kernel has all drivers compiled in - so having backup disk with small bootable NetBSD can be beneficial when tried on different PCs.

So I started with test VM under KVM - 30 GB disk, only 20 GB used for installation:

  • booted NetBSD 9.3 ISO
  • installed in GPT + BIOS mode
  • here is view from NetBSD 9.3 right after installation (no exFAT GPT partition created yet):
gpt show ld0
     start      size  index  contents
         0         1         PMBR
         1         1         Pri GPT header
         2        32         Pri GPT table
        34        30         Unused
        64  40960000      1  GPT part - NetBSD FFSv1/FFSv2
  40960064   2097152      2  GPT part - NetBSD swap
  43057216  19857311         Unused
  62914527        32         Sec GPT table
  62914559         1         Sec GPT header

dkctl ld0 listwedges
/dev/rld0: 2 wedges:
dk0: a049e2ba-d8cd-45c5-a312-f6b53ff9d61d, 40960000 blocks at 64, type: ffs
dk1: 59169732-4e80-4f36-a945-9c1ee89fe07c, 2097152 blocks at 40960064, type: swap

Now lets add new GPT partition for exFAT:

  • we must first note 1st free block at the end of disk:

       start      size  index  contents
...
40960064   2097152      2  GPT part - NetBSD swap
43057216  19857311         Unused

  • in our case it is the first number of "Unused" partition so it is 43057216

  • to create exFAT partition from this block run (we MUST specify starting block with -b, because it will otherwise create parition in first free space - just few sectors):

    gpt add -a 4k -t windows -l exfat1 -b 43057216 ld0

/dev/rld0: Partition 3 added: ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 43057216 19857304

  • NOTE: I use 4KB align (-a 4k) because I will use it on 1TB Seagate HDD which has 512 logical but 4096 physical sectors. So we have to avoid write amplification with 4KB alignemnt.

  • now again list whole GPT partition table:

    gpt show ld0
   start      size  index  contents
       0         1         PMBR
       1         1         Pri GPT header
       2        32         Pri GPT table
      34        30         Unused
      64  40960000      1  GPT part - NetBSD FFSv1/FFSv2
40960064   2097152      2  GPT part - NetBSD swap
43057216  19857304      3  GPT part - Windows basic data
62914520         7         Unused
62914527        32         Sec GPT table
62914559         1         Sec GPT header

  • our new partition is number 3 called GPT part - Windows basic data

  • now let's see if it was added to wedge (dkX) device:

    dkctl ld0 listwedges

/dev/rld0: 3 wedges:
dk0: a049e2ba-d8cd-45c5-a312-f6b53ff9d61d, 40960000 blocks at 64, type: ffs
dk1: 59169732-4e80-4f36-a945-9c1ee89fe07c, 2097152 blocks at 40960064, type: swap
dk2: exfat1, 19857304 blocks at 43057216, type: ntfs

  • looks good - `dk21 is our future exFAT filesystem.

Let's format that partition:

# first install package

pkgin in fuse-exfat
# to see all installed files try:
pkg_info -L fuse-exfat

# again find right device
dkctl ld0 listwedges
  
/dev/rld0: 3 wedges:
dk0: a049e2ba-d8cd-45c5-a312-f6b53ff9d61d, 40960000 blocks at 64, type: ffs
dk1: 59169732-4e80-4f36-a945-9c1ee89fe07c, 2097152 blocks at 40960064, type: swap
dk2: exfat1, 19857304 blocks at 43057216, type: ntfs
 
# !!!
# replace 'dk2' with PROPER device !!
# !!!
/usr/pkg/sbin/mkexfatfs -n EXFAT1 /dev/rdk2

mkexfatfs 1.3.0
ERROR: Unable to determine file system size.
# Hmm, ...

mkexfatfs -n EXFAT1 /dev/dk2
mkexfatfs 1.3.0
Creating... done.
Flushing... done.
File system created successfully.

And finally mount it and try to put something there:

mkdir -p /mnt/exfat

mount.exfat-fuse /dev/dk2 /mnt/exfat/

FUSE exfat 1.3.0

df -h /mnt/exfat

Filesystem         Size       Used      Avail %Cap Mounted on
/dev/puffs         4.7T       157M       4.7T   0% /mnt/exfat

PROBLEM! The reported size (4.7TB) is definitely wrong! Need to investiage

WARNING! You have to unmount exFAT manually before reboot - the system will not do it alone:

cd / # step away from unmounted exFAT volume
umount /mnt/exfat/

If you are brave you can try on unmounted filesystem:

exfatfsck -n /dev/dk2

exfatfsck 1.3.0
Checking file system on /dev/dk2.
File system version           1.0
Sector size                 512 bytes
Cluster size                128 KB
Volume size                4848 GB
Used space                  368 MB
Available space            4848 GB
Totally 2 directories and 21 files.
File system checking finished. No errors found.

PROBLEM! Volume size is incorrect (4.8TB - should be 20GB) and available space is WRONG!

Here is my experimental patch, version 2:

--- /home/builder/orig/io.c	2023-07-01 08:28:46.534337339 +0000
+++ libexfat/io.c	2023-07-02 07:39:07.883316521 +0000
@@ -244,7 +244,7 @@ struct exfat_dev* exfat_open(const char*
 			/* getdisksize() needs the raw device name      */
 			getdiskrawname(device, sizeof(device), spec);
 			getdisksize(device, &secsize, &dksize);
-			dev->size = secsize * dksize;
+			dev->size = dksize;
 		}
 		if (dev->size <= 0) {
 			exfat_error("Unable to determine file system size");

The problem is, that getdisksize() already returns scaled value in &dksize...

Need to do more testing though...

Reported here:

Another problem:

  • it si impossible to call getcwd() on exFAT in subdirectory:
cd /mnt/exfat/
pwd

/mnt/exfat # OK


# but in subdir:
mkdir test
cd test
pwd

pwd: getcwd() failed: No such file or directory

There is nothing wrong in puffs log (when mount.exfat-fuse is called with -d), but ktrace of shell confirms problem:

   991      1 sh       NAMI  "/mnt/exfat/test"
   991      1 sh       RET   __stat50 0
   991      1 sh       CALL  chdir(0x11ee2e7a0)
   991      1 sh       NAMI  "/mnt/exfat/test"
   991      1 sh       RET   chdir 0
   991      1 sh       CALL  __getcwd(0x11ee2e7a0,0x100)
   991      1 sh       RET   __getcwd -1 errno 2 No such file or directory

Have to study:

Resources:

fuse-exfat: wrong attempt

Below is attempt to reproduce it on regular file. But it uses different path - so there will be no error in this case !

Attempt to reproduce on image file:

  • create quickly 4GB image file using holes:

    # convert 20 GB to number of sectors (= blocks for dd )
echo $(( 20 * 1024 * 1024 * 2 ))

41943040 # number of sectors for 20GB

cd
dd if=/dev/zero count=1 of=exfat.img seek=41943040
ls -lh exfat.img

ls -lh exfat.img
-rw-r--r--  1 root  wheel   20G Jul  1 07:39 exfat.img

mkexfatfs -n TEST_EXFAT exfat.img

exfatfsck -n exfat.img
exfatfsck 1.3.0
Checking file system on exfat.img.
File system version           1.0
Sector size                 512 bytes
Cluster size                 32 KB
Volume size                  20 GB
Used space                 2785 KB
Available space              20 GB
Totally 0 directories and 0 files.
File system checking finished. No errors found.

  • so it is not problem of exfatfsck...

The problem occurs only with wedges, because exfat routines are deeply incorrect...

Example of using wedges:

Original package url:

Rebuilding pkgsrc

The NetBSD's package system is called pkgsrc (packages that you install with pkg_add or pkgin command). There is some documentation at:

But few items are missing.

There is very likely to occur conflict between installed binary packages and build source packages - because they use same path for installation. Therefore I strongly recommend to use different target for build source packages to avoid clashing versions:

  • as root install userful programs to download source:

    pkgin in mozilla-rootcerts mozilla-rootcerts-openssl curl

  • so create dedicated user for building, in my case:

    /usr/sbin/useradd -s /bin/sh -m builder
passwd builder

  • and prepare unique target directories:

    mkdir -p /opt/pkg /var/opt/pkg
chown builder /opt/pkg /var/opt/pkg

  • now as user builder download and unpack source packages:

    cd 
curl -fLO https://cdn.netbsd.org/pub/pkgsrc/stable/pkgsrc.tar.bz2
tar xf pkgsrc.tar.bz2

  • change target directories to avoid packages conflicts:

diff -u pkgsrc/mk/defaults/mk.conf.orig pkgsrc/mk/defaults/mk.conf
--- pkgsrc/mk/defaults/mk.conf.orig     2023-07-01 07:42:05.832164577 +0000
+++ pkgsrc/mk/defaults/mk.conf  2023-07-01 07:58:18.628059865 +0000
@@ -362,7 +362,7 @@
 #
 # Keywords: work tmp
 
-LOCALBASE?=    /usr/pkg
+LOCALBASE?=    /opt/pkg
 #      This is the base directory where all packages will be installed.
 #
 #      Possible values: Any absolute path that does not contain special
@@ -376,7 +376,7 @@
 # Possible: any path
 # Default: ${LOCALBASE}/cross
 
-VARBASE?=      /var
+VARBASE?=      /var/opt/pkg
 # Where files containing local state information are installed
 # Possible: any path
 # Default: /var
@@ -1810,7 +1810,7 @@
 # Possible: any group name
 # Default: ucspissl
 
-UNPRIVILEGED?= NO
+UNPRIVILEGED?= YES
 # Set this to YES to enable unprivileged support (see mk/unprivileged.mk).
 # Possible: YES, NO
 # Default: NO

Example of build for fuse-exfat package:

find pkgsrc -type d -a -name 'fuse-exfat'

  pkgsrc/filesystems/fuse-exfat

cd pkgsrc/filesystems/fuse-exfat/
unset PKG_PATH

Now tricky part - find dependencies (see https://wiki.netbsd.org/pkgsrc/targets/ for full list of targets):

make show-depends
# empty :-/
make clean-depends
# much better: output
===> Cleaning for cwrappers-20220403
===> Cleaning for mktools-20220614
===> Cleaning for digest-20220214
===> Cleaning for gmake-4.4.1
===> Cleaning for perl-5.36.1
===> Cleaning for p5-gettext-1.07nb7
===> Cleaning for help2man-1.49.3
===> Cleaning for m4-1.4.19nb1
===> Cleaning for autoconf-2.71nb2
===> Cleaning for automake-1.16.5nb3
===> Cleaning for libtool-base-2.4.7nb1
===> Cleaning for pkgconf-1.9.5

So try build:

# in subdir: pkgsrc/filesystems/fuse-exfat
# fetch and install dependencies
make depends
# fetch and build fuse-exfat
make
# install our version of fuse-exfat...
...

Puffs quick-start

To understand Puffs we may use trivial example. Tested on NetBSD 9.3 stable (CVS tag netbsd-9, see https://www.netbsd.org/docs/guide/en/chap-fetch.html#chap-fetch-cvs for branches and tag names)

We will test so called nullfs, to quote /usr/src/lib/libpuffs/null.c

A "nullfs" using puffs, i.e. maps one location in the hierarchy to another using standard system calls.

It is similar to Linux "bind mount" - where you basically "link" one directory to another location. There even exists "regular" null filesystem - see mount_null(8) where you can find this description:

The mount_null command creates a null layer, duplicating a sub-tree of the file system name space under another part of the global file system namespace. This allows existing files and directories to be accessed using a different pathname.

So "null" in this context is NOT empty (or NOT fake) - it is basically "bind" or "alias" filesystem.

To setup nullfs example do this:

  • you need to additionally install sharesrc in case of NetBSD 9.3:

  • full source installation example (skipping just X11 - not needed for nullfs):

    cd
curl -fLO https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.3/source/sets/src.tgz
curl -fLO https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.3/source/sets/syssrc.tgz
curl -fLO https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.3/source/sets/sharesrc.tgz
tar xpvzf src.tgz -C /
tar xpvzf syssrc.tgz -C /
tar xpvzf sharesrc.tgz -C /

  • build nullfs example:

    cd /usr/src/share/examples/puffs/pnullfs
# USETOOLS=no is required to skip cross-building but use system's gcc
make USETOOLS=no

  • please be aware that main nullfs code is actually part of puffs library, you can find it in

    • implementation: /usr/src/lib/libpuffs/null.c
    • declarations: /usr/src/lib/libpuffs/puffs.h

  • so the code in /usr/src/share/examples/puffs/pnullfs/pnullfs.c is basically tiny wrapper around puffs library calls

  • now create sample directories

    # run as root
# create pass-through ("device" - actually tree) - 1st argument
mkdir -p /mnt/puffs-dev/aaaa/
touch /mnt/puffs-dev/bbbbb
# create mount point - 2nd argument 
mkdir -p /mnt/puffs-mnt/

  • now mount passthrough directory using puffs:

    # run as root
cd /usr/src/share/examples/puffs/pnullfs
./pnullfs /mnt/puffs-dev/ /mnt/puffs-mnt/

# and see
ls -l /mnt/puffs-mnt/

# it is actually content of /mnt/puffs-dev !
total 2
drwxr-xr-x  2 root  wheel  512 Jul  7 13:23 aaaa
-rw-r--r--  1 root  wheel    0 Jul  7 13:41 bbbbb

# will show same parameters as /mnt/puffs-dev filesystem
df -h /mnt/puffs-mnt/

Filesystem            Size       Used      Avail %Cap Mounted on
/mnt/puffs-dev/        47G       9.5G        35G  21% /mnt/puffs-mnt

mount | fgrep puffs

/mnt/puffs-dev/ on /mnt/puffs-mnt type puffs|pnullfs

  • to finish just unmount it standard way:

    umount /mnt/puffs-mnt

Backup/restore

I use trivial .tar.gz archive to backup system and restore + dedicated backup partition

In my case NetBSD is installed on 2nd drive wd1, using MBR partitions and BIOS mode.

To backup:

  • boot in single mode, confirm /bin/sh as shell

  • mount backup partition I use something like mount /dev/wd1e /backups (in my example I use partition on same disk - to avoid loss of data I then copy it over network to other machine).

  • ensure that you have copy of important files:

    • copy of /etc/fstab - my example (NetBSD on 2nd drive): 
      # NetBSD /etc/fstab
# See /usr/share/examples/fstab/ for more examples.
/dev/wd1a               /       ffs     rw,log,noatime           1 1
/dev/wd1e               /backups        ffs     rw,log,noatime,noauto 1 2
/dev/wd1b               none    swap    sw,dp            0 0
kernfs          /kern   kernfs  rw
ptyfs           /dev/pts        ptyfs   rw
procfs          /proc   procfs  rw
/dev/cd0a               /cdrom  cd9660  ro,noauto
tmpfs           /var/shm        tmpfs   rw,-m1777,-sram%25
    • MBR partition output of fdisk wd1: 
      doas fdisk wd1
...
Partition table:
0: NetBSD (sysid 169)
    start 2048, size 716800753 (350000 MB, Cyls 0/32/33-44618/232/15), Active
1: <UNUSED>
2: <UNUSED>
3: <UNUSED>
...
    • disklabel 
      doas disklabel wd1
...

5 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a: 409600000      2048     4.2BSD      0     0     0  # (Cyl.      2*- 406351*)
 b:  67108864 409602048       swap                     # (Cyl. 406351*- 472927*)
 c: 716800753      2048     unused      0     0        # (Cyl.      2*- 711113*)
 d: 937703088         0     unused      0     0        # (Cyl.      0 - 930260)
 e: 240091136 476710912     4.2BSD      0     0     0  # (Cyl. 472927*- 711113*)
    • I use a root filesystem, b swap and e backup partition.
    • IMPORTANT! Note which version of FFS filesystem you use (you will need to specify it to newfs command when formatting new filesystem: 
      doas dumpfs -s wd1a | grep '^\(format\|fsmnt\)'

format  FFSv1
fsmnt   /

doas dumpfs -s wd1e | grep '^\(format\|fsmnt\)'

format  FFSv2
fsmnt   /backups

  • use tar to backup filesystem:

    tar -cvz --numeric-owner --one-file-system \
      -f /backups/netbsd93-rootfs-01-clean-`date '+%Y%m%d-%H%M'`.tar.gz -C / .

  • example of NetBSD 10 RC3 backup script backup_wd500.sh installed on other disk (Wester Digital 500GB - so nicknamed wd500):

    #!/bin/sh
set -eux
# following required to stop opening devices: --no-acls --no-xattrs
# see: bin/55815
os_name=`uname -s | tr '[:upper:]' '[:lower:]'`
tar -cvz --numeric-owner --one-file-system \
	--no-acls --no-xattrs \
        -f /backups-wd/backups/netbsd`uname -r`-rootfs-02-source-ready-`date '+%Y%m%d-%H%M'`.tar.gz -C / .
exit 0

  • unmount backup partition

To restore:

  • boot from NetBSD installation CD, select Utilities -> Shell
  • setup all partitions using disk and disklabel
  • when running newfs be sure to specify proper version: 
    # for FFSv2
newfs -O 2 /dev/rwd1a
    For FFSv1 just omit -O 2 - it is default for NetBSD 9.3
  • mount both backup partition and target partition: 
    # mount backup partition read-only
mount -r /dev/wd1e /mnt
# mount formatted root filesystem
mount /dev/wd1a /mnt2
  • unpack tar with p (preserve file permissions) 
    tar xpvf /mnt/my_backup.tar.gz -C /mnt2

Restoring boot records:

  • valid for MBR and BIOS mode only!
  • to restore MBR boot record I run: 
    fdisk -c /usr/mdec/mbr /dev/rwd1d
  • to restore boot record on root filesystem (wd1a) you have to: 
    # expecting that you have mounted rootfs on /mnt2
# cp below should not be needed (should be part of .tar.gz backup)
cp /usr/mdec/boot /mnt2/boot
# NOW important:
# 1. for FFSv1:
installboot -v /dev/rwd1a /usr/mdec/bootxx_ffsv1
# 2. for FFSv2:
installboot -v /dev/rwd1a /usr/mdec/bootxx_ffsv2
  • double check that you use right ffsv1 of ffsv2 boot block - otherwise system will fail to boot with just confusing error!

Building or debugging packages from sources

Example - trying to find which URLs are fetched by pkgin

NOTE: Found much easier way - passing -V BEFORE command name:

pkgin -fV up

But using GDB never hurts...

First we have to build package pkgin with debug symbols:

cd /usr/pkgsrc/pkgtools/pkgin
make print-build-depends-list
doas pkgin in ... pass packages listed in above command to avoid building everything ...
env CFLAGS=-g LDFLAGS=-g INSTALL_UNSTRIPPED=yes make clean build

Now trying as root:

gdb --args work/pkgin-23.8.1/pkgin up -f
(gdb) break main
(gdb) run
(gdb) break fetchXGet
(gdb) c
(gdb) bt
#0  0x000000000801ad74 in fetchXGet ()
#1  0x000000000800a0bf in sum_open (
    str_url=0x7f7fff5e6710 "https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/pkg_summary.xz", 
    db_mtime=0x7f7fff5e6b90) at download.c:49
#2  0x00000000080122d9 in fetch_summary (
    cur_repo=0x7d4030647190 "https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All", 
    repo_mtime=0x7f7fff5e6b90) at summary.c:133
#3  0x0000000008013f93 in update_remotedb (verbose=1) at summary.c:788
#4  0x000000000801411c in update_db (which=1, verbose=1) at summary.c:828
#5  0x000000000800cabc in main (argc=2, argv=0x7f7fff5e6c80) at main.c:222

Please note that fetchXGet() has no visible parameters, because it was not build with debug symbols (it is library -lfetch in dependent package net/libfetch)...

There were following URLs:

str_url=0x7f7fff5e6710 "https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/pkg_summary.xz"
str_url=0x7f7fff5e6710 "https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/pkg_summary.bz2"
str_url=0x7f7fff5e6710 "https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/pkg_summary.gz"

And yes, it is out of date:

Name	Last modified	Size
vim-9.1.0698.tgz	28-Aug-2024 17:15	1436kB
pkg_summary.gz	27-Aug-2024 17:03	6171kB

Details on (but shown .pkgcache):

Resources

How to install packages (both binary and source):

Copyright © Henryk Paluch. All rights reserved.

Creative Commons License This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

Add a custom sidebar
Clone this wiki locally